-
Notifications
You must be signed in to change notification settings - Fork 0
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: add CI #1
base: main
Are you sure you want to change the base?
feat: add CI #1
Conversation
cool! |
An issue I have thought of since the hiatus on here:
|
What do you need to cache? If it is dependencies, you could use - https://github.com/actions/cache. |
Currently, only information coming from GitHub is cached (but I'd like to extend to further purposes). Furthermore, I don't think we're making fully good use of it yet, and I think it would be crucial to do so before we roll out, since otherwise it's highly likely (IMO) that CI would take too long to run regardless, and developers need fast feedback. Also, thanks for the link! |
Could you please explain this? I don't know about it because I have not followed the project that much 😅 |
Essentially, when we run However, implementing caching in other areas is crucial to actually make the tool usable in CI: as an example, package analysis in my computer for spoon never took less than 15 minutes, now imagine waiting for the CI to run for at least 15 minutes every time! I'm going to investigate other places where caching makes sense, and create a PR with all of that |
namely, caching package analysis information as well -- and running the tool for spoon now takes 5s
That's cool 🎉
I don't know if you could cache SQL database with GitHub actions, but good that you are exploring.
…________________________________
From: Diogo Gaspar ***@***.***>
Sent: Friday, January 17, 2025 10:55:41 AM
To: chains-project/dirty-waters-action
Cc: Aman Sharma; Comment
Subject: Re: [chains-project/dirty-waters-action] feat: add CI (PR #1)
Could you please explain this? I don't know about it because I have not followed the project that much 😅
Essentially, when we run dirty-waters now, we cache some information gotten through GitHub, an example can be seen here<https://github.com/chains-project/dirty-waters/blob/42a655802123860c9309a6242ce882fc3316ef53/tool/github_repo.py#L124>. I'm assuming this was done just for GitHub initially because it effectively is a major bottleneck, concerning rate limits and so on.
However, implementing caching in other areas is crucial to actually make the tool usable in CI: as an example, package analysis in my computer for spoon never took less than 15 minutes, now imagine waiting for the CI to run for at least 15 minutes every time!
I've been doing some experiments locally -- namely, caching package analysis information as well -- and running the tool for spoon now takes 5s 🕺🏽 The core of it is, for every analyze_package_data<https://github.com/chains-project/dirty-waters/blob/42a655802123860c9309a6242ce882fc3316ef53/tool/static_analysis.py#L491> call, checking at first whether for this dep and its version if we've analyzed it, and if so, we use the cached information.
I'm going to investigate other places where caching makes sense, and create a PR with all of that
—
Reply to this email directly, view it on GitHub<#1 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AIMPTOKVI73KZQLE5F3I3HD2LDHR3AVCNFSM6AAAAABUAQZREKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKOJXHA2TGMZRG4>.
You are receiving this because you commented.Message ID: ***@***.***>
|
When finished:
Closes chains-project/dirty-waters#9