Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add CI #1

Draft
wants to merge 2 commits into
base: main
Choose a base branch
from
Draft

feat: add CI #1

wants to merge 2 commits into from

Conversation

randomicecube
Copy link
Collaborator

When finished:
Closes chains-project/dirty-waters#9

@randomicecube randomicecube added the enhancement New feature or request label Dec 21, 2024
@randomicecube randomicecube self-assigned this Dec 21, 2024
@monperrus
Copy link

cool!

@randomicecube
Copy link
Collaborator Author

An issue I have thought of since the hiatus on here:

  • Caching: this workflow needs to be fast(er), and dirty-waters relies on caching when ran locally; I don't know how to make use of it if it's a CI, will investigate

@algomaster99
Copy link
Member

What do you need to cache? If it is dependencies, you could use - https://github.com/actions/cache.

@randomicecube
Copy link
Collaborator Author

randomicecube commented Jan 16, 2025

What do you need to cache?

Currently, only information coming from GitHub is cached (but I'd like to extend to further purposes). Furthermore, I don't think we're making fully good use of it yet, and I think it would be crucial to do so before we roll out, since otherwise it's highly likely (IMO) that CI would take too long to run regardless, and developers need fast feedback.
I'll come back here once I improve it.

Also, thanks for the link!

@algomaster99
Copy link
Member

Currently, only information coming from GitHub is cached (but I'd like to extend to further purposes).

Could you please explain this? I don't know about it because I have not followed the project that much 😅

@randomicecube
Copy link
Collaborator Author

Could you please explain this? I don't know about it because I have not followed the project that much 😅

Essentially, when we run dirty-waters now, we cache some information gotten through GitHub, an example can be seen here. I'm assuming this was done just for GitHub initially because it effectively is a major bottleneck, concerning rate limits and so on.

However, implementing caching in other areas is crucial to actually make the tool usable in CI: as an example, package analysis in my computer for spoon never took less than 15 minutes, now imagine waiting for the CI to run for at least 15 minutes every time!
I've been doing some experiments locally -- namely, caching package analysis information as well -- and running the tool for spoon now takes 5s 🕺🏽 The core of it is, for every analyze_package_data call, checking at first whether for this dep and its version if we've analyzed it, and if so, we use the cached information.

I'm going to investigate other places where caching makes sense, and create a PR with all of that

@algomaster99
Copy link
Member

algomaster99 commented Jan 17, 2025 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

add support for checking transparency in CI
3 participants