Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor stats, removing aggregate fields from project, projectlocale & locale #3536

Merged
merged 4 commits into from
Jan 23, 2025

Conversation

eemeli
Copy link
Member

@eemeli eemeli commented Jan 17, 2025

Based on my unscientific local testing, the rendering speed benefits from eagerly populating the aggregated stats into Project, ProjectLocale & Locale don't exist, compared to collating the data on the fly from TranslatedResource objects. Therefore, we should not be doing this complex extra work. This significantly simplifies the logic around stats updates, and makes the stats much easier to reason about.

The AggregatedStats class no longer extends Model, and it's mostly a crutch for reducing the risks in this change; hence its move from base/models/ to base/. With it, fields like total_strings and strings_with_errors will continue to work for the Django models from which the corresponding DB fields were removed.

@eemeli eemeli requested a review from mathjazz January 17, 2025 21:34
@codecov-commenter
Copy link

codecov-commenter commented Jan 17, 2025

Codecov Report

Attention: Patch coverage is 90.49587% with 23 lines in your changes missing coverage. Please review.

Project coverage is 78.89%. Comparing base (729cc71) to head (3c5cd20).
Report is 2 commits behind head on main.

Additional details and impacted files

@eemeli
Copy link
Member Author

eemeli commented Jan 22, 2025

It's likely (but not certain) that this will also provide a solution for #3514.

@mathjazz
Copy link
Collaborator

The branch is deployed to stage for testing.

@flodolo
Copy link
Collaborator

flodolo commented Jan 22, 2025

The branch is deployed to stage for testing.

Stats for Italian are completely off?
https://mozilla-pontoon-staging.herokuapp.com/it/

@mathjazz
Copy link
Collaborator

https://mozilla-pontoon-staging.herokuapp.com/sl/ also shows 495 more Missing strings in the heading than the sum of Missing ProjectLocale strings in the table.

Are we including private projects?

@eemeli
Copy link
Member Author

eemeli commented Jan 22, 2025

Are we including private projects?

No, but we aren't filtering out disabled projects in some cases where we should. I'll apply a fix later today.

pontoon/base/models/translated_resource.py Outdated Show resolved Hide resolved
pontoon/base/views.py Outdated Show resolved Hide resolved
@eemeli eemeli requested a review from mathjazz January 22, 2025 22:27
Copy link
Collaborator

@mathjazz mathjazz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good!

We tested on stage and not only do stats look fine, but we also didn't notice performance regressions.

We should check New Relic after deploying to prod and monitor any performance impact. We should also verify the Insights are still collected as expected.

@mathjazz mathjazz merged commit 948e6cf into mozilla:main Jan 23, 2025
7 checks passed
@eemeli eemeli deleted the refactor-stats branch January 23, 2025 11:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants