Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tests: Reduce AMQP & PCNTL Tests flakiness #3043

Draft
wants to merge 5 commits into
base: master
Choose a base branch
from

Conversation

PROFeNoM
Copy link
Contributor

@PROFeNoM PROFeNoM commented Jan 17, 2025

Description

I was getting a bit bored of these tests failing here and there in CI jobs... I believe this quick workaround could help us trust the CI a bit more.

The workaround simply consist in retrying the tests, up to five times.

This only affects integration_snapshots-test_integrations

Reviewer checklist

  • Test coverage seems ok.
  • Appropriate labels assigned.

@PROFeNoM PROFeNoM self-assigned this Jan 17, 2025
@PROFeNoM PROFeNoM marked this pull request as ready for review January 17, 2025 10:51
@PROFeNoM PROFeNoM requested a review from a team as a code owner January 17, 2025 10:51
@codecov-commenter
Copy link

codecov-commenter commented Jan 17, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 74.78%. Comparing base (6b3218e) to head (2948eb7).

Additional details and impacted files

Impacted file tree graph

@@             Coverage Diff              @@
##             master    #3043      +/-   ##
============================================
- Coverage     74.80%   74.78%   -0.02%     
  Complexity     2784     2784              
============================================
  Files           112      112              
  Lines         11020    11020              
============================================
- Hits           8243     8241       -2     
- Misses         2777     2779       +2     
Flag Coverage Δ
tracer-php 74.78% <ø> (-0.02%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

see 1 file with indirect coverage changes


Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6b3218e...2948eb7. Read the comment docs.

@PROFeNoM PROFeNoM changed the title Reduce AMQP & PCNTL Tests flakiness tests: Reduce AMQP & PCNTL Tests flakiness Jan 17, 2025
Copy link
Collaborator

@bwoebi bwoebi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good idea, thank you!

@PROFeNoM PROFeNoM force-pushed the alex/tests/pcntl-flakiness branch from 055b40c to 2948eb7 Compare January 17, 2025 14:48
@iamluc
Copy link
Contributor

iamluc commented Jan 17, 2025

LGTM, but to make it a bit less intrusive, you could implement it with an annotation on the test, for example using https://gist.github.com/makasim/989fcaa6da8ff579f7914d973e68280c or https://github.com/bshaffer/phpunit-retry-annotations

@PROFeNoM
Copy link
Contributor Author

PROFeNoM commented Jan 17, 2025

Hmmm, it's still failing (example) despite retries... I'll put this back to draft and may investigate further later on.

PS: Note that this still helps with flakiness (amqp example)

@PROFeNoM PROFeNoM marked this pull request as draft January 17, 2025 14:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants