Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Issue 4815 - RFE - Add Replication Log Analysis Tool with CLI Support #6466

Open
wants to merge 5 commits into
base: main
Choose a base branch
from

Conversation

droideck
Copy link
Member

Description: Add a new ReplicationLogAnalyzer class and supporting infrastructure
to analyze replication performance across multiple servers. The tool parses
replication logs, tracks CSNs, calculates lag times, and generates comprehensive
reports in interactive HTML and CSV formats.

Based on: https://github.com/droideck/ansible-ds389-repl-monitoring

Fixes: #6465

Reviewed by: ?

@droideck droideck marked this pull request as draft December 31, 2024 03:41
@droideck droideck added the work in progress Work in Progress - can be reviewed, but not ready for merge. label Dec 31, 2024
Copy link
Contributor

@tbordaz tbordaz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very nice. Few comments

dirsrvtests/tests/suites/replication/acceptance_test.py Outdated Show resolved Hide resolved
dirsrvtests/tests/suites/replication/acceptance_test.py Outdated Show resolved Hide resolved
src/lib389/lib389/repltools.py Outdated Show resolved Hide resolved
@droideck droideck changed the title Issue 4815 - RFE - Add Replication Log Analysis Tool with CLI/UI Support Issue 4815 - RFE - Add Replication Log Analysis Tool with CLI Support Jan 20, 2025
Description: Add a new ReplicationLogAnalyzer class and supporting infrastructure
to analyze replication performance across multiple servers. The tool parses
replication logs, tracks CSNs, calculates lag times, and generates comprehensive
reports in interactive HTML and CSV formats.

Based on: https://github.com/droideck/ansible-ds389-repl-monitoring

Fixes: 389ds#6465

Reviewed by: ?
@droideck
Copy link
Member Author

Okay, I think the CLI and lib389 parts are ready for review.
I updated the design here: - see docs/389ds/design/replication-lag-report-design.md file

I'll do the WebUI part in a separate PR to make the review easier.

P.S. if my numpy, plotly, mathplotlib usage is approved, I'll update our ci-image, so the workflow files' changes will be removed from this PR before the merge.

@droideck droideck removed the work in progress Work in Progress - can be reviewed, but not ready for merge. label Jan 23, 2025
@droideck droideck marked this pull request as ready for review January 23, 2025 05:07
@@ -40,6 +40,9 @@ jobs:
id: set-matrix
run: echo "matrix=$(python3 .github/scripts/generate_matrix.py ${{ github.event.inputs.pytest_tests }})" >>$GITHUB_OUTPUT

- name: Install python3 additional packages
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this step really needed ?
maybe we should rather rebuild the docker CI test image with the build prerequisite
@vashirov What do you think ?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, we should add this to our base test image. I think Simon added it here to make the CI happy without changing the image first.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, that's correct. :)

src/lib389/lib389/cli_conf/replication.py Outdated Show resolved Hide resolved
src/lib389/lib389/cli_conf/replication.py Outdated Show resolved Hide resolved
Copy link
Contributor

@progier389 progier389 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

rpm/389-ds-base.spec.in Outdated Show resolved Hide resolved
Co-authored-by: Viktor Ashirov <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add a replication log analysis tool for performance monitoring
4 participants