Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sync Activity generating high data volume - log rotation is missing #40

Open
ICTRV opened this issue Apr 14, 2023 · 4 comments
Open

Sync Activity generating high data volume - log rotation is missing #40

ICTRV opened this issue Apr 14, 2023 · 4 comments

Comments

@ICTRV
Copy link
Collaborator

ICTRV commented Apr 14, 2023

The Sync Activity is creating a huge number of records.
Two days of testing with a few calendars by tow users primarily, has created more than 20K activity records.

We need a possibility to purge such records every week or make it as a setting parameter. Else we will possibly end up with a very heavy database.

Screenshot from 2023-04-14 10-29-33

@iscale-odoo
Copy link
Contributor

Please tell us how do you wish to manage the rotation

@bvanjeelharia
Copy link

You need to provide a configuration option under configuration->settings wherein users can specify for how long the logs should be kept(in days).Keep the default as 7 days. Then the logs older than the number of days specified should be cleared from the system

iscale-odoo pushed a commit that referenced this issue Jun 2, 2023
@astirpe
Copy link

astirpe commented Jun 7, 2023

Adding log rotation is good but is not enough.

The logging mechanism should be fast and light, since it's invoked very often. But looking at the code of nextcloud_odoo_sync, the logging seems more complex than what is really necessary.

For example every time log_event() is invoked, some children records of model nc.sync.log.line are created, having a record of nc.sync.log as parent. So every time a log entry is written, multiple records are created in the database. Try to limit the logging records in database to one record per each log, for example by merging model nc.sync.log.line into nc.sync.log

When log_event() is invoked, other methods are also executed, like for example check_and_log_users(). This method does not only create multiple log lines but also it invokes search_read() and write() on model nc.sync.user, resulting in expensive usage of resources. Try to limit the logging mechanism to just doing the logging, without adding extra workload.

@astirpe
Copy link

astirpe commented Jun 7, 2023

In addition to my previous post, I would also say to remove nc.sync.error model and the 2 values defined in nc_sync_error_data.xml. Field error_code_id of type Many2one can be replaced with the 4 corresponding fields, eg. like: error_code, error_description, error_type, error_severity.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants