Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Handle PartialECChangeUnifier Changes On-Disk #7553

Open
mattbjordan opened this issue Jan 15, 2025 · 0 comments
Open

Handle PartialECChangeUnifier Changes On-Disk #7553

mattbjordan opened this issue Jan 15, 2025 · 0 comments
Assignees
Labels
ecdb ECDb and ECSQL related issues enhancement New feature or request

Comments

@mattbjordan
Copy link
Contributor

Is your feature request related to a problem? Please describe.
For really large iModels or really large changesets, the current PartialECChangeUnifier cannot handle doing it's job while storing all of the elements in memory.

Describe the solution you'd like
Instead, in these extreme cases, the PartialECChangeUnifier can store the changes on disk within the iModel in custom sqlite tables. This will make the processing significantly slower, but it will allow the process to actually finish.

Describe alternatives you've considered
I have already created a proof of concept that this solution works here: https://github.com/iTwin/itwinjs-core/tree/mattbjordan/disk-based-unifier-poc.

Additional context
A few things still need to be figured out:

  • How do we determine when we should be storing the changes on-disk versus in-memory. In-memory is significantly faster and totally fine for the large majority of cases, so that should be the preference. Perhaps the user will need to know beforehand to set a boolean to instead process slower, but on-disk?
@mattbjordan mattbjordan added the enhancement New feature or request label Jan 15, 2025
@khanaffan khanaffan added the ecdb ECDb and ECSQL related issues label Jan 15, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ecdb ECDb and ECSQL related issues enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants