Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix reorg - Create savepoint when close to tip and at max reorg depth #4169

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

emilcondrea
Copy link
Contributor

Details

Create savepoint when close to tip and at max reorg depth

  • this is useful as previously some times the first savepoint would have been the tip/really close to tip sometimes and if a reorg happened it could not restore at all
  • if above happened it tried to infinitely restore the snapshot and it increased the index size by 1GB every few minutes (would be good to understand why did that happen)

Context

This happened while trying to index testnet4(ord 22.1), and it indexes all the way to the tip (in my case 63038), then this happens:

  • it commits to the tip and creates the savepoint at the tip.
  • Then a reorg of deep 2 happens, ord tries to rollback to oldest savepoint which is still 63038, thinks its recoverable (but its not, as the savepoint is not that old),
  • reverts to the oldest savepoint which basically does nothing,
  • and then does this in an infinite loop and increases the used disk space exponentially its like 1 more GB each 2-3 mins. (ended up filling all disk space on my mac over night)

Here are some additional logs I captured while debugging this

2025-01-07T14:23:40Z INFO  ord::index::reorg] creating savepoint at height 63038
fetched block 00000000566eb6fe93c9c72553e97aa64d37667df74b9f2de423789ae4a420fe

index hash(00000000f399a5a201c25a0fcff0d4331c0fc819da001ac199f8c787a321f6df) vs bitcoind_hash(000000009cfa3d0e34a663389d28cf721674211ffeda3b5b601b2f6ababb9c1f)
detect_reorg: 00000000566eb6fe93c9c72553e97aa64d37667df74b9f2de423789ae4a420fe
bitcoind height63037
index hash(00000000f399a5a201c25a0fcff0d4331c0fc819da001ac199f8c787a321f6df) vs bitcoind_hash(000000009cfa3d0e34a663389d28cf721674211ffeda3b5b601b2f6ababb9c1f)
detect_reorg1: 00000000566eb6fe93c9c72553e97aa64d37667df74b9f2de423789ae4a420fe
bitcoind height63036
index hash(0000000028db011b8cab6a5e58ebb2105015f065a37ae0cf32885c94ac5a4bf3) vs bitcoind_hash(0000000028db011b8cab6a5e58ebb2105015f065a37ae0cf32885c94ac5a4bf3)
[2025-01-07T14:23:45Z INFO  ord::index] 2 block deep reorg detected at height 63038
[2025-01-07T14:23:45Z INFO  ord::index::reorg] rolling back database after reorg of depth 2 at height 63038
existing savepoints: [1]
[2025-01-07T14:23:46Z INFO  ord::index::reorg] successfully rolled back database to height 63038
...

Tested

Indexed testnet 4 and it created the savepoint 21 blocks away from the tip

...
[2025-01-07T15:48:04Z INFO  ord::index::reorg] is_savepoint_required=true: height=63024, last_savepoint_height=57205, blocks=63045
[2025-01-07T15:48:04Z INFO  ord::index::updater] Committing at block height 63024, 0 outputs traversed, 599370 in map, 300205 cached
[2025-01-07T15:48:28Z INFO  ord::index::reorg] is_savepoint_required=true: height=63024, last_savepoint_height=57205, blocks=63045
[2025-01-07T15:48:28Z INFO  ord::index::reorg] creating savepoint at height 63024
[2025-01-07T15:48:28Z INFO  ord::index::updater] Block 63024 at 2025-01-07 15:23:21 UTC with 60 transactions…
[2025-01-07T15:48:28Z INFO  ord::index::updater] Wrote 0 sat ranges from 0 outputs in 5 ms
[2025-01-07T15:48:28Z INFO  ord::index::reorg] is_savepoint_required=false: height=63025, last_savepoint_height=63024, blocks=63045
[2025-01-07T15:48:28Z INFO  ord::index::updater] Block 63025 at 2025-01-07 14:23:19 UTC with 34 transactions…
[2025-01-07T15:48:28Z INFO  ord::index::updater] Wrote 0 sat ranges from 0 outputs in 1 ms
[2025-01-07T15:48:28Z INFO  ord::index::reorg] is_savepoint_required=false: height=63026, last_savepoint_height=63024, blocks=63045
[2025-01-07T15:48:28Z INFO  ord::index::updater] Block 63026 at 2025-01-07 14:43:20 UTC with 3 transactions…
[2025-01-07T15:48:28Z INFO  ord::index::updater] Wrote 0 sat ranges from 0 outputs in 0 ms
[2025-01-07T15:48:28Z INFO  ord::index::reorg] is_savepoint_required=false: height=63027, last_savepoint_height=63024, blocks=63045
[2025-01-07T15:48:28Z INFO  ord::index::updater] Block 63027 at 2025-01-07 15:03:21 UTC with 1 transactions…
[2025-01-07T15:48:28Z INFO  ord::index::updater] Wrote 0 sat ranges from 0 outputs in 0 ms
[2025-01-07T15:48:28Z INFO  ord::index::reorg] is_savepoint_required=false: height=63028, last_savepoint_height=63024, blocks=63045
[2025-01-07T15:48:28Z INFO  ord::index::updater] Block 63028 at 2025-01-07 15:23:22 UTC with 1 transactions…
[2025-01-07T15:48:28Z INFO  ord::index::updater] Wrote 0 sat ranges from 0 outputs in 0 ms
[

* this is useful as previously some times the first savepoint would
have been the tip/really close to tip sometimes and if a reorg happened
it could not restore at all and infinitely tried it
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant