Suggestion to utilize RocksDB checkpoint in archival node to quickly obtain snapshot at any historical block height

[Hecate2]

Summary or problem description
#3639 took the first step for archival nodes. Maybe we can make some enhancements in RocksDBStore plugin and ISnapshot GetSnapshot(); interfaces to retrieve a historical snapshot quickly, at very little additional costs.

Do you have any solution you want to propose?
RocksDB checkpoint offers a historical state of the database immediately, at a small cost of storage. The RocksDBStore plugin should decide (based on config) whether it is needed to generate a new checkpoint, when Blockchain.Committed is invoked.
The ISnapshot GetSnapshot(); in interface IStore can have an additional parameter with default value, to request for a historical block. I think a uint height=uint.MaxValue can be good enough to get the latest snapshot as default. If historical snapshot is not supported by a storage provider, and the height is not uint.MaxValue, the storage can simply throw exceptions.
These capabilities should enable us to quickly retrieve and debug historical blocks, and historical transactions.

Where in the software does this update applies to?

• Plugins
• Ledger
• SDK

[nan01ab]

How long will a checkpoint be maintained?
If checkpoints are saved permanently, there will be a large storage overhead

[Jim8y]

How long will a checkpoint be maintained?
If checkpoints are saved permanently, there will be a large storage overhead

for debugging purpose, will be there permanently, but its optional.

[Hecate2]

How long will a checkpoint be maintained? If checkpoints are saved permanently, there will be a large storage overhead

I do not have a determinate idea now. Personally I am thinking of leaving only -log(1-current_height/total_height) checkpoints for very ancient history. This may require complex controls in deleting and merging checkpoints. Besides, ancient checkpoints with very long computation time to a certain later block may be meaningless for users waiting online.
A checkpoint for every 1k or 10k blocks can be nice for now, but I think it does make a burden for the future. It really depends on the user's wishes. Better to leave it in configs.

[roman-khimov]

That'd be DB-specific, but maybe somewhat more efficient in terms of speed. We do historic things via MPT (https://github.com/nspcc-dev/neo-go/blob/master/docs/rpc.md#historic-calls).

[Jim8y]

That'd be DB-specific, but maybe somewhat more efficient in terms of speed. We do historic things via MPT (https://github.com/nspcc-dev/neo-go/blob/master/docs/rpc.md#historic-calls).

MPT is way larger, but verifiable...

[roman-khimov]

We don't know what's gonna happen to RocksDB if one is try snapshotting checkpointing every block. Can it handle 6M snapshotscheckpoints? What's gonna be the size? Mystery to me.

[nan01ab]

It’s a great idea for debug

[Hecate2]

virtual void CompactRange(const Slice* begin, const Slice* end)

db->CompactRange(NULL, NULL); may be needed for nodes to compress the data size when not busy. This applies to not only RocksDB but also LevelDB

[shargon]

virtual void CompactRange(const Slice* begin, const Slice* end)

db->CompactRange(NULL, NULL); may be needed for nodes to compress the data size when not busy. This applies to not only RocksDB but also LevelDB

If nothing was removed, this wont work