Skip to content

Commit

Permalink
filestore docs: BlockList and Rebase logic (#2833)
Browse files Browse the repository at this point in the history
  • Loading branch information
qkrorlqr authored Jan 10, 2025
1 parent 5b63942 commit 2799a2e
Show file tree
Hide file tree
Showing 4 changed files with 2,998 additions and 0 deletions.
7 changes: 7 additions & 0 deletions doc/filestore/design/storage/tablet/block_list.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
# Block List format and Rebase logic

[MixedBlocks](data_index.md#mixedblocks) stores block metadata for each blob in the form of [BlockLists](https://github.com/ydb-platform/nbs/blob/main/cloud/filestore/libs/storage/tablet/model/block_list.h). BlockLists have complex binary format and are encoded by [this code](https://github.com/ydb-platform/nbs/blob/main/cloud/filestore/libs/storage/tablet/model/block_list_encode.cpp). CommitIds of the blocks described by BlockLists can be changed by the [rebase logic](https://github.com/ydb-platform/nbs/blob/main/cloud/filestore/libs/storage/tablet/rebase_logic.cpp) to improve metadata grouping and decrease BlockList size.

Deletions in BlockList code simply mean block MaxCommitIds and are not directly related to DeletionMarkers.

![mixed_blocks_svg](../../../excalidraw/mixed_blocks_rebase_logic.svg)
1 change: 1 addition & 0 deletions doc/filestore/design/storage/tablet/data_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ It is the main layer intended for long-term data storage.
* Each blob may contain blocks belonging to multiple different inodes.
* Each range is supposed to contain up to 4MiB of data (sometimes slightly more due to hash collisions).
* Each write generates DeletionMarkers which are later applied to the MixedBlocks table via the [Cleanup operation](https://github.com/ydb-platform/nbs/blob/main/cloud/filestore/libs/storage/tablet/tablet_actor_cleanup.cpp).
* Block metadata for each blob is stored in a [BlockList](block_list.md).

![mixed_blocks_svg](../../../excalidraw/mixed_blocks.svg)

Expand Down
Loading

0 comments on commit 2799a2e

Please sign in to comment.