How to store a multimap?

[JamesHinshelwood]

Hi, thanks for the cool library :)

I'm currently evaluating redb and fjall. One of the features we use in redb is multimap tables which allow you to store a list of values with each key.

At a first glance, I don't see an equivalent in fjall. Are there any plans to implement this or other ways we could store the data that would fit with fjall's model?

[marvin-j97]

It definitely depends on your data.

Naively, you could just encode your array into a single KV, and update that using update_fetch.

Alternatively, you could use a common prefix:

key1#value1
key1#value2
key1#value3

This probably is the way to go if there can be many values, and/or you want to only update, delete or scan some of the values.

For example, a secondary index in a RDBMS is logically a multi map (indexed column -> Rows), and is implemented using such common prefix.

---

To me, a MultiMap API kind of hides of what's actually going on at the storage level (I'm guessing redb uses something like the 1st approach), so I don't think it's really necessary.

[enomado]

I have related question. Do all partitions share the keys? I mean, can I immitate columnar on that principle? I have one key(timestamp) and a lot of columns,

Actually, i try to find some database whitch would be simple embedd clickhouse. I have some RAM storage like

RowColumn{
     keys: Vec<Timestamp>,
     // [[aaaaaaaaaaa][bbbbbbbbb][cccccccccc] - same lenght
     columns: Vec<Vec<f64>>,
}

[marvin-j97]

Each partition is isolated, one partition is typically one table/index/..., but you could model everything inside a single partition (single table design)

You could store your columns using keys such as:

col1#<timestamp1>
col1#<timestamp2>
col2#<timestamp4>
...

with timestamps being big endian encoded integers.