Non Cosine-Sim Similarity functions for Sentence Transformer models are broken in v2.0.0

[orionw]

I tested using model facebook/contriever-msmarco (which should use dot) and dataset InstructIR (but TBH any dataset works).

If you stick a breakpoint in the sim function after this line you can check, it is always cos_sim. This might be because the SentenceTransformer models are only defined via ModelMeta and the default sim function is not stored in ST (AFAIK).

[Samoed]

Default similarity function is stored in sentence_transformer_config.json. Example https://huggingface.co/infgrad/jasper_en_vision_language_v1/blob/c6991f5a54805d15ea1af42f9db2eee4cca96edc/config_sentence_transformers.json#L12

[orionw]

Ah, awesome @Samoed! But this is probably a newer update for ST then. I don't see that in contriever-msmarco's files. So then I guess this issue is for older models where that isn't defined and is only defined in the model_meta.

[Samoed]

We need to add the model meta for this model with a wrapper that uses the dot function. From what I found when adding a similar function, all models used cosine similarity, but I might be wrong.

[orionw]

The model meta says dot - I don't think I know the wrapper you are talking about though.

It does seem to be the only dot model in the file though!

[Samoed]

I'll add wrapper for this

[sam-hey]

@orionw ModelMeta.similarity_fn_name is currently ignored and does not affect which similarity function is used. However, #1749 has temporarily addressed this issue.

It seems that #1759 might provide a more permanent fix for this behavior.

[Samoed]

I think this can be closed for now, because issue was fixed for now