Changing/Specifying the path to all-MiniLM-L6-v2

[jvhuang1786]

Hi I just started exploring this library today. Is it possible to change the path to the sentence transformer or specify the path or change it to another sentence transformer? I'm unable to connect to the internet through my environment so I have to download the models locally and upload them to ADLS and load them from there.

[drazvan]

Hi @jvhuang1786 !

Currently it only possible to change the name of the model (https://github.com/NVIDIA/NeMo-Guardrails/blob/main/nemoguardrails/actions/llm/generation.py#L73). This is not documented so we'll fix this.
Adding support for a cache_path parameter is not very difficult. If you have a bit of time to contribute, I can point you in the right direction (needs to be added here: https://github.com/NVIDIA/NeMo-Guardrails/blob/main/nemoguardrails/kb/basic.py#L31) and we need to make sure we propagate it correctly from the config:

models:
   ...
   - type: embedding
     engine: SentenceTransformer
     model: all-MiniLM-L6-v2
     parameters: 
       cache_path: ...

[Haxeebraja]

Facing the same issue with no internet access. Has this been implemented?
Can I pass local folder path in cache_path for an embedding model?

[Haxeebraja]

Hi @jvhuang1786 !

Currently it only possible to change the name of the model (https://github.com/NVIDIA/NeMo-Guardrails/blob/main/nemoguardrails/actions/llm/generation.py#L73). This is not documented so we'll fix this. Adding support for a cache_path parameter is not very difficult. If you have a bit of time to contribute, I can point you in the right direction (needs to be added here: https://github.com/NVIDIA/NeMo-Guardrails/blob/main/nemoguardrails/kb/basic.py#L31) and we need to make sure we propagate it correctly from the config:

models:
   ...
   - type: embedding
     engine: SentenceTransformer
     model: all-MiniLM-L6-v2
     parameters: 
       cache_path: ...

The link shared is broken. Can you please share if there is any alternative to use this package offline?

[drazvan]

Hi @Haxeebraja !

The easiest way would be to add support for using the cache_dir parameter using an environment variable, here: https://github.com/NVIDIA/NeMo-Guardrails/blob/develop/nemoguardrails/embeddings/basic.py#L120

class SentenceTransformerEmbeddingModel(EmbeddingModel):
    """Embedding model using sentence-transformers."""

    def __init__(self, embedding_model: str):
        from sentence_transformers import SentenceTransformer

        device = "cuda" if cuda.is_available() else "cpu"
        cache_folder = os.environ.get("SENTENCE_TRANSFORMERS_CACHE_FOLDER")
        self.model = SentenceTransformer(embedding_model, device=device, cache_folder=cache_folder)

If you have a bit of time to test this and contribute back a PR that would be great.

Razvan

[Haxeebraja]

One of the ways I tried that works is to set local path for embeddings model in config file:

models:

• type: embeddings
  engine: SentenceTransformers
  model: /workspace/mounted/bge-large-en-v1.5