> ## Documentation Index
> Fetch the complete documentation index at: https://agno-v2-docs-scavio-google-v2.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# vLLM Embedder

> Generate embeddings with vLLM, either in-process or against a vLLM server.

`VLLMEmbedder` runs in two modes. Local mode loads the model in-process with no server or API key. Remote mode connects to a running vLLM server through its OpenAI-compatible API, selected by setting `base_url`.

```python vllm_embedder.py theme={null}
from agno.knowledge.embedder.vllm import VLLMEmbedder

# Local mode: vLLM loads the model in-process
embedder = VLLMEmbedder(
    id="sentence-transformers/all-MiniLM-L6-v2",
    dimensions=384,
    enforce_eager=True,
    vllm_kwargs={
        "disable_sliding_window": True,
        "max_model_len": 256,
    },
)
embedding = embedder.get_embedding("The quick brown fox jumps over the lazy dog.")

print(embedding[:5])
print(len(embedding))

# Remote mode: connect to a running vLLM server
remote_embedder = VLLMEmbedder(
    id="sentence-transformers/all-MiniLM-L6-v2",
    dimensions=384,
    base_url="http://localhost:8000/v1",
    api_key="your-api-key",  # optional, also read from VLLM_API_KEY
)
```

Local mode downloads the model from Hugging Face on first use. Larger models need matching GPU memory; `intfloat/e5-mistral-7b-instruct` (4096 dimensions) needs roughly 14GB of VRAM.

## Run the Example

<Steps>
  <Snippet file="create-venv-step.mdx" />

  <Step title="Install dependencies">
    ```bash theme={null}
    uv pip install -U agno vllm openai
    ```
  </Step>

  <Step title="Run the example">
    ```bash theme={null}
    python vllm_embedder.py
    ```
  </Step>
</Steps>

## Developer Resources

* [VLLMEmbedder reference](/reference/knowledge/embedder/vllm)
* [vLLM OpenAI-compatible server](https://docs.vllm.ai/en/latest/serving/openai_compatible_server.html)
* [Embedders overview](/knowledge/concepts/embedder/overview)