Remove LiteLLM embedding provider, default to Sentence Transformers

Signed-off-by: Chaitany Patel <patelchaitany93@gmail.com>
franciscojavierarceo · Jul 22, 2026 · Mar 17, 2026 · Mar 24, 2026 · May 19, 2026 · Jun 23, 2026
commit 7629ab51d26a35b2d3d2490eaa1ef36b2d363709
@@ -50,26 +50,16 @@ This is useful for AI agents and LLM tool-calling workflows where the client can
 
 ### Requirements
 
-- An `embedding_model` section in your `feature_store.yaml`. Two providers are supported:
-
-  **LiteLLM** (default) — API-backed models via [LiteLLM](https://docs.litellm.ai/) (OpenAI, Azure, Ollama, Cohere, etc.):
+- An `embedding_model` section in your `feature_store.yaml`. Feast uses [Sentence Transformers](https://www.sbert.net/) for local embedding — no external API key required (`pip install sentence-transformers`):
 
   ```yaml
   embedding_model:
-    provider: litellm          # default; can be omitted
-    model: text-embedding-3-small
-    api_key: ${OPENAI_API_KEY}
-    # api_base, api_version, dimensions are optional
-  ```
-
-  **Sentence Transformers** — local inference, no API key required (`pip install sentence-transformers`):
-
-  ```yaml
-  embedding_model:
-    provider: sentence_transformers
+    provider: sentence_transformers   # default; can be omitted
     model: all-MiniLM-L6-v2
   ```
 
+  Custom embedding providers can be plugged in by implementing the `EmbeddingProvider` protocol and passing an instance to `FeatureStore`.
+
 - A feature view with `vector_index=True` on a vector field, materialized to an online store that supports vector search.
 - For metadata filtering with numeric or boolean comparisons, set `enable_openai_compatible_store: true` on your online store config and run `feast apply` to update the schema.
 

@@ -553,11 +553,11 @@ Add an `embedding_model` section to your `feature_store.yaml`:
 
 ```yaml
 embedding_model:
-  model: text-embedding-3-small
-  api_key: ${OPENAI_API_KEY}
+  provider: sentence_transformers   # default; can be omitted
+  model: all-MiniLM-L6-v2
 ```
 
-Two providers are supported: **LiteLLM** (default) for API-backed models (OpenAI, Ollama, Azure, Cohere, etc.) and **Sentence Transformers** (`provider: sentence_transformers`) for local inference without an API key. See [Alpha Vector Database](../alpha-vector-database.md#openai-compatible-vector-store-search) for full configuration and filter details.
+Feast uses **Sentence Transformers** (default) for local embedding inference — no external API key required. See [Alpha Vector Database](../alpha-vector-database.md#openai-compatible-vector-store-search) for full configuration and filter details.
 
 ### List vector stores
 

@@ -338,7 +338,7 @@ export OPENAI_BASE_URL="http://localhost:11434/v1"
 export LLM_MODEL="llama3.1:8b"
 ./run_demo.sh
 
-# Any OpenAI-compatible provider (Azure, vLLM, LiteLLM, etc.)
+# Any OpenAI-compatible provider (Azure, vLLM, etc.)
 export OPENAI_API_KEY="your-key"  # pragma: allowlist secret
 export OPENAI_BASE_URL="https://your-endpoint/v1"
 export LLM_MODEL="your-model"

@@ -1,7 +1,7 @@
 ---
-title: "Making Feast Speak OpenAI: Vector Search Without the Glue Code"
+title: "Using Feast's OpenAI Compatible Search API"
 description: "Feast now exposes an OpenAI-compatible vector store search endpoint. Send a plain text query, get results back in the standard OpenAI format. No client-side embeddings required."
-date: 2026-05-15
+date: 2026-07-07
 authors: ["Chaitanya Patel", "Nikhil Kathole"]
 ---
 
@@ -80,7 +80,7 @@ No embedding SDK. No raw vectors. The request and response match OpenAI's format
 
 When Feast receives this request, it:
 
-1. Embeds the query server-side using the model configured in `feature_store.yaml` (via [LiteLLM](https://docs.litellm.ai/), which supports OpenAI, Ollama, Azure, Cohere, HuggingFace, and 100+ other providers).
+1. Embeds the query server-side using the model configured in `feature_store.yaml` (via [Sentence Transformers](https://www.sbert.net/) for local inference — no external API key required).
 2. Runs vector similarity search against the feature view's online store (Postgres/pgvector, Milvus, Elasticsearch, SQLite, or whatever backend you've configured).
 3. Applies filters if you provided any, using string equality, numeric comparisons, or compound AND/OR conditions in the OpenAI filter format.
 4. Returns results in OpenAI's `vector_store.search_results.page` format.
@@ -110,35 +110,20 @@ online_store:
   enable_openai_compatible_store: true
 
 embedding_model:
-  model: text-embedding-3-small
-  api_key: ${OPENAI_API_KEY}
+  provider: sentence_transformers   # default; can be omitted
+  model: all-MiniLM-L6-v2
 ```
 
-Feast uses [LiteLLM](https://docs.litellm.ai/) under the hood, so any provider works:
+Feast uses [Sentence Transformers](https://www.sbert.net/) for embedding, so everything runs locally — no external API key required. You can use any HuggingFace model compatible with `SentenceTransformer`:
 
 ```yaml
-# OpenAI
+# Default — lightweight, fast
 embedding_model:
-  model: text-embedding-3-small
-  api_key: sk-...
+  model: all-MiniLM-L6-v2
 
-# Ollama (local, no API key needed)
+# Higher quality, larger model
 embedding_model:
-  model: ollama/nomic-embed-text
-  api_base: http://localhost:11434
-
-# Azure OpenAI
-embedding_model:
-  model: azure/text-embedding-ada-002
-  api_key: ${AZURE_API_KEY}
-  api_base: https://your-resource.openai.azure.com
-  api_version: "2024-02-01"
-
-# Any OpenAI-compatible endpoint (vLLM, LiteLLM proxy, etc.)
-embedding_model:
-  model: text-embedding-3-small
-  api_key: ${API_KEY}
-  api_base: https://your-endpoint/v1
+  model: BAAI/bge-small-en-v1.5
 ```
 
 ### Step 2: Define a feature view with vector search
@@ -331,13 +316,13 @@ Feature views are still defined in Python and managed through `feast apply`. Dat
 
 ## Deploying on Kubernetes
 
-Below is an example Kubernetes setup that deploys the feature server with an Ollama sidecar for local embedding:
+Below is an example Kubernetes setup that deploys the feature server with Sentence Transformers for local embedding:
 
 ```yaml
 # configmap.yaml (embedding model section)
 embedding_model:
-  model: ollama/nomic-embed-text
-  api_base: http://feast-ollama:11434
+  provider: sentence_transformers
+  model: all-MiniLM-L6-v2
 ```
 
 ```yaml
@@ -354,11 +339,8 @@ With this setup, embedding happens in-cluster. Nothing leaves your network.
 ## Try it yourself
 
 ```bash
-# Install Feast with LiteLLM support
-pip install feast litellm
-
-# If using Ollama for local embeddings (no API key needed)
-ollama pull nomic-embed-text
+# Install Feast with Sentence Transformers support
+pip install feast sentence-transformers
 ```
 
 Configure your `feature_store.yaml` with an `embedding_model` section, define a feature view with vector search enabled, run `feast apply`, load your data, start the server with `feast serve`, and search:

@@ -23,7 +23,6 @@
     BaseEmbedder,
     EmbeddingConfig,
     EmbeddingProvider,
-    LiteLLMEmbeddingProvider,
     MultiModalEmbedder,
     SentenceTransformersEmbeddingProvider,
     get_embedding_provider,
@@ -89,7 +88,6 @@
     "MultiModalEmbedder",
     "EmbeddingConfig",
     "EmbeddingProvider",
-    "LiteLLMEmbeddingProvider",
     "SentenceTransformersEmbeddingProvider",
     "get_embedding_provider",
     "FilterTranslator",

@@ -46,81 +46,6 @@ async def aembed(self, texts: List[str]) -> List[List[float]]:
         ...
 
 
-class LiteLLMEmbeddingProvider(EmbeddingProvider):
-    """EmbeddingProvider backed by LiteLLM.
-
-    Constructed automatically when ``embedding_model`` is set in
-    ``feature_store.yaml`` and no custom provider is supplied.
-    """
-
-    def __init__(
-        self,
-        model: str,
-        api_key: Optional[str] = None,
-        api_base: Optional[str] = None,
-        api_version: Optional[str] = None,
-        dimensions: Optional[int] = None,
-    ):
-        self.model = model
-        self.api_key = api_key
-        self.api_base = api_base
-        self.api_version = api_version
-        self.dimensions = dimensions
-
-    @classmethod
-    def from_config(cls, config: "EmbeddingModelConfig") -> "LiteLLMEmbeddingProvider":
-        """Build a provider from a :class:`~feast.repo_config.EmbeddingModelConfig`."""
-        from feast.repo_config import EmbeddingModelConfig
-
-        assert isinstance(config, EmbeddingModelConfig)
-        return cls(
-            model=config.model,
-            api_key=config.api_key,
-            api_base=config.api_base,
-            api_version=config.api_version,
-            dimensions=config.dimensions,
-        )
-
-    def _litellm_kwargs(self, texts: List[str]) -> dict:
-        kwargs: dict = {"model": self.model, "input": texts}
-        if self.api_key:
-            kwargs["api_key"] = self.api_key
-        if self.api_base:
-            kwargs["api_base"] = self.api_base
-        if self.api_version:
-            kwargs["api_version"] = self.api_version
-        if self.dimensions:
-            kwargs["dimensions"] = self.dimensions
-        return kwargs
-
-    def _import_litellm(self):
-        try:
-            from litellm import embedding as litellm_embedding
-
-            return litellm_embedding
-        except ImportError:
-            raise ImportError(
-                "litellm is required for query embedding. "
-                "Install with: pip install litellm"
-            )
-
-    def embed(self, texts: List[str]) -> List[List[float]]:
-        litellm_embedding = self._import_litellm()
-        response = litellm_embedding(**self._litellm_kwargs(texts))
-        return [item["embedding"] for item in response.data]
-
-    async def aembed(self, texts: List[str]) -> List[List[float]]:
-        try:
-            from litellm import aembedding as litellm_aembedding
-        except ImportError:
-            raise ImportError(
-                "litellm is required for async query embedding. "
-                "Install with: pip install litellm"
-            )
-        response = await litellm_aembedding(**self._litellm_kwargs(texts))
-        return [item["embedding"] for item in response.data]
-
-
 class SentenceTransformersEmbeddingProvider:
     """EmbeddingProvider backed by Sentence Transformers (local inference).
 
@@ -186,20 +111,17 @@ def get_embedding_provider(
 
     Dispatches on ``config.provider``:
 
-    * ``"litellm"`` (default) → :class:`LiteLLMEmbeddingProvider`
-    * ``"sentence_transformers"`` → :class:`SentenceTransformersEmbeddingProvider`
+    * ``"sentence_transformers"`` (default) → :class:`SentenceTransformersEmbeddingProvider`
 
     Raises:
         ValueError: If ``config.provider`` is not a recognised value.
     """
-    provider = (config.provider or "litellm").lower()
-    if provider == "litellm":
-        return LiteLLMEmbeddingProvider.from_config(config)
+    provider = (config.provider or "sentence_transformers").lower()
     if provider == "sentence_transformers":
         return SentenceTransformersEmbeddingProvider.from_config(config)
     raise ValueError(
         f"Unknown embedding provider: '{config.provider}'. "
-        "Supported values: 'litellm', 'sentence_transformers'."
+        "Supported values: 'sentence_transformers'."
     )
 
 

@@ -198,7 +198,7 @@ def __init__(
                 At most one of 'fs_yaml_file' and 'config' can be set.
             embedding_provider (optional): Custom embedding provider implementing
                 the :class:`~feast.embedder.EmbeddingProvider` protocol. When not
-                supplied, a :class:`~feast.embedder.LiteLLMEmbeddingProvider` is
+                supplied, a :class:`~feast.embedder.SentenceTransformersEmbeddingProvider` is
                 created from ``embedding_model`` in ``feature_store.yaml``.
 
         Raises:
@@ -410,13 +410,7 @@ def embedding_provider(self) -> "EmbeddingProvider":
                     "configured in feature_store.yaml. Either pass an "
                     "embedding_provider to FeatureStore() or add an "
                     "'embedding_model' section to feature_store.yaml.\n"
-                    "Examples:\n"
-                    "  # LiteLLM (API-backed — OpenAI, Azure, Ollama, …)\n"
-                    "  embedding_model:\n"
-                    "    provider: litellm\n"
-                    "    model: text-embedding-3-small\n"
-                    "    api_key: sk-...\n\n"
-                    "  # Sentence Transformers (local, no API key required)\n"
+                    "Example:\n"
                     "  embedding_model:\n"
                     "    provider: sentence_transformers\n"
                     "    model: all-MiniLM-L6-v2"
@@ -4165,8 +4159,8 @@ async def retrieve_online_documents_openai(
         """
         OpenAI-compatible vector store search.
 
-        Accepts a raw query string, optionally embeds it via LiteLLM
-        (when ``query_embedding_model`` is configured in feature_store.yaml),
+        Accepts a raw query string, embeds it via the configured embedding
+        provider (when ``embedding_model`` is configured in feature_store.yaml),
         and returns results in OpenAI's ``vector_store.search_results.page``
         format.
 

@@ -343,64 +343,32 @@ class EmbeddingModelConfig(FeastConfigBaseModel):
     Required when using ``retrieve_online_documents_openai`` or the
     ``/v1/vector_stores/{vector_store_id}/search`` endpoint.
 
-    Two providers are supported:
-
-    **LiteLLM** (default) — API-backed: OpenAI, Azure, Ollama, Cohere, HuggingFace, etc.
-
-    Example in ``feature_store.yaml``::
-
-        embedding_model:
-          provider: litellm                 # default; can be omitted
-          model: text-embedding-3-small
-          api_key: ${OPENAI_API_KEY}
-          api_base: https://api.openai.com/v1
-
-    **Sentence Transformers** — runs locally, no API key required.  Ideal for
-    air-gapped or cost-sensitive deployments.  Requires the
+    **Sentence Transformers** (default) — runs locally, no API key required.
+    Ideal for air-gapped or cost-sensitive deployments.  Requires the
     ``sentence-transformers`` package (``pip install sentence-transformers``).
 
     Example in ``feature_store.yaml``::
 
         embedding_model:
-          provider: sentence_transformers
+          provider: sentence_transformers   # default; can be omitted
           model: all-MiniLM-L6-v2
+
+    Custom providers can be plugged in by implementing the
+    :class:`~feast.embedder.EmbeddingProvider` protocol and passing an
+    instance to :class:`~feast.feature_store.FeatureStore`.
     """
 
-    provider: str = "litellm"
+    provider: str = "sentence_transformers"
     """Embedding backend to use.  Supported values:
-    ``'litellm'`` (default) or ``'sentence_transformers'``."""
+    ``'sentence_transformers'`` (default)."""
 
     model: str
     """Model identifier.
 
-    * For ``litellm``: any LiteLLM-supported model string, e.g.
-      ``'text-embedding-3-small'``, ``'cohere/embed-english-v3.0'``,
-      ``'azure/my-deployment'``.
-    * For ``sentence_transformers``: any HuggingFace model name compatible
-      with ``SentenceTransformer``, e.g. ``'all-MiniLM-L6-v2'``,
-      ``'BAAI/bge-small-en-v1.5'``.
+    Any HuggingFace model name compatible with ``SentenceTransformer``,
+    e.g. ``'all-MiniLM-L6-v2'``, ``'BAAI/bge-small-en-v1.5'``.
     """
 
-    api_key: Optional[str] = None
-    """API key for the embedding provider (LiteLLM only).  If not set,
-    LiteLLM falls back to the relevant environment variable (e.g.
-    ``OPENAI_API_KEY``).  Ignored for ``sentence_transformers``."""
-
-    api_base: Optional[str] = None
-    """Custom API base URL (LiteLLM only), e.g.
-    ``'https://my-azure-deployment.openai.azure.com/'``.
-    Ignored for ``sentence_transformers``."""
-
-    api_version: Optional[str] = None
-    """API version (LiteLLM / Azure OpenAI only).
-    Ignored for ``sentence_transformers``."""
-
-    dimensions: Optional[StrictInt] = None
-    """Output embedding dimensionality (LiteLLM only).  Supported by
-    ``text-embedding-3`` and newer models.  If not set, the model's default
-    dimension is used.  Ignored for ``sentence_transformers`` (dimension is
-    determined by the model)."""
-
 
 class RepoConfig(FeastBaseModel):
     """Repo config. Typically loaded from `feature_store.yaml`"""
@@ -444,7 +412,7 @@ class RepoConfig(FeastBaseModel):
     embedding_model: Optional[EmbeddingModelConfig] = Field(
         None, alias="embedding_model"
     )
-    """ EmbeddingModelConfig: LiteLLM embedding model configuration.
+    """ EmbeddingModelConfig: Embedding model configuration.
     Required when using retrieve_online_documents_openai or the
     OpenAI-compatible vector store search endpoint. """