Commit a7f7a02

title: "Tuning the Feast Feature Server for Sub-2ms Online Serving"

description: "A practical guide to achieving low-latency, high-throughput feature serving with Feast on Kubernetes — from default configuration to production-grade performance with pre-computed feature vectors and benchmarks at every step."

date: 2026-06-02

authors: ["Nikhil Kathole"]

**Feast supports production-grade worker configuration, connection pooling, async reads, batched pipelines, serialization optimizations, and pre-computed feature vectors for the Python feature server.** This post walks through a real-world performance tuning exercise in two stages: first, server and client tuning that brings p99 latency down to **sub-5ms** for single-row requests; then, **pre-computed feature vectors** that push it further to **sub-2ms p99** — regardless of how many feature views your FeatureService spans. We share the benchmarking methodology, the exact configuration changes, and the measured impact of each step so you can apply the same approach to your own deployments.

When you deploy Feast on Kubernetes using the [Feast Operator](https://docs.feast.dev/how-to-guides/production-deployment-topologies), the default configuration is designed for simplicity — a single Gunicorn worker, short keep-alive timeouts, and frequent registry refreshes. This is fine for development but leaves significant performance on the table for production workloads where every millisecond matters.

We set out to answer a practical question: **how low can we push the Feast online server's p99 latency, and what does it take to get there?**

| Component | Configuration |

|-----------|--------------|

| **Online Store** | Redis 7.0.12 (standalone, in-cluster) |

| **Registry** | PostgreSQL 16 (SQL registry) |

| **Platform** | Kubernetes |

| **Deployment** | Feast Operator with `FeatureStore` CR |

We used a banking feature store project with multiple feature views spanning customer demographics, transactions, and behavioral profiles.

All benchmarks run 200 iterations (after 30–50 warmup) for each scenario, measuring p50, p95, p99, and mean latency. Throughput is measured with 10 concurrent workers over 15 seconds.

Feast supports three ways to retrieve online features. Understanding how each one works is key to knowing where latency comes from — and where to optimize.

The simplest and most common pattern. Your application sends a JSON request to the feature server's `/get-online-features` endpoint. The server holds persistent Redis connections and a pre-loaded registry, so each request is just a Redis read plus JSON serialization. HTTP keep-alive reuses TCP/TLS connections across requests.

The Python SDK connects to Redis directly — no HTTP hop, no JSON overhead. However, it pays for in-process registry lookups and entity key serialization on every call, and reads each FeatureView sequentially.

The SDK delegates feature retrieval to a remote feature server over HTTP. This combines the worst of both worlds: SDK-side overhead *plus* an HTTP round-trip. Without connection pooling, each call creates a new TCP connection and TLS handshake.

With no tuning applied — a single Gunicorn worker, default timeouts, and no connection pooling:

| Mode | p99 (1 row) | p99 (5 rows) | Throughput |

|------|----------------|-------------------|------------|

| **REST API** | 6.92 ms | 4.94 ms | 480 RPS |

| **Direct SDK** | 5.83 ms | 5.59 ms | — |

| **Remote SDK** | 11.71 ms | **74.31 ms** | ~2 RPS |

The REST API and Direct SDK are already in the 5–7ms range out of the box, but the Remote SDK fails badly — p99 spiking to **74ms** at just 5 rows due to per-request TCP/TLS setup overhead. This is our starting point.

These are changes you apply to the **feature server deployment** — no code changes needed, just configuration via the `FeatureStore` CR and Redis runtime settings.

The Feast Operator exposes `workerConfigs` in the `FeatureStore` CR, letting you tune the Gunicorn server without rebuilding images:

apiVersion: feast.dev/v1alpha1

kind: FeatureStore

spec:

services:

onlineStore:

server:

workerConfigs:

workers: -1 # Auto: 2 × CPU cores + 1

keepAliveTimeout: 120 # Reuse connections longer

maxRequests: 5000 # Recycle workers to prevent memory leaks

maxRequestsJitter: 200 # Stagger recycling

registryTTLSeconds: 300 # Reduce registry refresh overhead

workerConnections: 2000 # High-concurrency support

Three Redis settings made a measurable difference:

For production, we added horizontal scaling and availability guarantees using the Feast Operator's [built-in HA support](https://docs.feast.dev/how-to-guides/feast-snowflake-gcp-aws/scaling-feast):

1. **Set `workers: -1`** — single most impactful change

3. **Set `registryTTLSeconds: 300`** — reduce registry refresh overhead

4. **Use `replicas: 2`** minimum with HPA for burst capacity

---

Feast now includes `HttpSessionManager` — a thread-safe, singleton session manager that reuses HTTP connections across requests with configurable pooling and retry:

---

`_check_versioned_read_support()` performed up to 7 lazy module imports on **every request** to determine if the current online store supports versioned reads. We cache the result per store instance, resolving imports once and eliminating ~0.5–1ms of overhead per request.

---

After applying all server-side configuration, client-side configuration, and code enhancements:

- For FeatureServices spanning multiple feature views, **`precompute_online=True` is the single most impactful optimization** — it changes the read complexity from O(N feature views) to O(1).

- At large batch sizes, the bottleneck shifts from store I/O to Python CPU overhead (protobuf deserialization). For these workloads, split large requests into smaller batches on the client side.

---

Feast also supports PostgreSQL, SQLite, Snowflake, Bigtable, and more. The general rule is: **the online store is the single largest factor in `get_online_features()` latency** — choose based on your latency budget, throughput needs, and operational requirements. The tuning steps in this post (worker configuration, registry caching, connection pooling, serialization optimization) apply equally to all stores — they optimize the layers above.

---

The tuning steps above achieve our first target: **sub-5ms p99** for single-row requests. But for FeatureServices spanning multiple feature views, per-FV read fan-out becomes the dominant bottleneck — each request issues N separate store reads, N protobuf deserializations, and N response assemblies. To reach our final target of **sub-2ms**, we need to eliminate this fan-out entirely.

1. **Define** a FeatureService with `precompute_online=True`:

2. **Apply** and **materialize** as usual — vectors are built automatically:

3. **Read** features as usual — the server automatically uses the pre-computed path:

---

To deploy the same setup: