Visitar URL original
feat: Add enabled/disabled toggle for feature views (#6401) · feast-dev/feast@5f1fa0d · GitHub
Skip to content

Commit 5f1fa0d

Browse files
rpathadentkathole1fanwangjvincent-mongodbJwrede
authored
feat: Add enabled/disabled toggle for feature views (#6401)
* feat: Add enabled/disabled toggle for feature views Signed-off-by: RutujaPathade <73137503+RutujaPathade@users.noreply.github.com> * feat: Add demo noteboooks for users Signed-off-by: ntkathole <nikhilkathole2683@gmail.com> Signed-off-by: RutujaPathade <73137503+RutujaPathade@users.noreply.github.com> * feat: Add CLI enable/disable commands and registry metadata support Signed-off-by: RutujaPathade <73137503+RutujaPathade@users.noreply.github.com> * Added features Signed-off-by: RutujaPathade <73137503+RutujaPathade@users.noreply.github.com> * fix(compute-engine/local): Honor field_mapping on join keys in dedup + join nodes (#6395) * fix: Apply field mapping to join keys in local compute engine nodes When a batch source defines a `field_mapping` that renames an entity join key (e.g. `USERID` -> `user_id`), the source-read node renames the columns on the pulled Arrow table to their mapped names. Downstream `LocalDedupNode` and `LocalJoinNode` then look up the *pre-mapping* names from `column_info.join_keys`, which raises `KeyError: Index(['USERID'])` during materialization (or returns an empty join). Add a `join_keys_columns` property on `ColumnInfo` that mirrors the existing `timestamp_column` / `created_timestamp_column` properties — returning join keys translated through `field_mapping` — and use it from the dedup and join nodes. Fixes #5942. Signed-off-by: 1fanwang <1fannnw@gmail.com> * test: also cover LocalJoinNode field_mapping case Signed-off-by: 1fanwang <1fannnw@gmail.com> --------- Signed-off-by: 1fanwang <1fannnw@gmail.com> Signed-off-by: RutujaPathade <73137503+RutujaPathade@users.noreply.github.com> * feat: Add Prometheus gauges for FeatureStore installation telemetry (#6354) Signed-off-by: ntkathole <nikhilkathole2683@gmail.com> Signed-off-by: RutujaPathade <73137503+RutujaPathade@users.noreply.github.com> * docs: Rename Atlas Vector Search to MongoDB Vector Search and fix code examples Signed-off-by: jvincent-mongodb <jeffrey.vincent@mongodb.com> Signed-off-by: RutujaPathade <73137503+RutujaPathade@users.noreply.github.com> * feat(dynamodb): Use ProjectionExpression when requested_features is set The requested_features parameter was accepted by online_read and online_read_async but never used -- DynamoDB always fetched all features stored in the values map regardless. Add a ProjectionExpression to BatchGetItem requests when requested_features is provided, reducing data transfer, latency, and read costs. Fixes #6058 Signed-off-by: Jonathan Wrede <wrede.jonathan00@gmail.com> Signed-off-by: RutujaPathade <73137503+RutujaPathade@users.noreply.github.com> * fix(dynamodb): Fix mypy type for _build_projection_expression return The return dict contains both str and Dict[str, str] values, so the return type must be Dict[str, Any] not Dict[str, str]. Signed-off-by: Jonathan Wrede <wrede.jonathan00@gmail.com> Signed-off-by: RutujaPathade <73137503+RutujaPathade@users.noreply.github.com> * fix(bigquery): Enable list inference for parquet loads in offline_write_batch When pushing features with array/list types (e.g. STRING_LIST) to BigQuery via offline_write_batch, the data arrives as empty arrays because BigQuery's parquet loader does not infer list structure by default. Set parquet_options.enable_list_inference = True on the LoadJobConfig so array columns are written correctly. Fixes #5845 Signed-off-by: Jonathan Wrede <wrede.jonathan00@gmail.com> Signed-off-by: RutujaPathade <73137503+RutujaPathade@users.noreply.github.com> * fix(trino): Clean up temporary entity tables after retrieval (#6381) * fix(trino): Clean up temporary entity tables after retrieval TrinoOfflineStore.get_historical_features() creates a temporary table for the entity DataFrame but never drops it, leaking tables indefinitely. Apply the same context manager pattern used by BigQuery, Redshift, and Athena offline stores: wrap the query in a generator that issues DROP TABLE IF EXISTS in a finally block. Fixes #6306 Signed-off-by: Jonathan Wrede <wrede.jonathan00@gmail.com> * fix: sort imports for ruff compliance Signed-off-by: Jonathan Wrede <wrede.jonathan00@gmail.com> * fix: decouple temp table cleanup from query access Avoid dropping the temporary entity table on to_sql() calls. Previously, every method used a context manager that dropped the table on exit, so calling to_sql() before to_df() would destroy the table and cause subsequent queries to fail. Now the query is stored as a plain string and cleanup is handled by a dedicated _drop_temp_table() method called only after query execution (to_df, to_trino). A __del__ fallback ensures cleanup if execution methods are never called. The _cleaned_up flag makes the drop idempotent. Signed-off-by: Jonathan Wrede <wrede.jonathan00@gmail.com> --------- Signed-off-by: Jonathan Wrede <wrede.jonathan00@gmail.com> Signed-off-by: RutujaPathade <73137503+RutujaPathade@users.noreply.github.com> * feat(bigquery): Support DATE-type event timestamp columns (#6362) * feat(bigquery): Support DATE-type event timestamp columns When the event_timestamp column in BigQuery is a DATE type, the generated SQL wraps comparison values in TIMESTAMP(), causing a type mismatch error. This adds a timestamp_field_type parameter to BigQuerySource that, when set to "DATE", generates DATE() comparisons instead. Closes #2530 (part 2) Signed-off-by: Jonathan Wrede <wrede.jonathan00@gmail.com> * fix(bigquery): Use protobuf 4.25.x compatible generated code The proto files were regenerated with protobuf 6.31.1 / grpcio-tools 1.80.0, which imports runtime_version -- a module that does not exist in protobuf 4.25.x used by the project. Revert generated code to 4.25.1 format while keeping the new timestamp_field_type field. Signed-off-by: Jonathan Wrede <wrede.jonathan00@gmail.com> * fix(bigquery): Add Literal type annotation for cast_style Mypy infers str from the ternary expression; annotate with the exact Literal union so the call to get_timestamp_filter_sql passes type checking. Signed-off-by: Jonathan Wrede <wrede.jonathan00@gmail.com> * fix: Make timestamp_field_type default to None in FeatureViewQueryContext Callers that do not use DATE-typed timestamp fields (e.g. Spark offline store tests) should not be forced to pass timestamp_field_type. Adding a default keeps the new field backward-compatible. Signed-off-by: Jonathan Wrede <wrede.jonathan00@gmail.com> * fix: Keep timestamp_field_type required in FeatureViewQueryContext A default value on timestamp_field_type breaks the SparkFeatureViewQueryContext subclass because its non-default fields (min_date_partition, max_date_partition) would follow a field with a default. Instead, keep it required and update the Spark test to pass it. Signed-off-by: Jonathan Wrede <wrede.jonathan00@gmail.com> * fix: regenerate protos matching upstream mypy-protobuf style Reset all non-DataSource generated files to match master. Only DataSource_pb2.py and DataSource_pb2.pyi contain our timestamp_field_type additions (field 28). The .pyi stub is hand-edited to match the existing import style used on master. Signed-off-by: Jonathan Wrede <wrede.jonathan00@gmail.com> --------- Signed-off-by: Jonathan Wrede <wrede.jonathan00@gmail.com> Signed-off-by: RutujaPathade <73137503+RutujaPathade@users.noreply.github.com> * fix: Fixes for ray source Signed-off-by: ntkathole <nikhilkathole2683@gmail.com> Signed-off-by: RutujaPathade <73137503+RutujaPathade@users.noreply.github.com> * feat: Expose registry endpoints on feature server for MCP access Mount the existing REST registry routers under /registry on the feature server so that fastapi_mcp automatically exposes registry introspection (list/get for entities, feature views, data sources, feature services, permissions, projects, saved datasets, lineage, search) as MCP tools. The RegistryServer is created in-process from store.registry — no external registry server is required. Auth is enforced via inject_user_details on every mounted router. Made-with: Cursor Signed-off-by: Chaitany patel <patelchaitany93@gmail.com> Made-with: Cursor Signed-off-by: RutujaPathade <73137503+RutujaPathade@users.noreply.github.com> * fix: Revert state propagation to always update in _update_metadata_fields Signed-off-by: RutujaPathade <73137503+RutujaPathade@users.noreply.github.com> * fix: Recompile protos for protobuf 4.x compatibility and fix state machine to be opt-in Signed-off-by: RutujaPathade <73137503+RutujaPathade@users.noreply.github.com> * feat: Add unit tests for state machine and clean up lazy imports in registry Signed-off-by: RutujaPathade <73137503+RutujaPathade@users.noreply.github.com> * fix: Address review comments for feature view state management Signed-off-by: RutujaPathade <73137503+RutujaPathade@users.noreply.github.com> * fix: Resolve integration test failures in apply loop Signed-off-by: RutujaPathade <73137503+RutujaPathade@users.noreply.github.com> * fix: Resolve integration test failures in apply loop Signed-off-by: RutujaPathade <73137503+RutujaPathade@users.noreply.github.com> * Apply suggestion from @ntkathole Co-authored-by: Nikhil Kathole <nikhilkathole2683@gmail.com> Signed-off-by: RutujaPathade <73137503+RutujaPathade@users.noreply.github.com> * fix: Resolve review comments for feature_store Signed-off-by: RutujaPathade <73137503+RutujaPathade@users.noreply.github.com> * fix: Resolve review comments for feature_views.py Signed-off-by: RutujaPathade <73137503+RutujaPathade@users.noreply.github.com> * feat: Add FeatureStore methods and update describe for enabled/state Signed-off-by: RutujaPathade <73137503+RutujaPathade@users.noreply.github.com> * fix: Add type: ignore comments for mypy on BaseFeatureView attr access Signed-off-by: RutujaPathade <73137503+RutujaPathade@users.noreply.github.com> * fix: Remove REST API endpoints for enable/disable/set-state (deferred to follow-up PR) Signed-off-by: RutujaPathade <73137503+RutujaPathade@users.noreply.github.com> --------- Signed-off-by: RutujaPathade <73137503+RutujaPathade@users.noreply.github.com> Signed-off-by: ntkathole <nikhilkathole2683@gmail.com> Signed-off-by: 1fanwang <1fannnw@gmail.com> Signed-off-by: jvincent-mongodb <jeffrey.vincent@mongodb.com> Signed-off-by: Jonathan Wrede <wrede.jonathan00@gmail.com> Signed-off-by: Chaitany patel <patelchaitany93@gmail.com> Co-authored-by: RutujaPathade <73137503+RutujaPathade@users.noreply.github.com> Co-authored-by: ntkathole <nikhilkathole2683@gmail.com> Co-authored-by: Stefan Wang <1fannnw@gmail.com> Co-authored-by: jvincent-mongodb <jeffrey.vincent@mongodb.com> Co-authored-by: Jonathan Wrede <wrede.jonathan00@gmail.com> Co-authored-by: Jwrede <62910358+Jwrede@users.noreply.github.com> Co-authored-by: Chaitany patel <patelchaitany93@gmail.com>
1 parent c9ae2b4 commit 5f1fa0d

46 files changed

Lines changed: 5177 additions & 5062 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎protos/feast/core/FeatureView.proto‎

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -28,6 +28,26 @@ import "feast/core/DataSource.proto";
2828
import "feast/core/Feature.proto";
2929
import "feast/core/Transformation.proto";
3030

31+
// Lifecycle state of a feature view.
32+
enum FeatureViewState {
33+
// Default value for backward compatibility. Treated as AVAILABLE_ONLINE
34+
// for existing feature views that predate the state machine.
35+
STATE_UNSPECIFIED = 0;
36+
37+
// Feature view has been registered via feast apply but no data is available.
38+
CREATED = 1;
39+
40+
// Feature engineering / offline data generation is complete.
41+
// The feature view is ready to be materialized.
42+
GENERATED = 2;
43+
44+
// Materialization is currently in progress.
45+
MATERIALIZING = 3;
46+
47+
// Materialization completed. Features are available in the online store.
48+
AVAILABLE_ONLINE = 4;
49+
}
50+
3151
message FeatureView {
3252
// User-specified specifications of this feature view.
3353
FeatureViewSpec spec = 1;
@@ -100,6 +120,11 @@ message FeatureViewSpec {
100120

101121
// Organizational unit that owns this feature view (e.g. "ads", "search").
102122
string org = 19;
123+
124+
// Whether this feature view is disabled for serving and materialization.
125+
// When true, the feature view will not serve online features or be materialized.
126+
// Defaults to false (enabled) for backward compatibility.
127+
bool disabled = 20;
103128
}
104129

105130
message FeatureViewMeta {
@@ -117,6 +142,9 @@ message FeatureViewMeta {
117142

118143
// Auto-generated UUID identifying this specific version.
119144
string version_id = 5;
145+
146+
// Lifecycle state of this feature view.
147+
FeatureViewState state = 6;
120148
}
121149

122150
message MaterializationInterval {

‎protos/feast/core/OnDemandFeatureView.proto‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -80,6 +80,11 @@ message OnDemandFeatureViewSpec {
8080

8181
// Organizational unit that owns this feature view (e.g. "ads", "search").
8282
string org = 18;
83+
84+
// Whether this feature view is disabled for serving.
85+
// When true, the feature view will not serve features.
86+
// Defaults to false (enabled) for backward compatibility.
87+
bool disabled = 19;
8388
}
8489

8590
message OnDemandFeatureViewMeta {
@@ -94,6 +99,9 @@ message OnDemandFeatureViewMeta {
9499

95100
// Auto-generated UUID identifying this specific version.
96101
string version_id = 4;
102+
103+
// Lifecycle state of this feature view.
104+
FeatureViewState state = 5;
97105
}
98106

99107
message OnDemandSource {

‎protos/feast/core/StreamFeatureView.proto‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -108,5 +108,10 @@ message StreamFeatureViewSpec {
108108

109109
// Organizational unit that owns this stream feature view (e.g. "ads", "search").
110110
string org = 22;
111+
112+
// Whether this feature view is disabled for serving and materialization.
113+
// When true, the feature view will not serve online features or be materialized.
114+
// Defaults to false (enabled) for backward compatibility.
115+
bool disabled = 23;
111116
}
112117

‎sdk/python/feast/__init__.py‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -24,7 +24,7 @@
2424
from .feature import Feature
2525
from .feature_service import FeatureService
2626
from .feature_store import FeatureStore
27-
from .feature_view import FeatureView
27+
from .feature_view import FeatureView, FeatureViewState
2828
from .field import Field
2929
from .on_demand_feature_view import OnDemandFeatureView
3030
from .project import Project
@@ -53,6 +53,7 @@
5353
"FeatureService",
5454
"FeatureStore",
5555
"FeatureView",
56+
"FeatureViewState",
5657
"OnDemandFeatureView",
5758
"RepoConfig",
5859
"StreamFeatureView",

‎sdk/python/feast/cli/feature_views.py‎

Lines changed: 127 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,16 @@
1+
import sys
2+
13
import click
24
import yaml
35

46
from feast import utils
57
from feast.cli.cli_options import tagsOption
68
from feast.errors import FeastObjectNotFoundException
7-
from feast.feature_view import FeatureView
9+
from feast.feature_view import (
10+
_VALID_STATE_TRANSITIONS,
11+
FeatureView,
12+
FeatureViewState,
13+
)
814
from feast.on_demand_feature_view import OnDemandFeatureView
915
from feast.repo_operations import create_feature_store
1016

@@ -32,11 +38,13 @@ def feature_view_describe(ctx: click.Context, name: str):
3238
print(e)
3339
exit(1)
3440

35-
print(
36-
yaml.dump(
37-
yaml.safe_load(str(feature_view)), default_flow_style=False, sort_keys=False
38-
)
39-
)
41+
data = yaml.safe_load(str(feature_view))
42+
# Always show enabled and state even when they are at default values.
43+
if hasattr(feature_view, "enabled"):
44+
data["enabled"] = feature_view.enabled
45+
if hasattr(feature_view, "state"):
46+
data["state"] = feature_view.state.name
47+
print(yaml.dump(data, default_flow_style=False, sort_keys=False))
4048

4149

4250
@feature_views_cmd.command(name="list")
@@ -59,17 +67,129 @@ def feature_view_list(ctx: click.Context, tags: list[str]):
5967
elif isinstance(feature_view, OnDemandFeatureView):
6068
for backing_fv in feature_view.source_feature_view_projections.values():
6169
entities.update(store.get_feature_view(backing_fv.name).entities)
70+
enabled = getattr(feature_view, "enabled", True)
71+
state = getattr(feature_view, "state", FeatureViewState.STATE_UNSPECIFIED)
72+
state_display = (
73+
state.name if isinstance(state, FeatureViewState) else str(state)
74+
)
6275
table.append(
6376
[
6477
feature_view.name,
6578
entities if len(entities) > 0 else "n/a",
6679
type(feature_view).__name__,
80+
"Yes" if enabled else "No",
81+
state_display,
6782
]
6883
)
6984

7085
from tabulate import tabulate
7186

72-
print(tabulate(table, headers=["NAME", "ENTITIES", "TYPE"], tablefmt="plain"))
87+
print(
88+
tabulate(
89+
table,
90+
headers=["NAME", "ENTITIES", "TYPE", "ENABLED", "STATE"],
91+
tablefmt="plain",
92+
)
93+
)
94+
95+
96+
@feature_views_cmd.command("enable")
97+
@click.argument("name", type=click.STRING)
98+
@click.pass_context
99+
def feature_view_enable(ctx: click.Context, name: str):
100+
"""
101+
Enable a feature view for serving and materialization.
102+
"""
103+
store = create_feature_store(ctx)
104+
try:
105+
fv = store.registry.get_any_feature_view(name, store.project)
106+
except FeastObjectNotFoundException as e:
107+
print(e)
108+
sys.exit(1)
109+
110+
if not isinstance(fv, (FeatureView, OnDemandFeatureView)):
111+
print(f"Feature view '{name}' does not support enable/disable.")
112+
return
113+
114+
if fv.enabled:
115+
print(f"Feature view '{name}' is already enabled.")
116+
return
117+
118+
fv.enabled = True
119+
store.registry.apply_feature_view(fv, store.project)
120+
print(f"Feature view '{name}' has been enabled.")
121+
122+
123+
@feature_views_cmd.command("disable")
124+
@click.argument("name", type=click.STRING)
125+
@click.pass_context
126+
def feature_view_disable(ctx: click.Context, name: str):
127+
"""
128+
Disable a feature view to prevent serving and materialization.
129+
"""
130+
store = create_feature_store(ctx)
131+
try:
132+
fv = store.registry.get_any_feature_view(name, store.project)
133+
except FeastObjectNotFoundException as e:
134+
print(e)
135+
sys.exit(1)
136+
137+
if not isinstance(fv, (FeatureView, OnDemandFeatureView)):
138+
print(f"Feature view '{name}' does not support enable/disable.")
139+
return
140+
141+
if not fv.enabled:
142+
print(f"Feature view '{name}' is already disabled.")
143+
return
144+
145+
fv.enabled = False
146+
store.registry.apply_feature_view(fv, store.project)
147+
print(f"Feature view '{name}' has been disabled.")
148+
149+
150+
@feature_views_cmd.command("set-state")
151+
@click.argument("name", type=click.STRING)
152+
@click.argument(
153+
"state",
154+
type=click.Choice(
155+
["CREATED", "GENERATED", "MATERIALIZING", "AVAILABLE_ONLINE"],
156+
case_sensitive=False,
157+
),
158+
)
159+
@click.pass_context
160+
def feature_view_set_state(ctx: click.Context, name: str, state: str):
161+
"""
162+
Set the lifecycle state of a feature view.
163+
"""
164+
store = create_feature_store(ctx)
165+
try:
166+
fv = store.registry.get_any_feature_view(name, store.project)
167+
except FeastObjectNotFoundException as e:
168+
print(e)
169+
sys.exit(1)
170+
171+
if not isinstance(fv, (FeatureView, OnDemandFeatureView)):
172+
print(f"Feature view '{name}' does not support state management.")
173+
return
174+
175+
new_state = FeatureViewState[state.upper()]
176+
if fv.state == new_state:
177+
print(f"Feature view '{name}' is already in state {new_state.name}.")
178+
return
179+
180+
if not fv.state.can_transition_to(new_state):
181+
current = fv.state.name
182+
allowed = _VALID_STATE_TRANSITIONS.get(fv.state, set())
183+
allowed_names = ", ".join(sorted(s.name for s in allowed)) or "none"
184+
print(
185+
f"Invalid state transition: {current} -> {new_state.name}. "
186+
f"Allowed transitions from {current}: {allowed_names}."
187+
)
188+
return
189+
190+
fv.state = new_state
191+
store.registry.apply_feature_view(fv, store.project)
192+
print(f"Feature view '{name}' state set to {new_state.name}.")
73193

74194

75195
@feature_views_cmd.command("list-versions")

0 commit comments

Comments
 (0)