Visitar URL original
feat: Offline store update pull_all_from_table_or_query to make timestampfield optional by haoxu0 · Pull Request #5281 · feast-dev/feast · GitHub
Skip to content

feat: Offline store update pull_all_from_table_or_query to make timestampfield optional #5281

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
franciscojavierarceo merged 20 commits into from
Apr 20, 2025
Merged

feat: Offline store update pull_all_from_table_or_query to make timestampfield optional #5281

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
112014e
Update offline store pull all API date field optional
haoxu0 Apr 18, 2025
c75c264
Update offline store pull all API date field optional
haoxu0 Apr 18, 2025
b62fd1a
update
haoxu0 Apr 18, 2025
8b5bdae
update
haoxu0 Apr 18, 2025
e92d015
update
haoxu0 Apr 18, 2025
4765d8f
fix test
haoxu0 Apr 18, 2025
0bbdbbc
update source read node
haoxu0 Apr 18, 2025
0336318
fix linting
haoxu0 Apr 18, 2025
ac03a75
fix linting
haoxu0 Apr 18, 2025
0033b9c
fix linting
haoxu0 Apr 18, 2025
743a490
fix linting
haoxu0 Apr 18, 2025
4285b5f
fix linting
haoxu0 Apr 18, 2025
618c4b8
fix test
haoxu0 Apr 18, 2025
afcaf9c
fix test
haoxu0 Apr 18, 2025
8e02747
fix test
haoxu0 Apr 18, 2025
71daec2
fix test
haoxu0 Apr 18, 2025
23a0fb4
fix test
haoxu0 Apr 18, 2025
bcf1d7a
fix test
haoxu0 Apr 18, 2025
89ad0da
fix test
haoxu0 Apr 18, 2025
cdae95b
fix test
haoxu0 Apr 18, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
fix test 
Signed-off-by: HaoXuAI <sduxuhao@gmail.com>
  • Loading branch information
@haoxu0
haoxu0 committed Apr 18, 2025
commit 71daec23b4671d5a75b6e4b63a5afe31f7b08425
14 changes: 7 additions & 7 deletions sdk/python/feast/infra/compute_engines/spark/node.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -211,13 +211,13 @@ def execute(self, context: ExecutionContext) -> DAGValue:
if ENTITY_TS_ALIAS in input_df.columns:
filtered_df = filtered_df.filter(F.col(ts_col) <= F.col(ENTITY_TS_ALIAS))

# Optional TTL filter: feature.ts >= entity.event_timestamp - ttl
if self.ttl:
ttl_seconds = int(self.ttl.total_seconds())
lower_bound = F.col(ENTITY_TS_ALIAS) - F.expr(
f"INTERVAL {ttl_seconds} seconds"
)
filtered_df = filtered_df.filter(F.col(ts_col) >= lower_bound)
# Optional TTL filter: feature.ts >= entity.event_timestamp - ttl
if self.ttl:
ttl_seconds = int(self.ttl.total_seconds())
lower_bound = F.col(ENTITY_TS_ALIAS) - F.expr(
f"INTERVAL {ttl_seconds} seconds"
)
filtered_df = filtered_df.filter(F.col(ts_col) >= lower_bound)

# Optional custom filter condition
if self.filter_condition:
Expand Down
2 changes: 0 additions & 2 deletions sdk/python/tests/integration/compute_engines/spark/test_compute.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -161,8 +161,6 @@ def transform_feature(df: DataFrame) -> DataFrame:
feature_view=driver_stats_fv,
full_feature_name=False,
registry=registry,
start_time=now - timedelta(days=1),
end_time=now,
)

# 🧪 Run SparkComputeEngine
Expand Down