Visitar URL original
fix: Fix five bugs in milvus online store by korbonits · Pull Request #6275 · feast-dev/feast · GitHub
Skip to content
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions sdk/python/docs/source/feast.aggregation.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
feast.aggregation package
=========================

Subpackages
-----------

.. toctree::
:maxdepth: 4

feast.aggregation.tiling

Module contents
---------------

.. automodule:: feast.aggregation
:members:
:show-inheritance:
9 changes: 9 additions & 0 deletions sdk/python/docs/source/feast.aggregation.tiling.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
feast.aggregation.tiling package
=================================

Module contents
---------------

.. automodule:: feast.aggregation.tiling
:members:
:show-inheritance:
37 changes: 37 additions & 0 deletions sdk/python/docs/source/feast.dbt.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
feast.dbt package
=================

Submodules
----------

feast.dbt.codegen module
------------------------

.. automodule:: feast.dbt.codegen
:members:
:undoc-members:
:show-inheritance:

feast.dbt.mapper module
-----------------------

.. automodule:: feast.dbt.mapper
:members:
:undoc-members:
:show-inheritance:

feast.dbt.parser module
-----------------------

.. automodule:: feast.dbt.parser
:members:
:undoc-members:
:show-inheritance:

Module contents
---------------

.. automodule:: feast.dbt
:members:
:undoc-members:
:show-inheritance:
8 changes: 8 additions & 0 deletions sdk/python/docs/source/feast.diff.rst
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,14 @@ feast.diff package
Submodules
----------

feast.diff.apply\_progress module
----------------------------------

.. automodule:: feast.diff.apply_progress
:members:
:undoc-members:
:show-inheritance:

feast.diff.infra\_diff module
-----------------------------

Expand Down
9 changes: 9 additions & 0 deletions sdk/python/docs/source/feast.openlineage.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
feast.openlineage package
=========================

Module contents
---------------

.. automodule:: feast.openlineage
:members:
:show-inheritance:
11 changes: 3 additions & 8 deletions sdk/python/docs/source/feast.rst
Original file line number Diff line number Diff line change
Expand Up @@ -7,13 +7,16 @@ Subpackages
.. toctree::
:maxdepth: 4

feast.aggregation
feast.api
feast.cli
feast.dbt
feast.diff
feast.dqm
feast.embedded_go
feast.infra
feast.lineage
feast.openlineage
feast.permissions
feast.protos
feast.transformation
Expand All @@ -22,14 +25,6 @@ Subpackages
Submodules
----------

feast.aggregation module
------------------------

.. automodule:: feast.aggregation
:members:
:undoc-members:
:show-inheritance:

feast.arrow\_error\_handler module
----------------------------------

Expand Down

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Falsy check on distance drops valid 0.0 distance values

At line 746, if distance is used to decide whether to wrap the distance in a ValueProto(float_val=...). When distance is 0.0 (a valid distance meaning an exact match for L2 or cosine metrics), Python evaluates 0.0 as falsy, so the code returns an empty ValueProto() instead of ValueProto(float_val=0.0). Because float_val is inside a oneof val in the proto definition (protos/feast/types/Value.proto:78), ValueProto() leaves the oneof unset (WhichOneof("val") → None), while ValueProto(float_val=0.0) correctly sets it to "float_val". This is the same bug pattern the PR fixes for raw_key at line 694. The postgres online store handles this correctly with if distance is not None: (sdk/python/feast/infra/online_stores/postgres_online_store/postgres.py:777).

(Refers to line 746)

Open in Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.

Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
import base64
import logging
from datetime import datetime
from pathlib import Path
from typing import Any, Callable, Dict, List, Literal, Optional, Sequence, Tuple, Union
Expand Down Expand Up @@ -42,6 +43,8 @@
to_naive_utc,
)

logger = logging.getLogger(__name__)

PROTO_TO_MILVUS_TYPE_MAPPING: Dict[ValueType, DataType] = {
PROTO_VALUE_TO_VALUE_TYPE_MAP["bytes_val"]: DataType.VARCHAR,
ValueType.IMAGE_BYTES: DataType.VARCHAR,
Expand Down Expand Up @@ -140,11 +143,13 @@ def _connect(self, config: RepoConfig) -> MilvusClient:
if not self.client:
if config.provider == "local" and config.online_store.path:
db_path = self._get_db_path(config)
print(f"Connecting to Milvus in local mode using {db_path}")
logger.info("Connecting to Milvus in local mode using %s", db_path)
self.client = MilvusClient(db_path)
else:
print(
f"Connecting to Milvus remotely at {config.online_store.host}:{config.online_store.port}"
logger.info(
"Connecting to Milvus remotely at %s:%s",
config.online_store.host,
config.online_store.port,
)
self.client = MilvusClient(
uri=f"{config.online_store.host}:{config.online_store.port}",
Expand Down Expand Up @@ -339,7 +344,6 @@ def online_read(
table: FeatureView,
entity_keys: List[EntityKeyProto],
requested_features: Optional[List[str]] = None,
full_feature_names: bool = False,
) -> List[Tuple[Optional[datetime], Optional[Dict[str, ValueProto]]]]:
self.client = self._connect(config)
collection_name = _table_id(config.project, table)
Expand Down Expand Up @@ -487,7 +491,7 @@ def update(
):
self.client = self._connect(config)
for table in tables_to_keep:
self._collections = self._get_or_create_collection(config, table)
self._get_or_create_collection(config, table)

for table in tables_to_delete:
collection_name = _table_id(config.project, table)
Expand All @@ -498,7 +502,7 @@ def update(
def plan(
self, config: RepoConfig, desired_registry_proto: RegistryProto
) -> List[InfraObject]:
raise NotImplementedError
return []

def teardown(
self,
Expand Down Expand Up @@ -686,9 +690,8 @@ def retrieve_online_documents_v2(
for hit in hits:
res = {}
res_ts = None
entity_key_bytes = bytes.fromhex(
hit.get("entity", {}).get(composite_key_name, None)
)
raw_key = hit.get("entity", {}).get(composite_key_name)
entity_key_bytes = bytes.fromhex(raw_key) if raw_key else None
entity_key_proto = (
deserialize_entity_key(entity_key_bytes)
if entity_key_bytes
Expand Down
34 changes: 17 additions & 17 deletions sdk/python/feast/openlineage/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -35,27 +35,27 @@
Usage:
Simply configure OpenLineage in your feature_store.yaml:

```yaml
project: my_project
# ... other config ...

openlineage:
enabled: true
transport_type: http
transport_url: http://localhost:5000
transport_endpoint: api/v1/lineage
namespace: my_namespace # Optional: defaults to project name
```
.. code-block:: yaml

project: my_project
# ... other config ...

openlineage:
enabled: true
transport_type: http
transport_url: http://localhost:5000
transport_endpoint: api/v1/lineage
namespace: my_namespace # Optional: defaults to project name

Then use Feast normally - lineage events are emitted automatically!

```python
from feast import FeatureStore
.. code-block:: python

from feast import FeatureStore

fs = FeatureStore(repo_path="feature_repo")
fs.apply([entity, feature_view, feature_service]) # Emits lineage
fs.materialize(start, end) # Emits START/COMPLETE/FAIL events
```
fs = FeatureStore(repo_path="feature_repo")
fs.apply([entity, feature_view, feature_service]) # Emits lineage
fs.materialize(start, end) # Emits START/COMPLETE/FAIL events
"""

from feast.openlineage.client import FeastOpenLineageClient
Expand Down
3 changes: 2 additions & 1 deletion sdk/python/feast/openlineage/client.py
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,8 @@ class FeastOpenLineageClient:
from Feast operations like materialization, feature retrieval, and
registry changes.

Example:
Example::

from feast.openlineage import FeastOpenLineageClient, OpenLineageConfig

config = OpenLineageConfig(
Expand Down
4 changes: 3 additions & 1 deletion sdk/python/feast/openlineage/emitter.py
Original file line number Diff line number Diff line change
Expand Up @@ -644,11 +644,13 @@ def emit_apply(
1. feast_feature_views_{project}: DataSources + Entities → FeatureViews
2. feast_feature_services_{project}: FeatureViews → FeatureServices

This creates a lineage graph matching Feast UI:
This creates a lineage graph matching Feast UI::

DataSource ──→ FeatureView ──→ FeatureService
↑
Entity


Args:
objects: List of Feast objects being applied
project: Project name
Expand Down
2 changes: 1 addition & 1 deletion sdk/python/feast/repo_config.py
Original file line number Diff line number Diff line change
Expand Up @@ -337,7 +337,7 @@ class RepoConfig(FeastBaseModel):
""" MaterializationConfig: Configuration options for feature materialization behavior. """

openlineage_config: Optional[OpenLineageConfig] = Field(None, alias="openlineage")
""" OpenLineageConfig: Configuration for OpenLineage data lineage integration (optional). """
""" Configuration for OpenLineage data lineage integration (optional). """

def __init__(self, **data: Any):
super().__init__(**data)
Expand Down
Loading