Visitar URL original
feat: Add Chronon online and offline store integrations by franciscojavierarceo · Pull Request #6188 · feast-dev/feast · GitHub
Skip to content
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
fix: Make Chronon demo setup reproducible and surface startup failures
Signed-off-by: Francisco Javier Arceo <arceofrancisco@gmail.com>
  • Loading branch information
franciscojavierarceo committed Oct 5, 2026
commit eae98902b0d76bad50592a2d592a3760a1bb65bd
2 changes: 1 addition & 1 deletion .github/workflows/pr_chronon_integration_tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -102,7 +102,7 @@ jobs:
sudo mkdir -p /usr/src/thrift
sudo tar zxf /tmp/thrift.tar.gz -C /usr/src/thrift --strip-components=1
cd /usr/src/thrift
sudo ./configure --without-python --without-cpp
sudo ./configure --disable-libs --disable-tests --disable-tutorial
sudo make -j2
sudo make install
python3 -m pip install --break-system-packages build
Expand Down
3 changes: 3 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -173,6 +173,7 @@ The list below contains the functionality that contributors are planning to deve
* [x] Vector Search (Alpha release. See [RFC](https://docs.google.com/document/d/18IWzLEA9i2lDWnbfbwXnMCg3StlqaLVI-uRpQjr_Vos/edit#heading=h.9gaqqtox9jg6))
* [ ] [Enhanced Feature Server and SDK for native support for NLP](https://github.com/feast-dev/feast/issues/4964)
* **Data Sources**
* [x] [Chronon source (contrib)](https://docs.feast.dev/reference/data-sources/chronon)
* [x] [Snowflake source](https://docs.feast.dev/reference/data-sources/snowflake)
* [x] [Redshift source](https://docs.feast.dev/reference/data-sources/redshift)
* [x] [BigQuery source](https://docs.feast.dev/reference/data-sources/bigquery)
Expand All @@ -189,6 +190,7 @@ The list below contains the functionality that contributors are planning to deve
* [x] [Ray source (contrib plugin)](https://docs.feast.dev/reference/data-sources/ray)
* [x] Kafka / Kinesis sources (via [push support into the online store](https://docs.feast.dev/reference/data-sources/push))
* **Offline Stores**
* [x] [Chronon (contrib)](https://docs.feast.dev/reference/offline-stores/chronon)
* [x] [Snowflake](https://docs.feast.dev/reference/offline-stores/snowflake)
* [x] [Redshift](https://docs.feast.dev/reference/offline-stores/redshift)
* [x] [BigQuery](https://docs.feast.dev/reference/offline-stores/bigquery)
Expand All @@ -209,6 +211,7 @@ The list below contains the functionality that contributors are planning to deve
* [x] [Hybrid](https://docs.feast.dev/reference/offline-stores/hybrid)
* [x] [Custom offline store support](https://docs.feast.dev/how-to-guides/customizing-feast/adding-a-new-offline-store)
* **Online Stores**
* [x] [Chronon](https://docs.feast.dev/reference/online-stores/chronon)
* [x] [Snowflake](https://docs.feast.dev/reference/online-stores/snowflake)
* [x] [DynamoDB](https://docs.feast.dev/reference/online-stores/dynamodb)
* [x] [Redis](https://docs.feast.dev/reference/online-stores/redis)
Expand Down
4 changes: 4 additions & 0 deletions docs/reference/offline-stores/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,10 @@ Please see [Offline Store](../../getting-started/components/offline-store.md) fo
[overview.md](overview.md)
{% endcontent-ref %}

{% content-ref url="chronon.md" %}
[chronon.md](chronon.md)
{% endcontent-ref %}

{% content-ref url="dask.md" %}
[dask.md](dask.md)
{% endcontent-ref %}
Expand Down
7 changes: 6 additions & 1 deletion docs/reference/offline-stores/overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,11 @@ There are currently four core offline store implementations: `DaskOfflineStore`,
There are several additional implementations contributed by the Feast community (`PostgreSQLOfflineStore`, `SparkOfflineStore`, `TrinoOfflineStore`, and `RayOfflineStore`), which are not guaranteed to be stable or to match the functionality of the core implementations.
Details for each specific offline store, such as how to configure it in a `feature_store.yaml`, can be found [here](README.md).

The community [Chronon offline store](chronon.md) reads materialized Parquet with
point-in-time joins and supports local on-demand transforms and Parquet saved
datasets. Computation and materialization remain managed by Chronon; see its
[functionality matrix](chronon.md#functionality-matrix) for supported operations.

Below is a matrix indicating which offline stores support which methods.

|| | Dask | BigQuery | Snowflake | Redshift | Postgres | Spark | Trino | Couchbase | Ray |
Expand All @@ -55,4 +60,4 @@ Below is a matrix indicating which `RetrievalJob`s support what functionality.
|| remote execution of Python-based on-demand transforms | no | no | no | no | no | no | no | no | no | no |
|| persist results in the offline store | yes | yes | yes | yes | yes | yes | no | yes | yes | yes |
|| preview the query plan before execution | yes | yes | yes | yes | yes | yes | yes | no | yes | yes |
|| read partitioned data | yes | yes | yes | yes | yes | yes | yes | yes | yes | yes |
|| read partitioned data | yes | yes | yes | yes | yes | yes | yes | yes | yes | yes |
4 changes: 4 additions & 0 deletions docs/reference/online-stores/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,10 @@ Please see [Online Store](../../getting-started/components/online-store.md) for
[overview.md](overview.md)
{% endcontent-ref %}

{% content-ref url="chronon.md" %}
[chronon.md](chronon.md)
{% endcontent-ref %}

{% content-ref url="sqlite.md" %}
[sqlite.md](sqlite.md)
{% endcontent-ref %}
Expand Down
5 changes: 5 additions & 0 deletions examples/chronon/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,11 @@ uv run python examples/chronon/run_demo.py --offline-only

## Live Chronon Service

The pinned quickstart Docker image is ARM64. Use an ARM64 Docker host (as in
the integration workflow), or provide a compatible image for your host with
`CHRONON_QUICKSTART_IMAGE`. Docker must be running and accessible to your user.
The launcher uses MongoDB 7.0; `CHRONON_MONGO_IMAGE` overrides that image.

Install `sbt` and a JDK. On macOS with Homebrew, this is enough:

```bash
Expand Down
11 changes: 9 additions & 2 deletions infra/scripts/chronon/start-local-chronon-service.sh
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,8 @@ SERVICE_PID_FILE="${CHRONON_SERVICE_PID_FILE:-/tmp/chronon-service.pid}"
SERVICE_LOG_FILE="${CHRONON_SERVICE_LOG_FILE:-/tmp/chronon-service.log}"
JAVA_BIN="${JAVA_BIN:-java}"
PYTHON_BIN="${PYTHON_BIN:-python3}"
QUICKSTART_IMAGE="${CHRONON_QUICKSTART_IMAGE:-ezvz/chronon@sha256:a3f6b5e1aa85bb23269ebd67fc7e4b0fb2d0cf3e53bc2a45abdc23c95769415c}"
MONGO_IMAGE="${CHRONON_MONGO_IMAGE:-mongo:7.0@sha256:1f995ad6fdb93244a1addab1b58f934a0bc2f5643c38e02f5e9d7f0c7d227a7b}"
CHRONON_PREFLIGHT_ONLY="${CHRONON_PREFLIGHT_ONLY:-0}"
MONGO_IMPL_JAR="${CHRONON_DIR}/quickstart/mongo-online-impl/target/scala-2.12/mongo-online-impl-assembly-0.1.0-SNAPSHOT.jar"
SERVICE_TARGET_DIR="${CHRONON_DIR}/service/target/scala-2.12"
Expand Down Expand Up @@ -94,6 +96,11 @@ cleanup_stale_service() {
wait_for_mongo() {
local attempts=60
until docker exec "${MONGO_CONTAINER}" mongosh --quiet --eval 'db.runCommand({ ping: 1 }).ok' >/dev/null 2>&1; do
if [[ "$(docker inspect --format '{{.State.Running}}' "${MONGO_CONTAINER}")" != "true" ]]; then
echo "Mongo container exited before becoming ready." >&2
docker logs "${MONGO_CONTAINER}" >&2 || true
exit 1
fi
attempts=$((attempts - 1))
if [[ "${attempts}" -le 0 ]]; then
echo "Mongo did not become ready in time." >&2
Expand Down Expand Up @@ -163,7 +170,7 @@ docker run -d \
-p 27017:27017 \
-e MONGO_INITDB_ROOT_USERNAME=admin \
-e MONGO_INITDB_ROOT_PASSWORD=admin \
mongo:latest >/dev/null
"${MONGO_IMAGE}" >/dev/null

wait_for_mongo

Expand All @@ -184,7 +191,7 @@ docker run -d \
-e CHRONON_ONLINE_CLASS=ai.chronon.quickstart.online.ChrononMongoOnlineImpl \
-e "CHRONON_ONLINE_ARGS=-Zuser=admin -Zpassword=admin -Zhost=${MONGO_CONTAINER} -Zport=27017 -Zdatabase=admin" \
-v "${CHRONON_DIR}/quickstart/mongo-online-impl:/srv/onlineImpl" \
ezvz/chronon \
"${QUICKSTART_IMAGE}" \
bash -lc '/opt/spark/bin/spark-shell -i scripts/data-loader.scala && touch /tmp/chronon-data-load.complete && tail -f /dev/null' >/dev/null

wait_for_data_load
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -43,9 +43,9 @@ def test_start_script_can_fail_before_docker_when_repo_missing(tmp_path: Path):
assert "git clone" in result.stderr


@pytest.mark.parametrize("loader_fails", [False, True])
def test_start_script_waits_for_completed_data_load(
tmp_path: Path, loader_fails: bool
@pytest.mark.parametrize("failed_container", ["", "chronon-main", "chronon-mongo"])
def test_start_script_requires_ready_containers(
tmp_path: Path, failed_container: str
) -> None:
chronon_repo = tmp_path / "chronon"
jar = (
Expand All @@ -68,17 +68,20 @@ def test_start_script_waits_for_completed_data_load(

args = sys.argv[1:]
state = Path(os.environ['TEST_DOCKER_STATE'])
failed = os.environ['TEST_LOADER_FAILS'] == '1'
failed_container = os.environ['TEST_FAILED_CONTAINER']
failed = failed_container == 'chronon-main'
if args[0] == 'run' and 'chronon-main' in args:
state.write_text('0')
elif args[:2] == ['exec', 'chronon-mongo']:
sys.exit(1 if failed_container == 'chronon-mongo' else 0)
elif args[:2] == ['logs', 'chronon-main']:
print("Spark session available as 'spark'.")
elif args[:3] == ['exec', 'chronon-main', 'test']:
polls = int(state.read_text()) + 1
state.write_text(str(polls))
sys.exit(0 if polls >= 2 and not failed else 1)
elif args[0] == 'inspect':
print('false' if failed else 'true')
print('false' if args[-1] == failed_container else 'true')
elif args[:3] == ['exec', 'chronon-main', 'bash']:
if failed or int(state.read_text()) < 2:
print('Upload started before successful data load', file=sys.stderr)
Expand All @@ -103,17 +106,22 @@ def test_start_script_waits_for_completed_data_load(
"CHRONON_PREFLIGHT_ONLY": "0",
"PYTHON_BIN": "/usr/bin/true",
"TEST_DOCKER_STATE": str(state),
"TEST_LOADER_FAILS": "1" if loader_fails else "0",
"TEST_FAILED_CONTAINER": failed_container,
},
capture_output=True,
text=True,
timeout=15,
check=False,
)
if loader_fails:
if failed_container:
assert result.returncode == 1
assert not state.with_suffix(".uploaded").exists()
assert "data loader exited" in result.stderr
expected = (
"data loader exited"
if failed_container == "chronon-main"
else "Mongo container exited"
)
assert expected in result.stderr
else:
assert result.returncode == 0, result.stderr
assert state.with_suffix(".uploaded").exists()
Expand Down
Loading