Python: [Bug]: An unreadable memory state file fails every later turn #9997

Observed Behavior

MemoryFileStore.read_state parses the owner's state.json on every memory bookkeeping path without handling malformed JSON or non-UTF-8 bytes. If an interrupted editor, partial external write, or manual edit leaves the file unreadable, read_state raises on that turn and every later turn because the same file remains in place.

This is different from a well-formed JSON document with the wrong top-level type. That case already raises a deliberate ValueError and is not the subject of this report.

Expected Behavior

An unreadable maintenance-state file should not permanently disable memory bookkeeping. The recovery policy should preserve the original bytes for diagnosis and either quarantine the bad file and restart with clean maintenance state, or expose another explicit recoverable path chosen by the maintainers.

The session store already uses a corrupt-snapshot quarantine convention. Applying the same convention here would keep recovery behavior consistent, but I would like confirmation that silently resetting consolidation bookkeeping is acceptable for memory state.

Steps to Reproduce

1. Create a MemoryFileStore and write valid state for an owned session.
2. Replace its state.json with malformed JSON such as {not json.
3. Call read_state twice for the same session and source.

Minimal Reproduction

No model or network call is involved.

import tempfile
from pathlib import Path

from agent_framework import AgentSession, MemoryFileStore
from agent_framework._harness._memory import DEFAULT_MEMORY_SOURCE_ID

with tempfile.TemporaryDirectory() as directory:
    store = MemoryFileStore(Path(directory), owner_state_key="owner_id")
    session = AgentSession(session_id="session-1")
    session.state["owner_id"] = "alice"
    store.write_state(
        session,
        {"sessions_since_consolidation": ["session-1"]},
        source_id=DEFAULT_MEMORY_SOURCE_ID,
    )
    state_path = store._get_state_path(session, source_id=DEFAULT_MEMORY_SOURCE_ID)
    state_path.write_text("{not json", encoding="utf-8")

    store.read_state(session, source_id=DEFAULT_MEMORY_SOURCE_ID)
    store.read_state(session, source_id=DEFAULT_MEMORY_SOURCE_ID)

Both reads raise json.JSONDecodeError. A non-UTF-8 payload similarly raises UnicodeDecodeError on every read.

Package Versions

Reproduced against the core source used for #9547 and rechecked against the current implementation shape on main. No live model, remote store, or external service is required.

Python Version

Python 3.14.2.

Operating System

macOS.

Regression

Unknown.

Additional Context

Candidate PR #9547 already contains an implementation and regression coverage from earlier work. It quarantines malformed JSON and non-UTF-8 state beside the original, returns clean maintenance state, and rechecks the bytes before renaming so a valid replacement landing during the read is not quarantined. The deliberate non-object ValueError remains unchanged.

No further implementation is planned until the desired recovery policy is agreed. If preserving the bad file but failing closed is preferred over resetting maintenance state, the candidate should be adjusted rather than merged as written.

AI Assistance

AI-assisted analysis, reproduction, implementation, and writing. The candidate diff and tests were reviewed and run locally.

Acknowledgements

• I searched existing issues and did not find a duplicate.
• I personally verified this behavior and the reproduction details are authentic.
• I will wait for explicit maintainer agreement before starting implementation of a non-trivial change.

The implementation in #9547 predates this tracking issue. Further work is paused pending maintainer agreement on the recovery direction.