Use annotations instead of labels for Compose-managed container runtime state #14050

Context

Spun off from #13959 (comment) (@thaJeztah):

compose is already heavily depending on labels to store state, but we should start looking if we could transition to using Annotations, especially for state that shouldn't be persisted (doing a docker commit on a container, or running a container from an image that already has labels set could probably still provide "interesting" results)

Problem

Compose stores both identity and runtime state as container labels (com.docker.compose.project, service, container-number, oneoff, config-hash, image, depends_on, replace, version, project.working_dir, project.config_files, environment_file, …). Labels have two persistence problems annotations don't have:

• docker commit on a Compose-managed container bakes all of this state into the resulting image;
• containers inherit image labels, so running a container from such an image (or any image shipping com.docker.compose.* labels) makes it look like Compose-managed state: wrong config-hash comparisons, orphan detection, ps/down matching, etc.

HostConfig.Annotations is per-container, non-identifying runtime metadata — exactly the right home for state that must never leak into images.

Proposal

Split the current label set by role:

• Runtime state → annotations: config-hash, image, depends_on, replace, container-number, oneoff, version, project.working_dir, project.config_files, environment_file — none of these should survive a docker commit.
• Identity stays as labels (for now): project and service are used in server-side filters (docker ps --filter label=…, events, networks/volumes listing); the Engine API has no annotation-based filtering, and annotations don't exist on networks/volumes/images at all. Per the original comment, containerd namespaces may eventually be the right primitive for project scoping — that's engine-side work to track separately.

Backward compatibility

Migration must be invisible to users with running stacks:

1. Phase 1 (dual write, fallback read): new containers get both labels and annotations; all readers prefer the annotation and fall back to the label. Stacks created by older Compose keep working (up recreates, down, ps).
2. Phase 2 (annotation-first): stop writing the state labels on new containers; keep the fallback read path for at least a major-version window.
3. Filters keep using the identity labels throughout; nothing changes for docker ps --filter label=com.docker.compose.project=… consumers, and third-party tooling relying on identity labels is unaffected.

A container recreated by a newer Compose gets the new layout; mixed projects (old + new containers) must reconcile correctly during phase 1, which the fallback read gives us.

Open questions

• Exact split for container-number/oneoff: they are identity-adjacent but not used in server-side filters everywhere — audit each call site.
• Annotations are only available for containers: networks, volumes and images keep labels regardless (their divergence-detection labels like com.docker.compose.network/volume and config-hash stay as-is).
• Whether third-party ecosystems read the state labels (not just identity) — a deprecation note in release notes should cover phase 2.

Known direct label consumers

Compose labels were never a documented API, yet some external tools are known to read com.docker.compose.* labels directly from docker ps output instead of using compose ps or another supported interface — Docker Desktop has been identified as one such consumer. These consumers will need to be updated; the phase 1 dual-write window is their migration period, and does not turn the state labels into a compatibility commitment.

Blocker: the Engine API has no annotation filter

ContainerList returns annotations since Engine v27.0.0 (moby/moby ca0529f984, HostConfig.Annotations in each summary — verified empirically against v29.6.1 / API 1.55), so list-driven reads keep working once state moves to annotations. But server-side filtering by annotation does not exist (invalid filter 'annotation'; label only). This is a blocker for moving any filtered label, and Compose filters on more than identity today:

Consequence: two labels from the runtime state basket are filtered on today. Migrating them requires either an annotation filter landing engine-side, or Compose first rewriting those filter usages — the config-hash presence marker can be replaced by the project label presence, and the container-number equality by client-side selection over a service-scoped list. Both rewrites are Compose-local and should land as a phase 1 prerequisite; identity labels (project, service, oneoff, network) stay labels regardless until the engine offers annotation filtering.