Visitar URL original
Document component state migration (#24310) · pulumi/pulumi@de4a60a · GitHub
Skip to content

Commit de4a60a

Browse files
authored
Document component state migration (#24310)
Fixes #24324
1 parent 098d56d commit de4a60a

3 files changed

Lines changed: 337 additions & 1 deletion

File tree

‎docs/architecture/deployment-execution/README.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -113,4 +113,5 @@ following pages.
113113
/docs/architecture/deployment-execution/import
114114
/docs/architecture/deployment-execution/steps
115115
/docs/architecture/deployment-execution/callbacks
116+
/docs/architecture/deployment-execution/state-migrations
116117
:::

‎docs/architecture/deployment-execution/callbacks.md‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@
33

44
The *callback system* allows programs, or even component libraries, to expose user-authored functions to the engine and call them across process boundaries. These functions are defined alongside the rest of the Pulumi program, and run in the same process, but are typically not called directly by user code. Instead they are called by the engine at specific moments of the deployment. This mechanism enables features like [resource transforms](https://www.pulumi.com/docs/concepts/options/transforms/) and [resource hooks](https://www.pulumi.com/docs/iac/concepts/resources/options/hooks/).
55

6-
The language SDKs start a gRPC server implementing the [](pulumirpc.Callbacks) service (for example, [_callbacks.py](gh-file:pulumi#sdk/python/lib/pulumi/runtime/_callbacks.py#L68)). This server exposes a single method: [](pulumirpc.Callbacks.Invoke). User-authored functions are registered with this server, and the engine is informed of their existence by calling [](pulumirpc.ResourceMonitor.RegisterStackTransform) or [](pulumirpc.ResourceMonitor.RegisterResourceHook).
6+
The language SDKs start a gRPC server implementing the [](pulumirpc.Callbacks) service (for example, [_callbacks.py](gh-file:pulumi#sdk/python/lib/pulumi/runtime/_callbacks.py#L68)). This server exposes a single method: [](pulumirpc.Callbacks.Invoke). User-authored functions are registered with this server, and a callback reference is passed to the engine through the relevant resource-monitor request, such as [](pulumirpc.ResourceMonitor.RegisterStackTransform), [](pulumirpc.ResourceMonitor.RegisterResourceHook), or [](pulumirpc.ResourceMonitor.RegisterResource).
77

88
The [](pulumirpc.Callback) message is used to register a callback. It represents a callback reference and contains two essential pieces of information:
99

@@ -26,3 +26,4 @@ The system supports multiple callback types, each with their own request/respons
2626
* **Resource transforms**: [](pulumirpc.TransformRequest) and [](pulumirpc.TransformResponse)
2727
* **Invoke transforms**: [](pulumirpc.TransformInvokeRequest) and [](pulumirpc.TransformInvokeResponse)
2828
* **Resource hooks**: [](pulumirpc.ResourceHookRequest) and [](pulumirpc.ResourceHookResponse)
29+
* **[Component state migrations](/docs/architecture/deployment-execution/state-migrations.md)**: [](pulumirpc.StateMigrationRequest) and [](pulumirpc.StateMigrationResponse)
Lines changed: 334 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,334 @@
1+
(state-migrations)=
2+
# Component state migrations
3+
4+
Component state migrations let a component reshape its persisted resource state
5+
before the engine compares it with the component's current registrations. A
6+
migration can rename, split, merge, or otherwise reorganize the component and
7+
its descendants without treating the change as unrelated deletes and creates.
8+
9+
Migrations change Pulumi's state, they do not call providers. A managed custom
10+
resource and its successor must retain the same physical ID, provider,
11+
extension, ownership, and provider lifecycle state.
12+
13+
For example, suppose component `Root` previously contained `OldChild`, but its
14+
new implementation replaces that child with `NewChild`. A migration returns the
15+
updated subtree `[Root, NewChild]` and identifies `NewChild` as `OldChild`'s
16+
successor. The engine can then rewrite references from `OldChild` to `NewChild`
17+
without deleting and recreating the physical resource.
18+
19+
## How migrations run
20+
21+
Migrations use the engine's
22+
[callback system](/docs/architecture/deployment-execution/callbacks.md). The
23+
callbacks run in the language SDK process, and a resource registration can
24+
carry an ordered list of them. While planning that registration, the step
25+
generator:
26+
27+
1. Resolves the prior subtree root using the registration URN and its aliases.
28+
2. Captures that root and its transitive descendants from
29+
`Deployment.prev.Resources` in snapshot order.
30+
3. Converts those `pkgresource.State` values to `ResourceV3` checkpoint JSON and
31+
invokes each callback in order.
32+
4. Converts the result back to resource state and validates the returned
33+
subtree and successor mappings.
34+
5. Prepares the complete post-migration prior-resource list, topologically sorts
35+
it, persists it, and commits one `StateMigrationTransaction`.
36+
6. Retains the successor rewrite for references the program may send after the
37+
commit.
38+
7. Normalizes the registration goal using the saved rewrites.
39+
8. Resumes planning the registration against the migrated state.
40+
41+
## Callback contract
42+
43+
In simplified form, the Python SDK callback types and a synchronous migration
44+
have this shape:
45+
46+
```python
47+
class StateMigrationArgs:
48+
urn: str
49+
old_state: list[dict[str, Any]]
50+
51+
class StateMigrationResult:
52+
new_state: list[dict[str, Any]]
53+
successors: dict[str, str] | None
54+
55+
def migrate(args: StateMigrationArgs) -> StateMigrationResult | None:
56+
return StateMigrationResult(
57+
new_state=[...],
58+
successors={old_urn: new_urn},
59+
)
60+
```
61+
62+
An asynchronous migration may return the same result from an `Awaitable`.
63+
64+
`args.urn` is the URN of the current registration. `args.old_state` contains the
65+
prior root and all of its descendants in checkpoint format.[^aliased-root]
66+
67+
Returning `None` means that the state is already in the desired shape.
68+
Otherwise, `new_state` replaces that complete prior subtree and `successors`
69+
maps each omitted prior URN to a URN present in `new_state`. The engine also
70+
treats a result that represents the same state as the input as a no-op. No-op
71+
migrations skip persistence and commit.
72+
73+
Migration callbacks remain attached to the resource after a stack has migrated,
74+
so they must be idempotent. A callback should recognize state in its final shape
75+
and return `None`, so later updates do not repeat the state change.
76+
77+
A callback must only transform the supplied checkpoint data. It must not perform
78+
Pulumi runtime operations[^monitor-reentrancy] or wait for unresolved `Output`
79+
values. The resource monitor waits for the callback to return, so work that
80+
calls back into the monitor would deadlock.
81+
82+
Any secrets in the supplied checkpoint data are decrypted before the callback
83+
runs. The callback must treat those values as sensitive.
84+
85+
[^aliased-root]:
86+
When the engine resolves the prior root through an alias, `args.urn`
87+
identifies the current registration while `args.old_state[0]["urn"]` keeps
88+
the matched prior resource's URN. The two URNs may therefore differ.
89+
90+
[^monitor-reentrancy]:
91+
Where possible, language SDKs should detect and reject attempts to call back
92+
into the resource monitor from a migration callback, producing a useful
93+
error instead of allowing the program to deadlock.
94+
95+
## Results and successors
96+
97+
Every resource in the prior subtree must either remain in the result or map to a
98+
successor in the result. This makes removal explicit: accidentally omitting a
99+
resource cannot silently forget a physical object that Pulumi manages.
100+
101+
Successors also tell the engine how to rewrite references. For example, if
102+
`OldChild` is replaced by `NewChild`, the callback maps `OldChild` to
103+
`NewChild`. References to `OldChild` in parents, dependencies, typed resource
104+
references, output dependencies, and other reference-bearing state fields are
105+
rewritten to `NewChild`. Plain strings that happen to contain a URN are not
106+
interpreted as references.
107+
108+
Successor mappings compose across callbacks. For example, if one callback maps
109+
`OldChild` to `IntermediateChild` and the next maps `IntermediateChild` to
110+
`NewChild`, the engine rewrites references from `OldChild` directly to
111+
`NewChild`.
112+
113+
Before invoking the next callback, the engine rewrites references within the
114+
returned subtree using the accumulated successor mappings. For example, after
115+
a callback renames a component, the next callback sees its children parented to
116+
the new component URN. The complete callback chain is still persisted as one
117+
transaction and intermediate results are not persisted separately.
118+
119+
A migration can also split inline settings out of a managed resource into a
120+
separate managed resource:
121+
122+
```text
123+
resource (inline settings) -> resource (settings removed)
124+
+ settings (same physical ID and provider)
125+
```
126+
127+
The callback retains `resource` at its existing URN and returns the additional
128+
`settings` state with the same physical ID, provider, and extension identity.
129+
No successor mapping is needed: references to `resource` still refer to that
130+
resource. The callback supplies the settings resource's inputs, outputs, and
131+
dependencies. The migration itself does not create or read a provider resource.
132+
133+
## Safety validation
134+
135+
Before preparing a transaction, the engine validates the callback result as a
136+
complete replacement subtree:
137+
138+
* It must contain exactly one resource at the top of the replacement subtree:
139+
either the current registration (`args.urn`) or its matched prior resource
140+
(`args.old_state[0]["urn"]`), but not both.
141+
* That resource must keep its prior parent, and every other returned resource
142+
must be its descendant.
143+
* Returned URNs must remain in the registration's stack and project, and all
144+
structural references must resolve in either the returned subtree or the
145+
retained prior state.
146+
* Provider resource states must be returned unchanged. A checkpoint rewrite
147+
cannot update the engine's live provider registry.
148+
* Every returned custom resource must have a retained or explicitly mapped
149+
custom predecessor, or match prior custom state in the migrated subtree by
150+
non-empty physical ID, provider, and extension identity. This last case allows
151+
additional resources produced by a split.
152+
* A custom resource and its successor must preserve physical ID, provider and
153+
extension identity, `External` ownership, `PendingReplacement`, and `Taint`.
154+
`Protect` and `RetainOnDelete` are inherited conservatively: a many-to-one
155+
successor carries either flag if any predecessor had it.
156+
* For an additional custom resource without a retained or explicitly mapped
157+
predecessor, ownership and lifecycle flags must agree with every matching
158+
prior custom resource. Conflicting matches are rejected, not ignored.
159+
`Protect` and `RetainOnDelete` must each be set if any match carries that flag.
160+
161+
Two custom resources can share a provider and physical ID while representing
162+
different parts of the same object. The engine cannot tell whether such a
163+
cross-type merge, rename, or split is correct. The callback must choose the
164+
right successors and supply state compatible with each resource's provider
165+
schema. Matching physical identity alone does not establish that the extracted
166+
settings state is valid for the new resource type.
167+
168+
## When migrations can run
169+
170+
Migrations are considered during updates and update previews when prior state
171+
exists. Migration callbacks do not run during refresh or destroy, even with
172+
`--run-program`.[^non-update-operations] The selected persistence mode must also
173+
support state migrations; full-checkpoint snapshot persistence and journal v2
174+
do, while older journal formats do not.
175+
176+
A state-changing result is rejected when the surrounding update cannot safely
177+
rewrite the complete checkpoint. This includes:
178+
179+
* updates constrained by `--target`, `--replace`, `--exclude`, or
180+
`--target-snippet`
181+
* generating or applying a saved update plan
182+
* snapshots with pending operations, or migrated subtrees with resources
183+
pending deletion from an earlier update
184+
* migrations involving resource views
185+
* persisted snippets whose references would need to be rewritten
186+
187+
These restrictions apply only when a callback changes state. A permanently
188+
attached callback can return `None` without blocking partial updates, plan use,
189+
or recovery of pending state.
190+
191+
[^non-update-operations]:
192+
Migrations reshape prior state before update planning. Refresh instead
193+
reconciles that state with providers, while destroy removes it.
194+
`--run-program` evaluates the program but does not turn either operation
195+
into an update.
196+
197+
## Preparing the transaction
198+
199+
The engine validates and prepares a
200+
[`StateMigrationTransaction`](https://github.com/pulumi/pulumi/blob/7551227c26a1a506f44dc19ef5dad91acd15a1c9/pkg/resource/deploy/state_migration_models.go#L60)
201+
before mutating shared deployment state. Suppose
202+
`Deployment.prev.Resources` is:
203+
204+
```text
205+
[Provider, Root, OldChild, Consumer, Tail]
206+
```
207+
208+
`Root` is the component root, and `OldChild` and `Tail` are its children.
209+
`Consumer` is outside the migrated subtree but is interleaved with its
210+
resources: `Consumer` depends on `OldChild`, and `Tail` depends on `Consumer`.
211+
The callback returns `[Root, NewChild, Tail]` and maps `OldChild` to `NewChild`.
212+
213+
The replacement subtree cannot be inserted as one contiguous block. Placing it
214+
at the old subtree's last position would put `Consumer` before its new
215+
dependency `NewChild`, while placing it at the first position would put `Tail`
216+
before its dependency `Consumer`. The engine rewrites `Consumer`, builds the
217+
complete candidate snapshot, and topologically sorts all of its resources. The
218+
prepared resource values are:
219+
220+
```text
221+
[Provider, Root, NewChild, Consumer′, Tail]
222+
```
223+
224+
where `Consumer′` is a prepared copy of `Consumer` whose dependency refers to
225+
`NewChild`. If this resource graph cannot be sorted, preparation fails without
226+
persisting or committing the migration. Otherwise, the transaction records
227+
both the values to persist and the information needed to commit them without
228+
replacing live resource pointers:
229+
230+
| Field | Value in the example | Purpose |
231+
|---|---|---|
232+
| `PriorSubtree` | `[Root, OldChild, Tail]` | Live states from `Deployment.prev.Resources` replaced by the migration |
233+
| `ResultSubtree` | `[Root, NewChild, Tail]` | Replacement subtree |
234+
| `SuccessorURNs` | `{OldChild: NewChild}` | Maps removed resources to their successors |
235+
| `PreparedPriorResources` | `[Provider, Root, NewChild, Consumer′, Tail]` | Topologically sorted replacement for `Deployment.prev.Resources` |
236+
| `RetainedResourceRewrites` | `{Consumer: Consumer′}` | Prepared values to copy into retained live objects |
237+
| `currentResourceRewrites` | As needed | Prepared values for earlier `Deployment.news` and `Deployment.reads` states |
238+
239+
`Deployment.prev.Resources` and other deployment bookkeeping may refer to the
240+
same `*pkgresource.State` object. If the engine replaced `Consumer` in
241+
`Deployment.prev.Resources` with the new `Consumer′` object, other structures
242+
could still refer to the original `Consumer` and see its old references. The
243+
engine therefore validates and persists `Consumer′`, then copies its fields
244+
into the original `Consumer`. It sets `Deployment.prev.Resources` to
245+
`[Provider, Root, NewChild, Consumer, Tail]` and rebuilds the deployment
246+
indexes. The persisted and in-memory states have the same values while shared
247+
pointer identity is preserved.
248+
249+
## Persistence
250+
251+
During a real update, persistence happens before the engine changes
252+
`Deployment.prev` or any other shared resource state. If persistence fails, the
253+
migration returns an error and the live deployment remains unchanged. A preview
254+
performs the same preparation and in-memory commit without durable persistence.
255+
256+
Once a migration is persisted, it is not rolled back if a later part of the
257+
update fails. The next update starts from the migrated state, so migration
258+
callbacks must be idempotent.
259+
260+
The full-checkpoint snapshot manager combines `PreparedPriorResources` with its
261+
current operation state, then saves the result through the usual persistence
262+
path.
263+
264+
The journal manager starts with an immutable base snapshot and records ordered
265+
operations on top of it. A migration may change both the base and resources
266+
produced by earlier journal entries, so journal v2 records one migration entry
267+
containing:
268+
269+
* `Layout`: the complete post-migration base snapshot order. Each item refers
270+
either to a retained resource by its index in the original base snapshot or
271+
to an inserted resource by its index in `States`. Base resources omitted from
272+
the layout are removed.
273+
* `States`: the complete replacement subtree. `stateIndex` items in `Layout`
274+
refer to resources in this list.
275+
* `BaseStatePatches`: complete replacements for retained base states whose
276+
references changed, addressed by their original base-snapshot index.
277+
* `NewStatePatches`: complete replacements for earlier operation results,
278+
addressed by operation ID.
279+
280+
These patches are complete resource states, not property-level diffs.
281+
282+
For the example above, the immutable base snapshot assigns indices `0` through
283+
`4` to `[Provider, Root, OldChild, Consumer, Tail]`. The migration entry has
284+
this shape:
285+
286+
```json
287+
{
288+
"kind": "state-migration",
289+
"states": ["Root", "NewChild", "Tail"],
290+
"layout": [
291+
{"baseIndex": 0},
292+
{"stateIndex": 0},
293+
{"stateIndex": 1},
294+
{"baseIndex": 3},
295+
{"stateIndex": 2}
296+
],
297+
"baseStatePatches": [
298+
{"index": 3, "state": "Consumer′"}
299+
]
300+
}
301+
```
302+
303+
The strings stand in for complete serialized resource states. Replay applies
304+
the patches and walks `Layout` to reconstruct
305+
`[Provider, Root, NewChild, Consumer′, Tail]` exactly; the omitted base indices
306+
for `Root`, `OldChild`, and `Tail` remove the prior subtree.
307+
308+
If the journal entry is acknowledged before a crash, replay recovers the exact
309+
prepared result. If persistence fails, there is no migration to commit.
310+
311+
## References sent after commit
312+
313+
Planning and step execution overlap while the program runs. Before a migration,
314+
the engine stops accepting source events, lets earlier planning and step chains
315+
finish, and blocks new execution while the migration commits. The transaction
316+
rewrites all engine-held state before work resumes, so engine-generated
317+
continuations do not need separate normalization.
318+
319+
The program may still hold a reference created before the migration and send it
320+
to the engine afterward. The deployment keeps a small `stateMigrationRewrite`
321+
with the successor URNs and physical identity needed to normalize such values.
322+
`ReadResource` and `RegisterResourceOutputs` events are normalized when they
323+
enter the engine. Registration goals are normalized later, after applying any
324+
migrations carried by that registration, because its own migration may create
325+
the required successor mapping.
326+
327+
URNs used for identity lookup are handled separately. A registration's parent
328+
is rewritten before generating its URN, and its aliases are rewritten before
329+
looking up prior state. The predecessor URN cannot be reused as the primary URN
330+
of a new registration or read during the same deployment. References to that
331+
URN now mean the successor, so reusing it would be ambiguous.
332+
333+
Providers and analyzers receive normalized inputs and old state and are expected
334+
to derive resource references from those values.

0 commit comments

Comments
 (0)