Repository navigation
Deployment stuck indefinitely when scaling during rolling update with maxSurge=0 due to stale annotations #135483
Description
What happened?
When a Deployment is scaled down during an in-progress rolling update (with maxSurge=0),
the deployment controller gets stuck in an infinite loop and the rolling update never completes.
The issue occurs because:
- During a rolling update, ReplicaSet annotations (deployment.kubernetes.io/desired-replicas)
are set to the initial deployment size - If the deployment is scaled down while the update is in progress, the replica count changes
but the annotations remain stale - isScalingEvent() detects the annotation mismatch and returns true, causing the controller
to call sync() instead of rolloutRolling() - scale() checks if replica counts match and returns early without updating annotations
(lines 303-304 in pkg/controller/deployment/sync.go) - scaleReplicaSetAndRecordEvent() also has an early return when replicas match (lines 400-401)
- The annotations are never updated, creating an infinite loop
This prevents the rolling update from proceeding. Pods remain on the old version indefinitely, and there are no error messages.
Impact: Applications cannot be updated, security patches cannot be deployed, and the
deployment appears healthy while being completely stuck.
What did you expect to happen?
The deployment controller should update ReplicaSet annotations even when replica counts
already match the desired state. After scaling during a rolling update, the controller should:
- Detect the annotation mismatch
- Update the stale annotations to match the current deployment.spec.replicas
- Continue with the rolling update by calling rolloutRolling()
- Create a new ReplicaSet with the updated template
- Complete the rolling update successfully
The annotations should be kept in sync with deployment.spec.replicas regardless of whether
the replica count changed, to ensure isScalingEvent() correctly identifies true scaling
events versus stale annotation states.
How can we reproduce it (as minimally and precisely as possible)?
Automated Reproduction
Run this automated verification script which reproduces the bug from scratch:
curl -fsSL https://gist.githubusercontent.com/SDkie/eb9527f434b5556fdf77f9a4207c56fc/raw/b9195d738eb5b99b8f1931818699ebe0378b9be9/gistfile1.txt | bashThe script will:
- Create a kind cluster
- Deploy a test application with maxSurge=0, maxUnavailable=1
- Simulate the stale annotation condition
- Trigger a rolling update
- Demonstrate that the update is blocked
Anything else we need to know?
No response
Kubernetes version
- Tested on self hosted cluster on 1.29 version
- Tested on kind cluster on 1.32 version
Issue still exists in the latest code.
Cloud provider
- Tested on self hosted / self managed Uber clusters
(I work in Uber compute team)
OS version
No response
Install tools
No response
Container runtime (CRI) and version (if applicable)
No response
Related plugins (CNI, CSI, ...) and versions (if applicable)
No response
Activity
k8s-ci-robot commentedon Nov 27, 2025
This issue is currently awaiting triage.
If a SIG or subproject determines this is a relevant issue, they will accept it by applying the
triage/acceptedlabel and provide further guidance.The
triage/acceptedlabel can be added by org members by writing/triage acceptedin a comment.Details
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.
P-Sreeshanth commentedon Nov 28, 2025
/sig apps
k8s-triage-robot commentedon Feb 26, 2026
The Kubernetes project currently lacks enough contributors to adequately respond to all issues.
This bot triages un-triaged issues according to the following rules:
lifecycle/staleis appliedlifecycle/stalewas applied,lifecycle/rottenis appliedlifecycle/rottenwas applied, the issue is closedYou can:
/remove-lifecycle stale/closePlease send feedback to sig-contributor-experience at kubernetes/community.
/lifecycle stale
deployment: fix stuck rollout when scaling during rolling update with…
deployment: fix stuck rollout when scaling during rolling update with…
deployment: fix stuck rollout when scaling during rolling update with…
deployment: fix stuck rollout when scaling during rolling update with…
k8s-triage-robot commentedon Mar 28, 2026
The Kubernetes project currently lacks enough active contributors to adequately respond to all issues.
This bot triages un-triaged issues according to the following rules:
lifecycle/staleis appliedlifecycle/stalewas applied,lifecycle/rottenis appliedlifecycle/rottenwas applied, the issue is closedYou can:
/remove-lifecycle rotten/closePlease send feedback to sig-contributor-experience at kubernetes/community.
/lifecycle rotten
k8s-triage-robot commentedon Apr 27, 2026
The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.
This bot triages issues according to the following rules:
lifecycle/staleis appliedlifecycle/stalewas applied,lifecycle/rottenis appliedlifecycle/rottenwas applied, the issue is closedYou can:
/reopen/remove-lifecycle rottenPlease send feedback to sig-contributor-experience at kubernetes/community.
/close not-planned
k8s-ci-robot commentedon Apr 27, 2026
@k8s-triage-robot: Closing this issue, marking it as "Not Planned".
Details
In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.
fix(n8n): keep default Helm values byte-identical in multi-main mode
fix(licensing): support single-main topology for Business-tier licens…