Visitar URL original
fix: preserve kickstart gzip fallback for curl HTTP errors by Shubham-Padkonde · Pull Request #24048 · netdata/netdata · GitHub
Skip to content

fix: preserve kickstart gzip fallback for curl HTTP errors - #24048

Open
Shubham-Padkonde wants to merge 2 commits into
netdata:masterfrom
Shubham-Padkonde:fix/kickstart-http-fallback
Open

Shubham-Padkonde wants to merge 2 commits into
netdata:masterfrom
Shubham-Padkonde:fix/kickstart-http-fallback

Conversation

@Shubham-Padkonde

@Shubham-Padkonde Shubham-Padkonde commented Sep 27, 2026 •

Copy link
Copy Markdown
Summary

Fixes #24046.

When curl returns receive-error code 56 alongside an HTTP 4xx/5xx response, classify it through the existing HTTP-status handler. This lets the source installer fall back to gzip when a zstd archive returns 404. Receive failures with HTTP 200 or 000 remain fatal; TLS and connection-error handling is unchanged.

Print deferred warnings with a constant %b format so a command containing %{http_code} no longer truncates the warning or produces a printf error. This preserves the accumulated warnings' existing escaped newlines.

Test Plan
  • Added tests/kickstart-http-fallback-test.sh: 11 curl/HTTP-status combinations, partial-download cleanup, percent-containing warnings, and the actual source-build selection function with transport/build stubs. Before the fix, three HTTP-status cases, warning output, and gzip fallback failed. Afterward it passes under both sh (dash) and bash on Linux.
  • Existing sh tests/kickstart-path-sanitizer-test.sh passes.
  • Shell syntax and git diff --check pass. ShellCheck passes on the new test; the installer has the same two baseline warnings (SC2044 and SC2034).
  • The local SOW audit's one failure is the Windows checkout representing the existing query-netdata-agents symlink as a regular file; its target exists in Git. No SOW/spec files are included in this PR.

No real installation or macOS run was performed.

Additional Information

Prepared with Codex assistance.

For users: How does this change affect me?

Local source installations can use the existing gzip fallback when an unavailable zstd archive produces curl error 56 with an HTTP error response. Deferred installer warnings containing percent signs remain readable.


Summary by cubic

Fixes the kickstart installer so source installs fall back to gzip when curl reports an HTTP 4xx/5xx error as receive error 56, and preserves deferred warnings containing percent signs and other shell metacharacters.

  • Maps curl exit 56 with an HTTP 4xx/5xx response to exit 22 so the existing gzip fallback triggers.
  • Stores warnings with literal newlines instead of \n strings and escapes shell metacharacters when writing the status file, so rendered output like %{http_code} survives unchanged across kickstart, updater, uninstaller, and helper scripts.
  • Adds tests/kickstart-http-fallback-test.sh and tests/kickstart-warning-test.sh, run on dash and bash in a new CI workflow.

Written for commit 6a32f99. Summary will update on new commits.

Review in cubic

Summary by CodeRabbit

  • Bug Fixes
    • Installer warnings now appear on separate lines, and backslashes and other special characters in diagnostic text are displayed literally.
    • Installer status details preserve special characters when written to status files.
    • Interrupted downloads that end with an HTTP error are handled appropriately; other transfer errors retain their existing behavior.
  • Tests
    • Added checks for warning output, status details, download error handling, cleanup, and fallback to an alternate source archive.
    • Added automated installer regression checks across supported shell environments.

@github-actions github-actions Bot added area/packaging Packaging and operating systems support area/tests labels Sep 27, 2026
@CLAassistant

CLAassistant commented Sep 27, 2026 •

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@coderabbitai

coderabbitai Bot commented Sep 27, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Advanced

Run ID: 9f92dae2-906a-4cbf-b3b6-f6166459fa84

📥 Commits

Reviewing files that changed from the base of the PR and between b547c06 and 6a32f99.

📒 Files selected for processing (8)
  • .github/workflows/installer-shell-tests.yml
  • netdata-installer.sh
  • packaging/installer/functions.sh
  • packaging/installer/kickstart.sh
  • packaging/installer/netdata-uninstaller.sh
  • packaging/installer/netdata-updater.sh
  • tests/kickstart-http-fallback-test.sh
  • tests/kickstart-warning-test.sh

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 8 remain after this review.


📝 Walkthrough

Walkthrough

Installer scripts now store warning entries on separate lines, print deferred warnings literally, and escape values written to status files. Kickstart also handles curl code 56 based on the final logged HTTP status. Regression tests and a GitHub Actions workflow cover installer shell behavior.

Changes

Installer warning and download handling

Layer / File(s) Summary
Warning storage, output, and status files
packaging/installer/functions.sh, packaging/installer/netdata-uninstaller.sh, packaging/installer/netdata-updater.sh, netdata-installer.sh, packaging/installer/kickstart.sh, tests/kickstart-warning-test.sh, .github/workflows/installer-shell-tests.yml
Installer scripts store warning entries with actual newlines and print deferred warnings literally. exit_reason escapes values before writing status files. Warning tests check stored and rendered diagnostics, and the workflow runs installer regression scripts under dash and bash.
Curl HTTP handling and archive fallback
packaging/installer/kickstart.sh, tests/kickstart-http-fallback-test.sh
Curl code 56 maps to code 22 when the final download-log line shows a 4xx or 5xx status. Tests check curl result handling, partial-download cleanup, and fallback from a failed .zst download to .tar.gz.

Priority: ➖ Normal

Estimated code review effort: 3 (Moderate) | ~20 minutes

Change: Bug fix · Severity of issue fixed: Medium

Suggested reviewers: ferroin

Merge Risk: ⚪ Minimal · up to 6a32f

The changes appear ready to merge after normal checks; no actionable merge-blocking risk is established.

Security Architecture Review

Security architecture risk: 🔵 Low · up to 6a32f

The installer can now try the existing gzip fallback after certain failed zstd downloads. Checksum verification remains in place, and warning values are escaped before being passed between scripts. No new exposed service or elevated access was identified, but the assessment does not cover a complete installation run.

Retained concerns
No architecture-level concerns identified.

Security review details

Security Blast Radius

  • inferred — The changed download classification affects installer callers of the shared curl-result handler, including source-archive retrieval and remote-file checks; the examined paths do not introduce a new service or test-to-production entrypoint.

Trust Boundaries and Controls

  • observed — HTTP status text enters through curl’s download log, but only a three-digit 4xx or 5xx final line changes code-56 handling. The archive checksum check remains after fallback selection. Child-script warning values cross a separate shell-source boundary through escaped assignments in a temporary status file.

Resilience and Maintainability Implications

  • observed — On a classified HTTP failure, the destination is removed before the caller considers another archive. If neither archive downloads or checksum verification fails, source installation takes a fatal path.
🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 3.70% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 27 functions across 7 files. (1 skipped: 1… Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly describes the primary functional change: preserving the kickstart gzip fallback when curl reports HTTP errors. Additional warning-handling and CI changes are secondary.
Linked Issues check ✅ Passed The PR meets the coding requirements in issue [#24046]. packaging/installer/kickstart.sh routes curl status 56 to the existing HTTP error path only when the response status is 4xx or 5xx. Other stat…
Out of Scope Changes check ✅ Passed The changes remain within issue [#24046]. The warning-preservation changes in the shared installer, updater, and uninstaller paths address the same reported deferred-warning formatting and serializati…
Full details: Docstring Coverage

Explanation

Docstring coverage is 3.70% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 27 functions across 7 files. (1 skipped: 1 unsupported.)

  • Fix all pre-merge checks with AI
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create a new PR

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@qodo-code-review

qodo-code-review Bot commented Sep 27, 2026 •

Copy link
Copy Markdown

PR Summary by Qodo

Preserve gzip fallback for curl HTTP receive errors

🐞 Bug fix 🧪 Tests 🕐 10-20 Minutes

Grey Divider

AI Description

• Routes curl error 56 with HTTP failures through existing status handling.
• Preserves gzip fallback and percent-containing deferred warning output.
• Adds coverage for error mappings, cleanup, warnings, and source fallback.
Diagram

graph TD
  A["curl download"] --> B{"Exit code 56?"} -->|Yes| C{"HTTP 4xx/5xx?"} -->|Yes| D["Status handler"] --> E["gzip fallback"]
  B -->|No| F["Existing handling"]
  C -->|No| F
  G["Deferred warnings"] --> H["Percent-safe output"]
Loading
High-Level Assessment

The targeted normalization is preferable because it reuses established HTTP-status semantics only when curl error 56 accompanies a 4xx/5xx status. Broadly classifying all receive errors or changing curl invocation flags could misclassify HTTP 200/000 transport failures and alter unrelated TLS or connection behavior.

Files changed (2) +93 / -2

Bug fix (1) +9 / -2
kickstart.shNormalize HTTP-backed curl receive errors and safely print warnings +9/-2

Normalize HTTP-backed curl receive errors and safely print warnings

• Treats curl exit code 56 as an HTTP failure only when the logged status is 4xx or 5xx, allowing existing status handling and gzip fallback to run. Prints deferred warnings through a constant '%b' format so embedded percent sequences remain intact while escaped newlines are preserved.

packaging/installer/kickstart.sh

Tests (1) +84 / -0
kickstart-http-fallback-test.shCover curl status handling and source archive fallback +84/-0

Cover curl status handling and source archive fallback

• Adds portable shell tests for curl and HTTP status combinations, partial-download cleanup, and percent-containing deferred warnings. It also exercises the real source-build selection function with stubs to verify fallback from an unavailable zstd archive to gzip.

tests/kickstart-http-fallback-test.sh

@qodo-free-for-open-source-projects

qodo-free-for-open-source-projects Bot commented Sep 27, 2026 •

Copy link
Copy Markdown

Code Review by Qodo

🐞 Bugs (0) 📘 Rule violations (0) 📎 Requirement gaps (0) 🎨 UX issues (0) 🔗 Cross-repo conflicts (0) 📜 Skill insights (0)

Grey Divider


Remediation recommended

1. Backslashes can hide deferred warnings ✓ Resolved 🐞 Bug ◔ Observability
Description
deferred_warnings passes the entire accumulated NETDATA_WARNINGS value to %b, so \c in any
interpolated warning or failed-command argument terminates printf and other escape-like path text
is transformed. This surfaces when values such as a user-selected temporary path reach run or
warning, suppressing that warning and every warning appended after it.
Code

packaging/installer/kickstart.sh[498]

+    printf >&2 "%b" "${NETDATA_WARNINGS}"
Relevance

●●● Strong

Concrete escape-sequence truncation; accepted history shows reviewers correct unsafe printf
formatting and warning output bugs.

PR-#23223
PR-#23252

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
The changed renderer applies %b to the entire accumulated value, while both run and warning
append interpolated text to that value. TMPDIR is accepted from the environment and later appears
in a command passed through run, providing a concrete route for backslashes from a path to enter
deferred warnings; POSIX specifies that \c in a %b operand stops all remaining output.

packaging/installer/kickstart.sh[495-500]
packaging/installer/kickstart.sh[575-580]
packaging/installer/kickstart.sh[635-640]
packaging/installer/kickstart.sh[644-672]
packaging/installer/kickstart.sh[2315-2315]
🌐 POSIX specifies that the %b conversion interprets backslash escapes and that \\c causes printf to ignore the remainder of the current operand, all remaining operands, and remaining format characters.: 🌐 POSIX specifies that the %b conversion interprets backslash escapes and that \\c causes printf to ignore the remainder of the current operand, all remaining operands, and remaining format characters.

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
Deferred warnings use one string for both encoded separators and arbitrary warning content. Rendering the whole value through `%b` lets backslash sequences in messages and failed-command arguments transform or terminate the warning output.
## Fix Focus Areas
- packaging/installer/kickstart.sh[495-500]
- packaging/installer/kickstart.sh[575-580]
- packaging/installer/kickstart.sh[635-640]
- packaging/installer/functions.sh[493-527]
- packaging/installer/functions.sh[568-573]
## Recommended Fix
Store warning separators as actual newline characters instead of literal `\n` escapes, then render the accumulated value with the constant format `printf '%s'`. Update all warning accumulation and propagation paths consistently, including saved warnings from child installer scripts, and add coverage proving that `\c`, `\t`, and ordinary backslashes are emitted literally while warning boundaries remain newlines.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


2. Later installer warnings can disappear ✓ Resolved 🐞 Bug ◔ Observability
Description
deferred_warnings passes the entire NETDATA_WARNINGS accumulator to %b, so backslash sequences
in dynamic warning text are interpreted as formatting controls. Under shells that preserve the
argument text, an option such as --claim-\c reaches warning() and makes printf stop before
subsequent diagnostics and final spacing.
Code

packaging/installer/kickstart.sh[498]

+    printf >&2 "%b" "${NETDATA_WARNINGS}"
Evidence
The changed %b conversion processes every escape sequence in the accumulator, while both
accumulator-producing paths append dynamic text. The command-line parser passes an unrecognized
claiming option into warning() without sanitizing its backslashes, providing a direct path for
\c to suppress the rest of the deferred output.

packaging/installer/kickstart.sh[495-500]
packaging/installer/kickstart.sh[576-581]
packaging/installer/kickstart.sh[635-641]
packaging/installer/kickstart.sh[2791-2819]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
Deferred warning text is rendered using `%b`, which interprets dynamic backslash sequences such as `\c` and can truncate all later diagnostics.
## Fix Focus Areas
- packaging/installer/kickstart.sh[495-500]
- packaging/installer/kickstart.sh[576-581]
- packaging/installer/kickstart.sh[635-641]
## Recommended Fix
Store actual newline characters when appending warnings instead of literal `\n` sequences, then render `NETDATA_WARNINGS` with a constant `%s` format. Update the warning-output test to cover `\c`, octal escapes, and ordinary backslashes as literal text.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


3. Fallback regressions bypass automation ✓ Resolved 🐞 Bug ☼ Reliability
Description
kickstart-http-fallback-test.sh is added as a standalone script, but run-unit-tests.sh neither
defines nor invokes it. The only unit-test workflow also excludes both this test and kickstart.sh
from its path filters, so future changes to the fallback cannot execute these assertions
automatically.
Code

tests/kickstart-http-fallback-test.sh[1]

+#!/bin/sh
Evidence
The repository test runner invokes the existing system-information and kickstart-path shell tests
but has no entry for the new script. Its workflow only triggers for C sources, headers, ACL tests,
and changes to the runner itself, leaving both files modified by this PR outside the trigger set.

tests/kickstart-http-fallback-test.sh[1-17]
tests/run-unit-tests.sh[25-40]
tests/run-unit-tests.sh[51-61]
.github/workflows/tests.yml[1-20]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The new fallback regression test is not called by the repository test runner, and relevant installer or test changes do not trigger the unit-test workflow.
## Fix Focus Areas
- tests/kickstart-http-fallback-test.sh[1-84]
- tests/run-unit-tests.sh[25-61]
- .github/workflows/tests.yml[1-20]
## Recommended Fix
Add a test-runner function that executes `kickstart-http-fallback-test.sh` with `/bin/sh` and invoke it from `run-unit-tests.sh`. Extend the workflow path filters to include the new test and `packaging/installer/kickstart.sh`, or add an equivalent lightweight installer-test workflow triggered by those paths.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


Grey Divider

Tip of the day
💡 Did you know, you can describe a rule in plain language on the Rules page and Qodo drafts it for you

More tips ↗ | Customize Qodo ↗ | Qodo docs ↗

Grey Divider

Qodo Logo

Comment thread packaging/installer/kickstart.sh Outdated
printf >&2 "%s\n" "The following non-fatal warnings or errors were encountered:"
# shellcheck disable=SC2059
printf >&2 "${NETDATA_WARNINGS}"
printf >&2 "%b" "${NETDATA_WARNINGS}"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remediation recommended

1. Later installer warnings can disappear 🐞 Bug ◔ Observability

deferred_warnings passes the entire NETDATA_WARNINGS accumulator to %b, so backslash sequences
in dynamic warning text are interpreted as formatting controls. Under shells that preserve the
argument text, an option such as --claim-\c reaches warning() and makes printf stop before
subsequent diagnostics and final spacing.
Agent Prompt
## Issue description
Deferred warning text is rendered using `%b`, which interprets dynamic backslash sequences such as `\c` and can truncate all later diagnostics.

## Fix Focus Areas
- packaging/installer/kickstart.sh[495-500]
- packaging/installer/kickstart.sh[576-581]
- packaging/installer/kickstart.sh[635-641]

## Recommended Fix
Store actual newline characters when appending warnings instead of literal `\n` sequences, then render `NETDATA_WARNINGS` with a constant `%s` format. Update the warning-output test to cover `\c`, octal escapes, and ordinary backslashes as literal text.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

Comment thread tests/kickstart-http-fallback-test.sh
@qodo-code-review

qodo-code-review Bot commented Sep 27, 2026 •

Copy link
Copy Markdown

Code Review by Qodo

🐞 Bugs (0) 📘 Rule violations (0) 🔗 Cross-repo conflicts (0) 📜 Skill insights (0)

Grey Divider


Remediation recommended

1. Backslashes can hide deferred warnings ✓ Resolved 🐞 Bug ◔ Observability
Description
deferred_warnings passes the entire accumulated NETDATA_WARNINGS value to %b, so \c in any
interpolated warning or failed-command argument terminates printf and other escape-like path text
is transformed. This surfaces when values such as a user-selected temporary path reach run or
warning, suppressing that warning and every warning appended after it.
Code

packaging/installer/kickstart.sh[498]

+    printf >&2 "%b" "${NETDATA_WARNINGS}"
Relevance

●●● Strong

Concrete escape-sequence truncation; accepted history shows reviewers correct unsafe printf
formatting and warning output bugs.

PR-#23223
PR-#23252

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
The changed renderer applies %b to the entire accumulated value, while both run and warning
append interpolated text to that value. TMPDIR is accepted from the environment and later appears
in a command passed through run, providing a concrete route for backslashes from a path to enter
deferred warnings; POSIX specifies that \c in a %b operand stops all remaining output.

packaging/installer/kickstart.sh[495-500]
packaging/installer/kickstart.sh[575-580]
packaging/installer/kickstart.sh[635-640]
packaging/installer/kickstart.sh[644-672]
packaging/installer/kickstart.sh[2315-2315]
🌐 POSIX specifies that the %b conversion interprets backslash escapes and that \\c causes printf to ignore the remainder of the current operand, all remaining operands, and remaining format characters.

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
Deferred warnings use one string for both encoded separators and arbitrary warning content. Rendering the whole value through `%b` lets backslash sequences in messages and failed-command arguments transform or terminate the warning output.

## Fix Focus Areas
- packaging/installer/kickstart.sh[495-500]
- packaging/installer/kickstart.sh[575-580]
- packaging/installer/kickstart.sh[635-640]
- packaging/installer/functions.sh[493-527]
- packaging/installer/functions.sh[568-573]

## Recommended Fix
Store warning separators as actual newline characters instead of literal `\n` escapes, then render the accumulated value with the constant format `printf '%s'`. Update all warning accumulation and propagation paths consistently, including saved warnings from child installer scripts, and add coverage proving that `\c`, `\t`, and ordinary backslashes are emitted literally while warning boundaries remain newlines.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


Grey Divider

Context sources
✅ Compliance rules (platform): 251 rules
✅ Skills: 10 invoked
  collectors-authoring
  repo-skill-authoring
  collectors-go-framework-v2
  collectors-metadata-yaml
  triage-support-bundle
  integrations-lifecycle
  collectors-go-design
  health-alert-authoring
  topology-authoring
  collectors-prometheus-profiles
✅ Web pages:
  +2 more
✅ Cross-repo context — repo relationships
Review mode: ⚖️ Balanced: This changes runtime installer error classification and warning formatting on download/fallback paths, so it carries meaningful behavioral risk but is localized rather than dense enough to warrant extended review.

Grey Divider

Tip of the day
💡 Did you know, you can describe a rule in plain language on the Rules page and Qodo drafts it for you

More tips ↗ | Customize Qodo ↗ | Qodo docs ↗

Grey Divider

Qodo Logo

Comment thread packaging/installer/kickstart.sh Outdated
@Shubham-Padkonde

Copy link
Copy Markdown
Author

Addressed the warning preservation and missing CI coverage findings. Warning accumulators now use real newline separators and are printed as data, including the shared installer, updater, and uninstaller paths. Child status-file serialization preserves backslashes and shell metacharacters.

Added a regression for literal \c, \t, octal-looking text, percent signs, quotes, and child status round trips. It fails on the previous commit and passes after the change. The HTTP fallback, warning, and path-sanitizer suites all pass under dash and bash; ShellCheck passes for the new tests and changed shared helper/updater/uninstaller scripts. A dedicated workflow now runs these shell regressions on relevant changes. No real installation or macOS validation was performed.

Prepared with Codex assistance.

@sonarqubecloud

Copy link
Copy Markdown

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/ci area/packaging Packaging and operating systems support area/tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: kickstart.sh local build aborts instead of falling back to .tar.gz when .tar.zst is missing (curl exit 56 unhandled)

2 participants