Guidance for coding agents working in this repository.

• Applies to the entire repository rooted at this file.
• If a deeper AGENTS.md exists, that file takes precedence for its subtree.

Kingfisher is a Rust secret scanner, live credential validator, revocation helper, and access-map tool. It scans repositories, git history, local files, archives, cloud storage, source-host artifacts, Docker images, and collaboration-platform exports.

• src/: main CLI binary and application code.
• src/cli/commands/: CLI command definitions and wiring.
• src/scanner/: scan orchestration, input enumeration, repository/artifact fetching, validation phase.
• src/matcher/: pattern matching, captures, filtering, deduplication.
• src/reporter/: TOON, JSON, JSONL, SARIF, BSON, HTML, and pretty output.
• src/access_map/: blast-radius and permission mapping.
• crates/kingfisher-core/: shared core types.
• crates/kingfisher-rules/: rule schema, rule loading, and bundled rule data.
• crates/kingfisher-rules/build_support/: Betterleaks TOML import and expression parsing.
• crates/kingfisher-scanner/: embeddable scanning API and shared validators.
• tests/ and testdata/: integration tests and fixtures.
• docs/, docs/viewer/, docs-site/: docs, report viewer assets, and generated MkDocs site.
• third-party/kingfisher-vectorscan/: Vectorscan attribution and license files.

• Workspace minimum Rust version is 1.99 in Cargo.toml; make check-rust enforces >= 1.99.0 for build targets.
• Rust formatting is defined by rustfmt.toml (max_width = 100, 4 spaces, Unix newlines, reordered imports).
• Build scripts assume bash with set -eu -o pipefail.
• Windows Makefile targets expect MSYS2 with pacman.

• Build: cargo build
• Release build: cargo build --release
• Preferred test wrapper: make tests
• Direct tests: cargo test --workspace --all-targets
• Nextest: cargo nextest run --workspace --all-targets
• Format: cargo fmt --all
• Lint: cargo clippy --workspace --all-targets -- -D warnings
• Clean: make clean

• Keep edits minimal, targeted, and consistent with touched code.
• Do not revert user-authored or unrelated in-progress changes.
• Prefer clear fixes over broad refactors unless requested.
• Run the narrowest relevant tests first; run broader checks when practical.
• If a validation/build command cannot be run, state exactly what was skipped and why.
• Prefer kingfisher scan --format toon for agent/LLM workflows; use pretty only when human-interactive output is explicitly desired.
• After markdown/doc changes, verify local documentation links when practical.
• After docs-site/ source changes, rebuild with docs-site/.venv/bin/mkdocs build -f docs-site/mkdocs.yml when practical so generated output stays in sync.

• New and modified tests must work on Windows (x64 and arm64), macOS, and Linux. Review fixture setup and subprocess calls for platform assumptions even when developing on Unix.
• Build filesystem paths with Path/PathBuf and pass subprocess arguments with Command::arg/args; avoid shell interpolation, hard-coded Unix paths, and assumptions about executable suffixes or available Unix utilities.
• For Git fixture url.<base>.insteadOf rewrites, use an absolute local repository path. On Windows, normalize backslashes to forward slashes for Git. Avoid file:///C:/... URLs: MSYS2 Git can interpret them as /C:/... and fail to clone. See tests/github_gists.rs and tests/provider_snippets.rs for examples.
• Account for Windows file locking, path separators, and line endings in fixtures and assertions. Close file handles before deleting or replacing files. Keep platform-specific behavior narrowly scoped; do not skip otherwise portable tests on Windows to hide failures.
• Run the affected tests locally. For changes involving paths, Git, subprocesses, or filesystem behavior, also verify Windows CI on both architectures before claiming cross-platform success. Windows test commands are make windows-test-x64 and make windows-test-arm64 in the corresponding MSYS2 environments.
• If Windows execution is unavailable, explicitly report that Windows verification is pending; passing macOS/Linux tests is not evidence that Windows passed.

• Kingfisher's candidate detector catalog is sourced from Betterleaks and selected Veles detectors. Built-in rules are generated by the maintainer tool from pinned upstream sources. Prefer pinning tools/rule-bundle/src/main.rs to a Betterleaks release; when the latest release does not contain a detector we ship, pin a full immutable commit and document the reason and expected source digest. Never use main or another floating ref. Keep the Veles commit pinned, and update matching documentation and release-dependent tests when either source changes. Run cargo run --locked -p kingfisher-rule-bundle -- --refresh after pin updates; ordinary builds embed the prepared bundle without rule downloads.
• Betterleaks TOML is supported both for the built-in catalog and for custom rules passed through --rules-path. Custom Betterleaks rules are imported under the custom. namespace.
• The Kingfisher rule format (.yml/.yaml) and Betterleaks TOML (.toml) are both fully supported through --rules-path, including directories containing both formats. Custom rules in either format may be shared or private.
• Allocator feature flags live in root Cargo.toml: use-mimalloc default, use-jemalloc, and system-alloc.
• Optional validator feature sets live in crates/kingfisher-scanner/Cargo.toml.
• Validation modules live primarily in crates/kingfisher-scanner/src/validation/ and src/validation.rs.

• Default to YAML validation (validation:), especially Http or Grpc; do not add Rust validation unless YAML cannot express the flow reliably.
• Typed validators are schema-level reusable families: AWS, AzureStorage, Coinbase, GCP, MongoDB, MySQL, Postgres, Jdbc, and JWT.
• Raw validators use validation: { type: Raw, content: <name> } and are implemented in crates/kingfisher-scanner/src/validation/raw.rs for provider-specific exceptions.
• If Rust validation is unavoidable, prefer a raw validator before introducing a new typed validator.
• Do not convert existing typed validators to Raw for consistency alone.
• For rules with validation, add revocation: when the third-party API safely supports revocation.

New generally useful rules should be contributed to the Betterleaks repository first. Custom rules may use either the Kingfisher rule format (YAML) or Betterleaks TOML. Both are fully supported; choose the format that fits the required detection and validation capabilities.

1. Read docs/RULES.md before generating custom rules or doing non-trivial rule/schema work.
2. Generate custom rules in either supported format: Kingfisher YAML or Betterleaks TOML. Use Betterleaks TOML for contributions to the upstream built-in catalog.
3. Use an organization-specific custom rule ID; custom Betterleaks TOML IDs are automatically imported under custom..
4. Write a valid Hyperscan/Vectorscan regex. Lookahead and lookbehind are not supported.
5. Put the reported secret in one unnamed capture for {{ TOKEN }} and use non-capturing groups for structure.
6. Prefer specific token formats and provider context; avoid broad generic patterns.
7. Use entropy, pattern requirements, filters, and checksum requirements when format constraints are known.
8. Include examples that must match and negative examples when nearby formats are false positives.
9. Use components/depends_on_rule for multi-part credentials; mark helper rules invisible.
10. Add HTTP/gRPC validation and revocation only when the provider response is a reliable and safe signal.

• The default catalog is prepared by kingfisher-rule-bundle, then embedded in the binary. Exact upstream sources, provenance, and notices are archived under the rules crate's generated/ directory. Run the tool without flags after importer or overlay changes, and use --check to verify reproducibility without network access.
• --rules-path accepts Betterleaks .toml and Kingfisher .yml/.yaml files. Betterleaks TOML is translated through the same importer used for built-ins, while custom TOML gets the custom. namespace.
• A rule's regex produces candidate matches. The selected capture becomes the reported secret; entropy, pattern requirements, and filters remove candidates before reporting.
• Components attach nearby supporting values to a primary rule. They can be required or optional and are made available to validation as named variables.
• Validation checks credentials live through Betterleaks expressions, Kingfisher YAML HTTP/gRPC definitions, or Kingfisher's typed/raw validator families. A successful response is not proof unless the provider-specific matcher or expression makes it so.
• Revocation, access-map metadata, and narrow false-positive filters are operational capabilities kept separate from the Betterleaks detection catalog.

• Repeatable update workflow: python3 scripts/update-rule-bundle.py regenerates from archived sources, checks reproducibility, runs rule/importer tests, and rebuilds MkDocs when installed. It stops on the first failure. The Python wrapper delegates all importing to the Rust tool.
• Use --refresh after changing an upstream pin or Veles selection: this downloads only the configured immutable sources, archives their licenses, and runs the same verification. It does not select the latest release or change pins for you.
• Use --check to verify the committed bundle and run tests without rewriting generated files.
• Rule crate: cargo test -p kingfisher-rules
• Rule syntax/check path: kingfisher rules check --rules-path <custom-rule.yml> --load-builtins=false --no-update-check
• Scan fixture/corpus: kingfisher scan ./testdata --rule <rule-family-or-id> --rule-stats
• Validator check: kingfisher validate --rule <rule-id> <token-or-secret>
• Broad regression when practical: cargo test --workspace --all-targets

1. For new Betterleaks rules, land the detector upstream first. Choose a reviewed upstream release's full commit (or document why a later commit is needed), then update BETTERLEAKS_REVISION and BETTERLEAKS_SHA256 in tools/rule-bundle/src/main.rs. Calculate the digest from the exact config/betterleaks.toml bytes at that commit.
2. For new Veles coverage, update crates/kingfisher-rules/data/veles-rules.yml with a full commit and selected plugin IDs. Add an explicit adapter and tests in build_support/veles.rs if the detector family is not supported already.
3. Update operational overlays and focused importer/runtime tests as needed. Run cargo fmt --all before generating; formatting changes provenance input hashes.
4. Run python3 scripts/update-rule-bundle.py --refresh for upstream/pin/selection changes. For importer/overlay-only changes, run it without flags. If an adapter now needs additional upstream files, use --refresh even when the upstream commit has not changed.
5. Review added/removed rule IDs, source and license changes, capability changes, and the generated rule listing. Update documented rule counts and release-dependent assertions when the catalog changes. Do not hide stale IDs or failed tests by deleting checks.
6. Commit inputs, crates/kingfisher-rules/generated/, generated crate licenses, and docs-site/docs/rules/builtin-rules.md together. Never hand-edit the bundle or its hashes. Run python3 scripts/update-rule-bundle.py --check after the final edits.

The provenance manifest also hashes the root Cargo.toml, Cargo.lock, and LICENSE. Regenerate after changes to these files, even when no detector changed. A normal cargo build does not regenerate the catalog. See publishing guidance for details.

• Add a detection rule: contribute generally useful built-in detectors to Betterleaks first. For custom rules, follow docs/RULES.md and use either the Kingfisher rule format (YAML) or Betterleaks TOML.
• Add a CLI command: implement under src/cli/commands/ and register it in CLI wiring.
• Add a validator: prefer YAML first; if Rust is required, use raw.rs and the narrowest feature/dependency wiring.
• Update Betterleaks import behavior in crates/kingfisher-rules/build_support/; do not add a second generated or hand-authored built-in catalog.

• Usage: docs/USAGE.md, docs/ADVANCED.md, docs/INTEGRATIONS.md
• Rules: docs/RULES.md
• Architecture: docs/ARCHITECTURE.md, docs/BLAST_RADIUS.md
• Deployment/install: docs/INSTALLATION.md, docs/DEPLOYMENT.md, docs/PYPI.md
• Rust embedding: library guide, runnable examples (kingfisher-scanner; validation and revocation via validation)
• Python embedding: SDK guide, runnable examples (install kingfisher-secret-scanner, import kingfisher_sdk)