Visitar URL original
Match a Side-Reviewer Ask to Its Head by the Push GitHub Logged by ptr727 · Pull Request #4174 · ptr727/ProjectTemplate · GitHub
Skip to content

Match a Side-Reviewer Ask to Its Head by the Push GitHub Logged - #4174

Merged
ptr727 merged 4 commits into
developfrom
burndown/side-ask-server-stamp
Nov 13, 2026
Merged

ptr727 merged 4 commits into
developfrom
burndown/side-ask-server-stamp

Conversation

@ptr727

@ptr727 ptr727 commented Nov 13, 2026 •

Copy link
Copy Markdown
Owner

Summary

side_asked_on_head in scripts/pr_review_side.py decides whether CodeRabbit or Qodo was already asked for the current head. Until now it compared each ask comment's server createdAt with the head commit's committedDate. That date comes from the author's clock, so it can run behind the push or ahead of it.

  • Backdated commit. A commit stamped 10:00 and pushed at 10:20 made a 10:05 ask for the previous head count for the new one. So the new head was never asked.
  • Future-dated commit. A commit stamped 10:30 and pushed at 10:20 made a 10:25 ask for this head read as older. So the next wait asked again, a duplicate paid round.

The head's arrival is now the timestamp GitHub's repository activity log gives the push that left the head on its branch, read through gh api repos/<head repo>/activity?ref=refs/heads/<branch>. A jq projection prints the newest entry whose after is the head. The comment's createdAt is also a server stamp, so GitHub stamps both sides of the comparison.

This change reintroduces none of the defects the earlier fix rounds on #3881 hit:

  • The log stamps a push when it arrives, before any check suite exists. So an ask made before the first suite still counts. A repository with no checks is asked once per head, not on every wait.
  • No commit date is read anywhere, and committedDate leaves the pull request query.
  • headRepository{ nameWithOwner } and headRefName, the fields pushed reads, are in Q_HELD, the query the asks' reads run.

Sometimes the push cannot be read: the read fails, the fork was deleted, or the log is missing the head. In that case a reviewer with an ask already on record is not asked again, and wait prints a note, because without the stamp that ask cannot be told from one for an earlier head. A reviewer with no ask on record is still asked, since there is nothing to duplicate. Neither bot gates the merge, so the worst case is one missed side round, never a wrong merge.

scripts/pr_review.py changes by its query and one wiring line, which hands gh_rest to side.ask. It stays at 2198 lines. scripts/pr_review_side.py is at 349 of its 350.

Verification

  • tests/test_pr_review_policy.py, new TestSideAskServerStamp:
    • A backdated commit and a future-dated commit are each run through side.ask with the commit date set to mislead. Both cases fail with the old committedDate comparison restored.
    • The read builds the activity path with the branch URL-encoded. A failed read, a missing head repository, a missing branch, and a non-hex head all read as empty.
    • The real jq binary runs the projection against a constructed log. A head pushed twice resolves to the later push, and a log without the head prints nothing.
  • An unread push with an ask on record asks nothing and prints the note. One with no ask on record still sends the first ask, and that case fails with the gate widened again. The wait-level cases that ask now answer the activity read.
  • The merged-while-checks-settle case now carries a push stamp. It fails with its post-wait state check removed.
  • Live read-only check against an open pull request in this repository. Q_HELD returns headRepository, and pushed returns the push's timestamp.
  • For eight recent heads in this repository, the activity stamp came 1 to 2 seconds before the first check suite's createdAt. Each commit date was minutes earlier than either.
  • The full suite passes except test_tooling_encoding, which caught the new jq spawn's missing encoding="utf-8". That is fixed, and the module passes. Run with uv run --no-project --with-requirements tests/requirements.txt python -m unittest discover -s tests -p 'test_*.py'. The pr_review modules and test_tooling_encoding pass after the last edit.
  • ruff format, ruff check, mypy, and the prose gate against develop pass.

Local strict pass findings

The first pass over 2f77e8c raised four:

  1. introduced, fixed in 11ede57. The merged-while-checks-settle test passed vacuously, because its payload carried no push stamp.
  2. introduced, declined. If the activity log lagged the PR head, an on-open side ask could miss that wait. Measured on this repository's recent heads, the log comes before the first check suite. A lag would cost one missed side round with a printed note, which is neither silent nor destructive. Per GOVERNANCE.md "Trust Boundaries and Hardening Effort", it earns no retry.
  3. introduced, declined. The scripts/README.md sentence "asked once per new head" does not name the unread-push skip. That sentence already omits the cap, cut-window, and closed-pull-request skips. The request_side_reviews docstring, which lists the skips, now names this one.
  4. Design, declined. head_arrived still drives checks_hold's grace window. It answers a different question: when checks could have started on a read await_green polls. Moving it onto the push would add a REST read to every poll. A head with no suite and no check reading PENDING is how checks_hold already behaved.

The second pass over 11ede57 raised one:

  1. introduced, fixed in 34f6585. The unread-push gate also withheld the first ask, even with no ask on record for an unread push to hide.

The third pass over 34f6585 raised one. It was handed to this pull request when the opening push's edit budget ran out:

  1. introduced, fixed in bb41f02. No test pinned headRepository{ nameWithOwner } in Q_HELD. The fixtures inject it, so the suite would have passed with it dropped from the query while every live read came back empty. A case now pins each field the push read takes from Q_HELD, and it fails with that field removed.

The third pass also asked whether the activity log's timestamps carry fractional seconds. A live read shows they do not, for example 2026-11-13T19:11:58Z. Finding 2 has a live data point too: this branch's own push was in the activity log 4 seconds after git push returned.

The fourth pass over bb41f02 raised none.

Closes on promotion: #3882

🤖 Generated with Claude Code

ptr727 and others added 3 commits November 13, 2026 11:44
A CodeRabbit or Qodo ask was placed against the head by the head
commit's own date, which its author's clock sets. A commit dated behind
its push carried an earlier head's ask onto the new head, so the ask was
skipped. One dated ahead of its push hid an ask already made for it, so
the next wait asked again, a duplicate paid round.

The head's arrival is now the timestamp GitHub's repository activity log
gave the push that left the head on its branch. The comment's createdAt
is the server's too, so both sides of the comparison are GitHub's own.
The log stamps a push on receipt, before any check suite exists, so an
ask made before the first suite still counts and a repository with no
checks is asked once per head. Where the push cannot be read, no ask is
sent and the wait prints a note, since an ask already made for the head
could not be told from one for an earlier head.

The pull request query reads the head repository and drops the commit
date, which nothing else reads.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ead Skip

The local strict pass found the merged-while-its-checks-settle case
passing vacuously: its payload carried no push stamp, so the reviewer
was skipped before the check wait and the merged re-read it exists to
test. It now carries one and fails with that re-read's state check
removed. The request docstring names the unread push among the skips.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The second local strict pass found the unread-push gate withholding the
first ask too. With no ask on record there is none an unread push could
hide, so a deleted fork or a failed activity read left the reviewer
never asked. The gate now holds only where an ask is on record, and a
case covers the first ask on an unread push.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Copilot AI lite review requested due to automatic review settings November 13, 2026 19:57
@coderabbitai

coderabbitai Bot commented Nov 13, 2026 •

Copy link
Copy Markdown

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 37 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

Run configuration

Commits

Reviewing files that changed from the base of the PR and between e319962 and bb41f02.

Files selected for processing: 3

  • scripts/pr_review.py
  • scripts/pr_review_side.py
  • tests/test_pr_review_policy.py
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@ptr727

ptr727 commented Nov 13, 2026

Copy link
Copy Markdown
Owner Author

A recorded local strict-review pass covers head 34f6585bf844ec86878dd78c5e0483add82514ff, the content this pull request carries at that commit against develop, and it recorded 1 finding.

The third local strict pass found that no test held the head repository
in the query the side asks read. The fixtures inject it, so dropping it
from the query would pass every case while each live read came back
empty. A case now pins each field the push read takes from that query.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

An active P2 finding remains because the query contract lacks coverage for the required headRepository{ nameWithOwner } field, risking silent skipping of subsequent heads if the field is removed.

1 open issue
What changed in this PR

This pull request matches CodeRabbit and Qodo requests to pull request heads using GitHub's server-recorded push timestamp instead of the commit author's timestamp.

Changes:

  • Adds head repository data and wires activity-log reads into side-review handling.
  • Handles unread pushes conservatively to avoid duplicate or unsafe requests.
  • Expands coverage for clock skew, activity-log lookup, URL encoding, repeated pushes, and jq projection behavior.
  • Records a test-coverage finding for pinning headRepository in Q_HELD.
File Description
tests/​test_pr_review_policy.py Adds coverage for server-stamped head matching, activity-log failures, URL encoding, repeated pushes, unread pushes, and jq projection behavior.
scripts/​pr_review.py Adds head repository data to the held query and passes the REST reader into side-review handling.
scripts/​pr_review_side.py Uses GitHub activity timestamps to identify the push associated with the current head and handles unread activity safely.

Review effort: Lite

Comment thread scripts/pr_review.py Resolved
@ptr727

ptr727 commented Nov 13, 2026

Copy link
Copy Markdown
Owner Author

A recorded local strict-review pass covers head bb41f02b152598f07c293fde41fe717b6d44c364, the content this pull request carries at that commit against develop, and it recorded 0 findings.

@codecov

codecov Bot commented Nov 13, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 76.71%. Comparing base (b38e084) to head (bb41f02).
⚠️ Report is 200 commits behind head on develop.

Additional details and impacted files
@@             Coverage Diff             @@
##           develop    #4174      +/-   ##
===========================================
- Coverage    81.60%   76.71%   -4.89%     
===========================================
  Files           26       41      +15     
  Lines         9070    14317    +5247     
===========================================
+ Hits          7402    10984    +3582     
- Misses        1668     3333    +1665     
Flag Coverage Δ
python-3.13 76.71% <100.00%> (-4.89%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@ptr727

ptr727 commented Nov 13, 2026

Copy link
Copy Markdown
Owner Author

@coderabbitai review

@ptr727

ptr727 commented Nov 13, 2026

Copy link
Copy Markdown
Owner Author

/review

@coderabbitai

coderabbitai Bot commented Nov 13, 2026 •

Copy link
Copy Markdown
Action not completed

Review rate limited.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@ptr727
ptr727 merged commit d76cabd into develop Nov 13, 2026
13 checks passed
@ptr727
ptr727 deleted the burndown/side-ask-server-stamp branch November 13, 2026 20:41
ptr727 added a commit that referenced this pull request Nov 13, 2026
… Rules (#4189)

## Summary

Promotes the backlog-burndown rounds and changes from other hub sessions
from `develop` to `main`.

- Hooks and the prose gate: #4132, #4134, #4136, #4145, #4152, #4184.
The Husky hook reads its path list once and diffs a merge against
`MERGE_HEAD`. Interrupts re-raise their signal, a Markdown code span
reads as quoted data, and the gate picks a commit's diff base and
recognizes a conflicted promotion's branch cut from `main`.
- Procedures and carried rules: #4133, #4139, #4141, #4149, #4166,
#4177, #4179, #4183, #4185. Resync, audit and check-this-repo procedure
gaps, code style and shell interrupt rules, review-loop gaps across four
skills, the context and delegation section of `AGENTS.md`, question
rules in governance, the promotion line in pull request bodies,
closing-keyword behavior on promotion, and the history-rewrite premise.
- Review and cleanup tooling: #4138, #4148, #4154, #4156, #4157, #4163,
#4164, #4168, #4174, #4180. Cleanliness reads, parked handoffs, the
backlog report's JSON output, exact inline script pins, trusted inputs,
env-file includes, the bounded review wait, side-reviewer matching, and
a Copilot billing-failure notice read as a refusal.
- Workflow and runbook contract: #4153, #4155, #4169, #4171, #4178,
#4182. The runtime-secrets check, a registry entry and spec references,
the Copilot runbook pointers, credential-free checkout steps, and the
deploy-key and runner wording.
- From other hub sessions: #4159 (a repository's own .NET line under a
fleet default), #4161 (the 1Password CLI as an optional host tool),
#4162 (Codex's Windows installer and one managed opencode copy). These
carry no promotion line.

#4182 carries no promotion line by design, and #4175 holds the split-out
part of #4057's scope. Both #1448 and #4175 stay open.

Fixes #1071
Fixes #1154
Fixes #1391
Fixes #1471
Fixes #1594
Fixes #1620
Fixes #1627
Fixes #1650
Fixes #1801
Fixes #2853
Fixes #3620
Fixes #3668
Fixes #3697
Fixes #3766
Fixes #3787
Fixes #3837
Fixes #3840
Fixes #3842
Fixes #3844
Fixes #3865
Fixes #3882
Fixes #3890
Fixes #3894
Fixes #3912
Fixes #3916
Fixes #3954
Fixes #4014
Fixes #4015
Fixes #4016
Fixes #4043
Fixes #4050
Fixes #4057
Fixes #4058
Fixes #4062
Fixes #4084
Fixes #4097
Fixes #4098
Fixes #4099
Fixes #4104
Fixes #4108
Fixes #4114
Fixes #4115
Fixes #4117
Fixes #4118
Fixes #4119
Fixes #4121
Fixes #4122
Fixes #4123
Fixes #4131
Fixes #4135
Fixes #4140
Fixes #4142
Fixes #4143
Fixes #4144
Fixes #4151

🤖 Generated with [Claude Code](https://claude.com/claude-code)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants