Skip to content

Apply Benchmark patch only where its needed#2798

Open
JohnReedV wants to merge 4 commits into
devnet-readyfrom
apply-benchmark-to-only-bad-benches
Open

Apply Benchmark patch only where its needed#2798
JohnReedV wants to merge 4 commits into
devnet-readyfrom
apply-benchmark-to-only-bad-benches

Conversation

@JohnReedV

@JohnReedV JohnReedV commented Jun 25, 2026

Copy link
Copy Markdown
Contributor

Description

This PR modifies the benchmark flow so that only extrinsics that have drifted passed the threshold or have incorrect read/writes are updated rather than replacing the entire file and creating conflicts everywhere.

Also fixes a bug where it would fail extrinsics where it shouldn't.

@JohnReedV JohnReedV added the skip-cargo-audit This PR fails cargo audit but needs to be merged anyway label Jun 25, 2026
@github-actions

github-actions Bot commented Jun 25, 2026

Copy link
Copy Markdown
Contributor

🛡️ AI Review — Skeptic (security review)

VERDICT: SAFE

Baseline scrutiny: established OpenTensor-associated contributor with repo write permission; no trusted-list Gittensor match found; branch apply-benchmark-to-only-bad-benches -> devnet-ready.

Reviewed the cached PR metadata, prior Skeptic comment, contributor signals, comments, and full diff. The PR does not modify .github/ai-review/*, .github/copilot-instructions.md, dependencies, lockfiles, or build scripts.

The runtime-facing changes are generated weight updates plus a required spec_version bump. The tooling changes are limited to benchmark comparison and selective patch generation; I did not find credential access, hidden network access, shell injection, supply-chain changes, origin/permission changes, unsafe runtime code, or a credible malicious patch path.

Findings

No findings.

Conclusion

No malicious intent or security vulnerability was found in the reviewed diff. Residual risk is limited to benchmark-tooling correctness and generated-weight accuracy, which do not cross the Skeptic security threshold here.


🔍 AI Review — Auditor (domain review)

VERDICT: 👍

Gittensor UNKNOWN; author is an established repository contributor with write permission, so review focused on benchmark tooling correctness rather than intent.

PR body is substantive enough to leave unchanged.

Overlap check: the overlapping open PRs touch common runtime/generated-weight files, but their titles and bases indicate unrelated runtime features or broad wiring work rather than this benchmark patch-generation behavior. I do not see duplicate work to recommend closing.

Spec version: bumped from 423 to 424 for the runtime/weights changes.

Validation: bash -n scripts/benchmark_action.sh passed. A targeted cargo test -p subtensor-weight-tools --bin weight-compare could not run in this sandbox because rustup/cargo tried to write under read-only /home/runner/.rustup / /home/runner/.cargo; no workspace files were modified and no auto-fixes were needed.

Findings

No findings.

Conclusion

The PR keeps the benchmark patch path scoped to drifted benchmark functions and updates the comparator so total weight drift is evaluated over benchmark component ranges. I did not find a domain correctness issue that should block merge.

@github-actions

Copy link
Copy Markdown
Contributor

🔄 AI review updated — Skeptic: SAFE Auditor: 👍

@github-actions

Copy link
Copy Markdown
Contributor

🔄 AI review updated — Skeptic: SAFE Auditor: 👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

skip-cargo-audit This PR fails cargo audit but needs to be merged anyway

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant