feat(eval): add a local override layer for machine-local execution policy

[christso]

Context

Follow-up from Harbor boundary issue #1392 and the local override design pass on 2026-06-19.

Current conclusion:

• AgentV eval YAML should remain the canonical portable authoring surface.
• Portable eval intent includes repo identity, checkout pin, hooks, template, isolation, and runner intent.
• Machine-local execution policy such as workspace.mode: static and workspace.path should not be the preferred shared-YAML shape.
• Harbor-backed execution and portable bundles should not depend on machine-local workspace paths.

Decision

Add a narrow local override layer for eval execution policy.

Recommended first shape:

• keep shared eval YAML as the base contract
• support a gitignored per-eval sidecar by inserting .local before the final YAML extension, for example repo-regression.eval.local.yaml
• add --local-overrides <path> and --no-local-overrides
• keep CLI flags as the highest-precedence escape hatch
• treat environment variables as interpolation inputs, not as a second structural override system

For compatibility, continue parsing workspace.mode: static and workspace.path in eval YAML for now, but stop promoting them in shared docs/examples.

Scope

Native run path:

• allow a small override allow-list for local execution policy
• start with workspace.mode, workspace.path, and selected execution-policy leaves such as workers, cache, cache_path, and existing local trace/output destinations
• reject local overrides that try to change portable eval semantics such as workspace.repos, hooks, template, isolation, runner, Harbor source selectors, targets, assertions, or thresholds

Portable/bundled paths:

• keep local-only fields out of agentv eval bundle and Harbor packaging
• fail or warn when Harbor runs encounter static local workspace fields

Non-goals

• Introduce a project-wide override map in the first slice
• Add arbitrary deep-merge semantics across eval YAML trees
• Let local override files redefine repo pins, hooks, graders, or pass/fail gates
• Make Harbor consume local workspace paths as task acquisition hints

Acceptance criteria

• AgentV can discover a per-eval .local sidecar and merge only an allow-listed set of fields.
• CLI supports --local-overrides <path> and --no-local-overrides.
• Native runs apply local static workspace reuse without requiring shared eval YAML to contain machine-local paths.
• Validation rejects local overrides that mutate portable eval semantics.
• Bundles and Harbor-backed execution omit or reject local-only static workspace fields.
• Docs show static mode via CLI or local override first, while shared-YAML static examples are treated as legacy/local-only compatibility.

Related

• plan: keep AgentV native workspaces, delegate benchmark execution to Harbor #1392 plan: keep AgentV native workspaces, delegate benchmark execution to Harbor