Skip to content

feat(vuln): OSV cache staleness + refresh flags — closes #30#45

Merged
Wolfvin merged 4 commits into
mainfrom
feat/issue-30-vuln-staleness
Jun 28, 2026
Merged

feat(vuln): OSV cache staleness + refresh flags — closes #30#45
Wolfvin merged 4 commits into
mainfrom
feat/issue-30-vuln-staleness

Conversation

@Wolfvin

@Wolfvin Wolfvin commented Jun 28, 2026

Copy link
Copy Markdown
Owner

Summary

Closes #30. Implements all three deliverables from the issue:

  1. cache_info block in vuln-scan output{last_refresh, age_hours, ttl_hours, is_stale, stale_packages[]} so agents can decide whether to trust cached CVE data.
  2. --refresh flag — bypasses the OSV cache and forces fresh OSV.dev API calls for every package. Updates the cache with new results. Ignored in --offline mode.
  3. --max-age Nh flag — treats cache entries older than N hours as stale for this run only (per-run TTL override; stored TTL unchanged). Accepts Nh/Nm/Ns/Nd/bare-int (hours).

This completes the Phase 1 roadmap (#21) checklist item: "Fix vuln DB staleness (OSV.dev API, update scheduler)".

Changes

scripts/osv_client.py (foundation)

  • OSVCache.peek(key) — new method. Returns (response, timestamp, ttl) WITHOUT TTL check or deletion. This is what --max-age relies on to apply a per-run TTL threshold without mutating stored state.
  • OSVClient.query_packages(packages, force_refresh=False, max_age=None) — new optional params. force_refresh bypasses cache; max_age uses peek() + custom threshold. Behaviour unchanged when both unset.
  • OSVClient._parse_cached_response(cached, package) — new private helper. Factors the two-shape cache parsing (vuln IDs vs full vuln dicts) out of query_packages so all three code paths share it. Zero dead code.
  • OSVClient.get_cache_info(packages) — new method. Returns the cache_info dict. Missing entries treated as stale; unsupported ecosystems skipped; stale_packages sorted for deterministic output.

scripts/commands/vuln_scan.py

  • _parse_max_age(raw) — new helper. Parses 6h/30m/2d/90s/bare-int into seconds.
  • --refresh and --max-age CLI flags added.
  • execute() validates --max-age and forwards refresh/max_age to the engine. Invalid --max-age returns a structured {status:'error', error:'invalid_argument'} dict.

scripts/vulnscan_engine.py

  • scan_vulnerabilities() gains refresh and max_age params.
  • Computes cache_info after the OSV query (3 paths: success → get_cache_info(); no packages → empty shape; OSV exception → empty shape with error).
  • cache_info added to return dict (additive — no existing key removed).

tests/test_vuln_staleness.py (new, 39 tests)

All network-free (API calls mocked via unittest.mock.patch.object). Covers:

  • _parse_max_age valid/invalid forms (11 tests)
  • OSVCache.peek — missing, tuple shape, TTL-agnostic vs get(), corrupt-JSON deletion (4 tests)
  • get_cache_info — empty, all-stale, all-fresh, mixed, sorted, TTL (6 tests)
  • force_refresh — bypasses cache, uses cache, ignored offline (3 tests)
  • max_age — old→stale, young→fresh, stored TTL unchanged, 0=refresh (4 tests)
  • End-to-end scan_vulnerabilities on clean_app + vulnerable_app fixtures, shape, additive (4 tests)
  • CLI arg wiring (3 tests)

Docs

  • README.md — vuln-scan command row lists new flags + describes cache_info.
  • SKILL-QUICK.md — output shape table, command list, and a new "are CVE results fresh?" trigger-map row.
  • CHANGELOG.md — new section under [8.2.0] — Unreleased with Added / Changed / Non-Breaking / Migration Notes.

Test Results

PYTHONPATH=scripts python3 -m pytest tests/test_vuln_staleness.py -v
→ 39 passed in 0.80s

python3 -m pytest tests/ --ignore=tests/test_integration.py
→ 706 passed, 12 skipped, 1 failed (pre-existing)

The single failure (tests/test_architecture.py::TestArchitectureBasic::test_auto_scans_on_fresh_workspace) is pre-existing on main — verified by stashing this PR's changes and re-running. No new failures introduced.

Sample Output

vuln-scan on clean_app fixture (offline, no deps):

{
  "status": "ok",
  "cache_info": {
    "last_refresh": null,
    "age_hours": null,
    "ttl_hours": 24.0,
    "is_stale": false,
    "stale_packages": []
  },
  "osv_stats": {"packages_queried": 0, "vulnerabilities_found": 0}
}

vuln-scan on vulnerable_app fixture (offline, npm deps but no cache entries):

{
  "cache_info": {
    "last_refresh": null,
    "age_hours": null,
    "ttl_hours": 24.0,
    "is_stale": true,
    "stale_packages": ["express@4.17.0", "lodash@4.17.15"]
  }
}

Acceptance Criteria

  • vuln-scan output includes cache_info object with last_refresh, age_hours, is_stale, stale_packages (also ttl_hours)
  • --refresh flag forces fresh API calls and updates cache
  • --max-age flag overrides TTL for the current run
  • Tests: cache_info populated correctly, --refresh bypasses cache, --max-age treats fresh entries as stale
  • Docs: README + SKILL-QUICK mention the new flags; CHANGELOG entry under [8.2.0]

Constraints Met

  • Python 3.8+ compatible (no walrus operators, no dict | dict syntax).
  • No new dependencies (uses stdlib re, sqlite3, time, unittest.mock).
  • Zero dead code — _parse_cached_response factors out previously-inline logic; no unused helpers.
  • All new public methods have docstrings.
  • Existing vuln-scan output is unchanged — cache_info is purely additive.
  • Network calls only happen when --refresh is set OR cache entries are expired/missing/stale-per---max-age.

Out of Scope

Issue #30 stretch goal #4 (background refresh scheduler in the MCP serve process) is intentionally not implemented — the issue body says "Out of scope for Phase 1 — file as follow-up if needed". This PR delivers the three in-scope items.

Commits

  1. feat(vuln): --refresh + --max-age flags — CLI flags + engine plumbing
  2. feat(vuln): cache_info in outputosv_client.py foundation (peek, query_packages params, get_cache_info, _parse_cached_response)
  3. test(vuln): staleness tests — 39 tests
  4. docs(vuln): README + SKILL-QUICK + CHANGELOG

worker-4-a added 4 commits June 28, 2026 08:38
Issue #30 deliverables 2 & 3: add --refresh and --max-age CLI flags
to vuln-scan, wiring them through scan_vulnerabilities() into the
OSV client.

- scripts/commands/vuln_scan.py:
  - New _parse_max_age() helper: parses duration strings like '6h',
    '30m', '2d', '90s', or a bare integer (interpreted as hours,
    matching --osv-ttl semantics). Returns seconds. Raises
    ValueError on invalid input.
  - New --refresh flag: bypass OSV cache, force fresh OSV API calls
    for every package. Updates the cache with new results. Ignored
    in --offline mode.
  - New --max-age flag: treat OSV cache entries older than the given
    duration as stale for this run only. Overrides the default 24h
    TTL for this run; does not change the stored TTL.
  - execute() now validates --max-age and forwards refresh/max_age
    to scan_vulnerabilities(). Invalid --max-age returns a structured
    {status:'error', error:'invalid_argument'} dict instead of raising.

- scripts/vulnscan_engine.py:
  - scan_vulnerabilities() gains refresh and max_age params.
  - Forwards them to osv_client.query_packages(force_refresh=,
    max_age=).
  - Computes a cache_info block (issue #30 deliverable 1) after the
    OSV query and includes it in the return dict. Three code paths
    produce cache_info: (a) successful query → from
    osv_client.get_cache_info(), (b) no packages → empty shape,
    (c) OSV exception → empty shape with error field.
  - Docstring updated to document refresh, max_age, and cache_info.

The cache_info block is additive — no existing vuln-scan output key
is removed or renamed.

Depends on osv_client.py changes (next commit) for the
get_cache_info() method and query_packages(force_refresh, max_age)
signature.
Issue #30 deliverable 1: surface OSV cache freshness info in the
vuln-scan output so agents can decide whether to trust the cached
CVE data or trigger a refresh.

- scripts/osv_client.py:
  - OSVCache.peek(key): new method. Returns the raw
    (response, timestamp, ttl) tuple WITHOUT applying the stored TTL
    or deleting the entry. Corrupt entries (invalid JSON) are deleted
    and treated as missing. This is what --max-age relies on to apply
    a per-run TTL threshold without mutating stored state.
  - OSVClient.query_packages(packages, force_refresh=False,
    max_age=None): new params.
    - force_refresh=True bypasses the cache entirely and forces a
      fresh API call for every package (issue #30 --refresh flag).
      Silently ignored in offline mode (no network to refresh from).
    - max_age=N (seconds) uses peek() to apply a per-run TTL
      threshold: entries older than N seconds are re-fetched; the
      stored TTL is unchanged (issue #30 --max-age flag). max_age=0
      is equivalent to force_refresh for cached entries.
    - Behaviour is unchanged when both are unset (normal TTL-based
      cache.get() path).
  - OSVClient._parse_cached_response(cached, package): new helper.
    Factors the two-shape cache parsing (list of vuln IDs vs list of
    full vuln dicts) out of query_packages so all three code paths
    (normal, force_refresh, max_age) share it.
  - OSVClient.get_cache_info(packages): new method. Returns the
    cache_info dict specified in issue #30:
      {
        last_refresh: ISO 8601 UTC of most-recent cache entry,
        age_hours:    age in hours of most-recent entry,
        ttl_hours:    self.cache.ttl / 3600,
        is_stale:     True if any package's entry is past TTL/missing,
        stale_packages: ['name@version', ...] (sorted, deterministic),
      }
    Packages with unsupported ecosystems (osv_client.ECOSYSTEM_MAP
    returns None) are skipped. Missing entries are treated as stale.

The cache_info block is additive — existing OSVClient public methods
(query_single, batch_query, get_stats) keep their signatures and
return shapes.
Issue #30 acceptance criterion: 'Tests: cache_info populated
correctly, --refresh bypasses cache, --max-age treats fresh entries
as stale'.

39 tests across 7 classes, all network-free (API calls mocked via
unittest.mock.patch.object(OSVClient, '_batch_query_api', ...)):

- TestParseMaxAge (11 tests): _parse_max_age handles '6h'/'30m'/'2d'/
  '90s'/bare-int/'1.5h'/'1H'/whitespace, None, and rejects 'abc'/
  '-5h'/''/'h'/'5x'/'5hrs'.
- TestOSVCachePeek (4 tests): missing key, entry tuple shape, TTL-
  agnostic return (vs get()), corrupt-JSON deletion.
- TestGetCacheInfo (6 tests): empty packages, all-stale (no entries),
  all-fresh, one-stale-mixed, sorted stale_packages, ttl_hours follows
  cache TTL.
- TestForceRefresh (3 tests): bypasses cache (API called), no-refresh
  uses cache (API not called), ignored in offline mode.
- TestMaxAge (4 tests): old entry → stale (API called), young entry →
  fresh (API skipped), stored TTL unchanged after run, max_age=0 acts
  like refresh.
- TestScanVulnerabilitiesCacheInfo (4 tests): clean_app (no deps) has
  cache_info, vulnerable_app (npm deps, offline) reports all stale,
  cache_info has exactly the 5 specified keys, cache_info is additive
  (pre-existing output keys still present).
- TestVulnScanCLI (3 tests): execute() forwards refresh+max_age,
  invalid max_age returns structured error, no-flags passes defaults.

Run: PYTHONPATH=scripts python3 -m pytest tests/test_vuln_staleness.py -v
Result: 39 passed in 0.77s
Issue #30 acceptance criterion: 'Docs: README + SKILL-QUICK mention
the new flags; CHANGELOG entry under [8.2.0]'.

- README.md: vuln-scan command row now lists --refresh / --max-age
  flags and describes the cache_info output block.
- SKILL-QUICK.md:
  - Output shape table: split vuln-scan out of the
    a11y/css-deep/regex-audit row to show its cache_info shape.
  - Security command list: vuln-scan entry now shows the new flags.
  - Trigger map: new 'are CVE results fresh?' row pointing agents at
    cache_info.is_stale and the --refresh / --max-age follow-up.
- CHANGELOG.md: new 'OSV Cache Staleness Flags + cache_info Output
  (issue #30)' section under [8.2.0] — Unreleased, with Added /
  Changed / Non-Breaking / Migration Notes subsections matching the
  format used by the issue #25 entry above it.
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@Wolfvin Wolfvin merged commit 17c54be into main Jun 28, 2026
2 of 8 checks passed
@Wolfvin Wolfvin deleted the feat/issue-30-vuln-staleness branch June 28, 2026 08:50
@sonarqubecloud

Copy link
Copy Markdown

Quality Gate Failed Quality Gate failed

Failed conditions
1 Security Hotspot
C Reliability Rating on New Code (required ≥ A)

See analysis details on SonarQube Cloud

Catch issues before they fail your Quality Gate with our IDE extension SonarQube for IDE

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[FEATURE] OSV cache auto-refresh scheduler + staleness flag in vuln-scan output

1 participant