feat(vuln): OSV cache staleness + refresh flags — closes #30#45
Merged
Conversation
added 4 commits
June 28, 2026 08:38
Issue #30 deliverables 2 & 3: add --refresh and --max-age CLI flags to vuln-scan, wiring them through scan_vulnerabilities() into the OSV client. - scripts/commands/vuln_scan.py: - New _parse_max_age() helper: parses duration strings like '6h', '30m', '2d', '90s', or a bare integer (interpreted as hours, matching --osv-ttl semantics). Returns seconds. Raises ValueError on invalid input. - New --refresh flag: bypass OSV cache, force fresh OSV API calls for every package. Updates the cache with new results. Ignored in --offline mode. - New --max-age flag: treat OSV cache entries older than the given duration as stale for this run only. Overrides the default 24h TTL for this run; does not change the stored TTL. - execute() now validates --max-age and forwards refresh/max_age to scan_vulnerabilities(). Invalid --max-age returns a structured {status:'error', error:'invalid_argument'} dict instead of raising. - scripts/vulnscan_engine.py: - scan_vulnerabilities() gains refresh and max_age params. - Forwards them to osv_client.query_packages(force_refresh=, max_age=). - Computes a cache_info block (issue #30 deliverable 1) after the OSV query and includes it in the return dict. Three code paths produce cache_info: (a) successful query → from osv_client.get_cache_info(), (b) no packages → empty shape, (c) OSV exception → empty shape with error field. - Docstring updated to document refresh, max_age, and cache_info. The cache_info block is additive — no existing vuln-scan output key is removed or renamed. Depends on osv_client.py changes (next commit) for the get_cache_info() method and query_packages(force_refresh, max_age) signature.
Issue #30 deliverable 1: surface OSV cache freshness info in the vuln-scan output so agents can decide whether to trust the cached CVE data or trigger a refresh. - scripts/osv_client.py: - OSVCache.peek(key): new method. Returns the raw (response, timestamp, ttl) tuple WITHOUT applying the stored TTL or deleting the entry. Corrupt entries (invalid JSON) are deleted and treated as missing. This is what --max-age relies on to apply a per-run TTL threshold without mutating stored state. - OSVClient.query_packages(packages, force_refresh=False, max_age=None): new params. - force_refresh=True bypasses the cache entirely and forces a fresh API call for every package (issue #30 --refresh flag). Silently ignored in offline mode (no network to refresh from). - max_age=N (seconds) uses peek() to apply a per-run TTL threshold: entries older than N seconds are re-fetched; the stored TTL is unchanged (issue #30 --max-age flag). max_age=0 is equivalent to force_refresh for cached entries. - Behaviour is unchanged when both are unset (normal TTL-based cache.get() path). - OSVClient._parse_cached_response(cached, package): new helper. Factors the two-shape cache parsing (list of vuln IDs vs list of full vuln dicts) out of query_packages so all three code paths (normal, force_refresh, max_age) share it. - OSVClient.get_cache_info(packages): new method. Returns the cache_info dict specified in issue #30: { last_refresh: ISO 8601 UTC of most-recent cache entry, age_hours: age in hours of most-recent entry, ttl_hours: self.cache.ttl / 3600, is_stale: True if any package's entry is past TTL/missing, stale_packages: ['name@version', ...] (sorted, deterministic), } Packages with unsupported ecosystems (osv_client.ECOSYSTEM_MAP returns None) are skipped. Missing entries are treated as stale. The cache_info block is additive — existing OSVClient public methods (query_single, batch_query, get_stats) keep their signatures and return shapes.
Issue #30 acceptance criterion: 'Tests: cache_info populated correctly, --refresh bypasses cache, --max-age treats fresh entries as stale'. 39 tests across 7 classes, all network-free (API calls mocked via unittest.mock.patch.object(OSVClient, '_batch_query_api', ...)): - TestParseMaxAge (11 tests): _parse_max_age handles '6h'/'30m'/'2d'/ '90s'/bare-int/'1.5h'/'1H'/whitespace, None, and rejects 'abc'/ '-5h'/''/'h'/'5x'/'5hrs'. - TestOSVCachePeek (4 tests): missing key, entry tuple shape, TTL- agnostic return (vs get()), corrupt-JSON deletion. - TestGetCacheInfo (6 tests): empty packages, all-stale (no entries), all-fresh, one-stale-mixed, sorted stale_packages, ttl_hours follows cache TTL. - TestForceRefresh (3 tests): bypasses cache (API called), no-refresh uses cache (API not called), ignored in offline mode. - TestMaxAge (4 tests): old entry → stale (API called), young entry → fresh (API skipped), stored TTL unchanged after run, max_age=0 acts like refresh. - TestScanVulnerabilitiesCacheInfo (4 tests): clean_app (no deps) has cache_info, vulnerable_app (npm deps, offline) reports all stale, cache_info has exactly the 5 specified keys, cache_info is additive (pre-existing output keys still present). - TestVulnScanCLI (3 tests): execute() forwards refresh+max_age, invalid max_age returns structured error, no-flags passes defaults. Run: PYTHONPATH=scripts python3 -m pytest tests/test_vuln_staleness.py -v Result: 39 passed in 0.77s
Issue #30 acceptance criterion: 'Docs: README + SKILL-QUICK mention the new flags; CHANGELOG entry under [8.2.0]'. - README.md: vuln-scan command row now lists --refresh / --max-age flags and describes the cache_info output block. - SKILL-QUICK.md: - Output shape table: split vuln-scan out of the a11y/css-deep/regex-audit row to show its cache_info shape. - Security command list: vuln-scan entry now shows the new flags. - Trigger map: new 'are CVE results fresh?' row pointing agents at cache_info.is_stale and the --refresh / --max-age follow-up. - CHANGELOG.md: new 'OSV Cache Staleness Flags + cache_info Output (issue #30)' section under [8.2.0] — Unreleased, with Added / Changed / Non-Breaking / Migration Notes subsections matching the format used by the issue #25 entry above it.
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.




Summary
Closes #30. Implements all three deliverables from the issue:
cache_infoblock in vuln-scan output —{last_refresh, age_hours, ttl_hours, is_stale, stale_packages[]}so agents can decide whether to trust cached CVE data.--refreshflag — bypasses the OSV cache and forces fresh OSV.dev API calls for every package. Updates the cache with new results. Ignored in--offlinemode.--max-age Nhflag — treats cache entries older than N hours as stale for this run only (per-run TTL override; stored TTL unchanged). AcceptsNh/Nm/Ns/Nd/bare-int (hours).This completes the Phase 1 roadmap (#21) checklist item: "Fix vuln DB staleness (OSV.dev API, update scheduler)".
Changes
scripts/osv_client.py(foundation)OSVCache.peek(key)— new method. Returns(response, timestamp, ttl)WITHOUT TTL check or deletion. This is what--max-agerelies on to apply a per-run TTL threshold without mutating stored state.OSVClient.query_packages(packages, force_refresh=False, max_age=None)— new optional params.force_refreshbypasses cache;max_ageusespeek()+ custom threshold. Behaviour unchanged when both unset.OSVClient._parse_cached_response(cached, package)— new private helper. Factors the two-shape cache parsing (vuln IDs vs full vuln dicts) out ofquery_packagesso all three code paths share it. Zero dead code.OSVClient.get_cache_info(packages)— new method. Returns thecache_infodict. Missing entries treated as stale; unsupported ecosystems skipped;stale_packagessorted for deterministic output.scripts/commands/vuln_scan.py_parse_max_age(raw)— new helper. Parses6h/30m/2d/90s/bare-int into seconds.--refreshand--max-ageCLI flags added.execute()validates--max-ageand forwardsrefresh/max_ageto the engine. Invalid--max-agereturns a structured{status:'error', error:'invalid_argument'}dict.scripts/vulnscan_engine.pyscan_vulnerabilities()gainsrefreshandmax_ageparams.cache_infoafter the OSV query (3 paths: success →get_cache_info(); no packages → empty shape; OSV exception → empty shape witherror).cache_infoadded to return dict (additive — no existing key removed).tests/test_vuln_staleness.py(new, 39 tests)All network-free (API calls mocked via
unittest.mock.patch.object). Covers:_parse_max_agevalid/invalid forms (11 tests)OSVCache.peek— missing, tuple shape, TTL-agnostic vsget(), corrupt-JSON deletion (4 tests)get_cache_info— empty, all-stale, all-fresh, mixed, sorted, TTL (6 tests)force_refresh— bypasses cache, uses cache, ignored offline (3 tests)max_age— old→stale, young→fresh, stored TTL unchanged,0=refresh (4 tests)scan_vulnerabilitiesonclean_app+vulnerable_appfixtures, shape, additive (4 tests)Docs
README.md— vuln-scan command row lists new flags + describescache_info.SKILL-QUICK.md— output shape table, command list, and a new "are CVE results fresh?" trigger-map row.CHANGELOG.md— new section under[8.2.0] — Unreleasedwith Added / Changed / Non-Breaking / Migration Notes.Test Results
The single failure (
tests/test_architecture.py::TestArchitectureBasic::test_auto_scans_on_fresh_workspace) is pre-existing onmain— verified by stashing this PR's changes and re-running. No new failures introduced.Sample Output
vuln-scanonclean_appfixture (offline, no deps):{ "status": "ok", "cache_info": { "last_refresh": null, "age_hours": null, "ttl_hours": 24.0, "is_stale": false, "stale_packages": [] }, "osv_stats": {"packages_queried": 0, "vulnerabilities_found": 0} }vuln-scanonvulnerable_appfixture (offline, npm deps but no cache entries):{ "cache_info": { "last_refresh": null, "age_hours": null, "ttl_hours": 24.0, "is_stale": true, "stale_packages": ["express@4.17.0", "lodash@4.17.15"] } }Acceptance Criteria
vuln-scanoutput includescache_infoobject withlast_refresh,age_hours,is_stale,stale_packages(alsottl_hours)--refreshflag forces fresh API calls and updates cache--max-ageflag overrides TTL for the current runConstraints Met
dict | dictsyntax).re,sqlite3,time,unittest.mock)._parse_cached_responsefactors out previously-inline logic; no unused helpers.cache_infois purely additive.--refreshis set OR cache entries are expired/missing/stale-per---max-age.Out of Scope
Issue #30 stretch goal #4 (background refresh scheduler in the MCP
serveprocess) is intentionally not implemented — the issue body says "Out of scope for Phase 1 — file as follow-up if needed". This PR delivers the three in-scope items.Commits
feat(vuln): --refresh + --max-age flags— CLI flags + engine plumbingfeat(vuln): cache_info in output—osv_client.pyfoundation (peek,query_packagesparams,get_cache_info,_parse_cached_response)test(vuln): staleness tests— 39 testsdocs(vuln): README + SKILL-QUICK + CHANGELOG