GeBüV anchoring pipeline: Merkle + OpenTimestamps + WORM provisioning (Säule 2)#3934
Merged
TaprootFreak merged 8 commits intoJun 18, 2026
Conversation
- merkle.ts: sha256, buildMerkleRoot, merkleInclusionProof, verifyMerkleProof (SHA-256, parent=sha256(left||right), odd-node duplicated, 0=throw/1=leaf) - opentimestamps.service.ts: stamp/upgrade/verify wrapper over the public calendars - merkle.spec.ts: 19 unit tests (roots, per-leaf inclusion proofs, tamper detection) - add opentimestamps dependency
- ArchiveBatch / ArchiveFile entities (IEntity base, BaseRepository pattern), migration for both tables + unique(bucket,name) + batch FK - ArchiveService: recordHash (idempotent), anchorPending (Merkle root over pending hashes -> batch -> OTS stamp), upgradeBatches (Bitcoin attestation), verifyDocument (hash match + Merkle inclusion proof + OTS verify) - ArchiveModule; archive.service.spec.ts (round-trip + tamper, real Merkle, mocked OTS)
…ule 2, stage 3) - Capture SHA-256 on compliance uploads: kyc-document (kyc) + fiat-output (EP2) call archiveService.recordHash after uploadBlob; support bucket untouched - ArchiveScheduler (@DfxCron + process guard): daily anchorPending, hourly upgradeBatches; ARCHIVE_ANCHOR/ARCHIVE_UPGRADE process entries - scripts/storage/provision-bucket.ts: create bucket with Object Lock + versioning + COMPLIANCE default retention (idempotent); note for dynamic EP2 - wire ArchiveModule into kyc + fiat-output modules; scheduler spec
The opentimestamps lib pulls request->request-promise->uuid via the legacy 'uuid/v4' subpath, which the repo-wide uuid@9 override breaks (ERR_PACKAGE_PATH_NOT_EXPORTED). Scope a uuid@3.4.0 override under 'request' only (nested override) so its calendar HTTP works, without touching the app's uuid@9. Surfaced by the m5me integration test.
…upgrade persistence) - recordHash: never overwrite the hash of an already-anchored file; identical -> no-op, differing -> hard error+log (deterministic blob names could otherwise silently rewrite an anchored leaf and break verifyDocument) - KYC + EP2: recordHash is best-effort (try/catch + log), never rolls back the completed upload; EP2 sets reportCreated before anchoring to avoid a WORM re-PUT retry trap - upgradeBatches: persist upgraded .ots bytes whenever they change, even while still pending; confirm only on a real Bitcoin attestation - verify result exposes confirmed vs pending; verifyDocument loads batch via relations (removed dead branch); tests extended
- s3-storage.service.spec (aws-sdk-client-mock): config validation, listBlobs pagination + per-key HeadObject, getBlob empty-body guard, uploadBlob, copyBlobs CopySource encoding -> 100% - mock-storage.service.spec: roundtrip, prefix filter, dummy-file fallbacks -> 100% - storage.factory.spec: LOC->Mock, otherwise->S3 -> 100% - opentimestamps.service.spec (lib mocked): stamp/upgrade/verify, confirmed vs pending -> 100% - archive.service.spec: closed upgrade-no-change + verifyDocument confirmed branches - kyc-document + fiat-output wiring specs: recordHash called with sha256, best-effort (upload not rolled back; reportCreated set before anchoring) - add aws-sdk-client-mock devDependency
- archive.service.spec fakeStore now honors relations:['batch'] (batch only hydrated when requested) + explicit assertion that recordHash/verifyDocument request the relation -> removing it in prod now fails the suite - fiat-output spec asserts invocationCallOrder uploadBlob < reportCreated < recordHash (WORM re-PUT deadlock guard) - fiat-output spec covers the routeId -> sell.id fallback Mutation-checked: each guard fails when the corresponding prod behavior is removed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Was
Säule 2 des GeBüV-Archivs: die Anchoring-Pipeline, die aus dem Storage (Säule 1, #3921) ein revisionssicheres Archiv macht. Aufbauend auf der
StorageService-Abstraktion.Komponenten
anchoring/merkle.ts): SHA-256,buildMerkleRoot,merkleInclusionProof,verifyMerkleProof(parent =sha256(left||right), ungerader Knoten dupliziert).anchoring/opentimestamps.service.ts):stamp/upgrade/verifyüber die öffentlichen Calendars → Bitcoin. Nur der Hash verlässt das Haus.recordHash(idempotent, schützt geankerte Leaves),anchorPending(Tages-Merkle-Root → OTS-Stamp, transaktional),upgradeBatches(Bitcoin-Attestation),verifyDocument(Hash-Match + Inclusion-Proof + OTS).ArchiveBatch/ArchiveFile(IEntity/BaseRepository) + Migration.@DfxCron+ Process-Guard): täglichanchorPending, stündlichupgradeBatches.scripts/storage/provision-bucket.ts): Bucket mit Object Lock + Versioning + COMPLIANCE-Default-Retention, idempotent.Lokal auf m5me getestet (real)
tsc --noEmitInvalidRequest); Overwrite → neue Version, Original unveränderlich.ots(700 B),verify → pendingInhärentes Limit: die finale Bitcoin-Attestation reift in Stunden (Blockzeit) — synchron nicht abwartbar; bis dahin korrekt
pending.Review
Zwei adversariale Subagenten-Runden (Korrektheit + Konventionen). Runde 1 fand 1 Major (recordHash überschrieb geankerte Hashes) + Minors (Upload-Kopplung, EP2-WORM-Retry-Falle, upgrade-Persistenz) → alle gefixt + Tests ergänzt. Runde 2: Freigabe, keine offenen Findings.
Bewusst offen / Hinweise
uuid-Override: opentimestamps zieht das deprecateterequest(→uuid/v4); per nested Overriderequest→uuid@3.4.0gelöst, App-weitesuuid@9unangetastet. Mittelfristig: OTS-Calendar-I/O hinter fetch-basierten Transport legen, umrequestrepo-weit loszuwerden.verifyDocumentist bisher ein internes Audit-Primitiv (kein Endpoint) — On-Demand-Verifikation per Admin-Endpoint ist Folgeschritt.Status
Draft. Nichts deployt. Build/Tests grün, real auf m5me verifiziert.