The evidence bundle and the search export
Two documents carry records out of a workspace for offline checking. The bundle carries whole chains with the checkpoints, proofs and reports that go with them. The search export carries the results of one search. Each names the keys it was signed with; a verifier pins them by fingerprint (log.md).
Vectors: 18-bundle, 17-search-export.
The bundle
A bundle is a tar archive of regular files:
manifest.json, the manifest below;- one JSON-lines file per chain, of record lines in
seqorder (records.md), at the path the manifest names; - the signed reports and the credential ledger file the manifest lists;
- the timestamp tokens the manifest lists, each a DER file (timestamps.md).
A verifier reads regular files only, by their cleaned slash-separated paths, and refuses a manifest over 16 MiB or another file over 1 GiB. antler-verify also accepts the directory an archive was extracted into.
The manifest
Format antler-bundle-v1. Members, in order:
| Member | Type | Meaning |
|---|---|---|
format | string | antler-bundle-v1 |
org_id | string | The workspace's id |
origin | string | The origin of the workspace's log, which every checkpoint must name |
created_at, created_by | string | When and by whom the bundle was made |
keys | object | The published keys: {"keys":[{purpose, key_id, key}, …]} |
checkpoint_key | string | The checkpoint key, as a verifier key |
checkpoint | string | The newest signed checkpoint note; it covers every exported record |
checkpoints | list of strings | The stored notes, oldest first |
consistency | list | Consistency proofs {from, to, hashes}, one from each checkpoint to the next |
chains | list | One entry per chain (below) |
workspace, region | string | The workspace and its region; optional |
fingerprints | map | The current keys' fingerprints by purpose; optional |
window | object | {from, to}, RFC 3339: the records at or after from and before to; absent for a whole ledger |
reports | list | {file, kind, sha256} for each signed report in the bundle; optional |
ledger | object | {file, sha256} of the credential ledger file; optional |
timestamps | list | {size, file} for each timestamp token in the bundle, oldest first: the size of the checkpoint it is over and the token's file; optional |
A chain entry:
| Member | Type | Meaning |
|---|---|---|
device_id | string | The chain: a device's id, or server |
public_key | string | The key the chain is signed with, base64: the device's key, or the audit key for the server chain |
file | string | The chain file's path in the bundle |
first_seq, last_seq | integer | The first and last seq of the chain as exported |
head_hash | string | The hash of the record at last_seq |
server_redacted | list of integers | The seqs whose stored copy the server redacted; optional |
missing | list of {from, to} | Inclusive ranges of seqs in [first_seq, last_seq] that the file does not hold: declared losses, or records past retention; optional |
before | integer | Set when the file starts after seq 1 because the window does: records 1 to before are older than the window; optional |
inclusion | object | The inclusion proof of the chain's last record in checkpoint; optional |
receipt | object | The newest ingest receipt of the chain, a signed object; optional |
Checking a bundle
With the keys pinned (fingerprints matched against keys and checkpoint_key, or the keys of an enrolled computer):
- Checkpoints. Verify every note in
checkpoints, andcheckpointwhen it is not among them, with the checkpoint key underorigin. A note that does not verify is a failure. Two of the same size with different roots are a failure. Order them by size: each consecutive pair must have a consistency proof inconsistencythat checks. The newest checkpoint is the largest that verified. - Each chain. Find its key: a key pinned for the device; for the server chain, the pinned audit key equal to
public_key; otherwisepublic_keyitself, which is accepted only as step 5 says. The key of a device chain must derivedevice_id. A bundle that names no usable key for a chain is a failure. Parse the file's lines strictly, check every record and link (records.md), starting fromseq1, or from the file's first record whenmissingandbeforecover everyseqbefore it. Every record must name the entry'sdevice_id. - The head. The file must end at
last_seqwithhead_hash. When it ends earlier andmissingcovers the rest, the end stays unproven; otherwise it is a failure. - The end. The chain's end is proven by
inclusion, when the proof is for the file's last record, against the newest checkpoint, and checks; or byreceipt, when it verifies with a pinned receipt key, names the file's lastseqand hash, and itslog_sizeis at least the newest checkpoint's size. An inclusion proof that does not check, or a receipt that does not verify, is a failure. A receipt that names aseqthe file holds must name that record's hash. A receipt ahead of the file, in a log the newest checkpoint already covers, with the records between not listed asmissing, is a failure: the server held records the bundle leaves out. - Device keys taken from the bundle. When the server's keys are pinned by fingerprint, a device chain's
public_keyis accepted when step 4 proved its head with the pinned log or a pinned receipt. A chain whose key is neither pinned nor vouched for this way is a failure. - Reports. Each file under
reportsmust be in the bundle with the listedsha256, and verify as a signed report (reports.md) with the pinned keys. A report that is itself unproven does not change the bundle's outcome; a report with a problem makes it invalid. - The ledger. The file under
ledgermust be in the bundle with the listedsha256. It is derived from the records and not signed. - Timestamps. Each file under
timestampsmust be in the bundle, the bundle must hold a verified checkpoint of the entry'ssize, and the token must check against that checkpoint and, when the reader pinned the authority's certificate, against it (timestamps.md). With a certificate pinned, a bundle without any token is a failure.
The bundle is valid when nothing failed and every chain's end is proven; unproven when nothing failed and some end is not; invalid otherwise. With no key pinned, signatures are not checked and the result is at best unproven.
A verifier told to expect a head hash or a last seq for a device chain requires the chain to end there, in place of step 4.
The witness file
An enrolled device keeps the checkpoints it verified in witness.jsonl, one JSON object a line: {"note":"<signed note>","seen_at":"<time>"}. Given a device's witness file, a verifier also checks, for each line: the note verifies; when its size is at most the bundle's newest checkpoint, the bundle holds a checkpoint of that size with the same root (another root proves the server showed the device a different log; a missing one is a failure, since an honest export holds every stored checkpoint). Notes newer than the bundle are counted and not checked. seen_at is the device's own note and proves nothing.
What nothing signs
These members of a manifest are stated by whoever made the bundle and are covered by no signature or hash. A verifier shows them as stated and relies on none of them: format, org_id, created_at, created_by, workspace, region, fingerprints, window, a chain's first_seq and a report entry's kind. Everything else in a manifest is either signed (notes, receipts, timestamp tokens), hashed (report and ledger files), or checked against signed or hashed data (keys against fingerprints, heads against the chain files, proofs against checkpoints, a timestamp's size against the checkpoint its token is over).
The records' own times are signed; a manifest's window is not. The fingerprints a manifest lists are never the ones a verifier pins.
The search export
A search export is a JSON-lines file: a header line, one line per record, and a trailer line.
The header
Format antler-search-export-v1. Members, in order:
| Member | Type | Meaning |
|---|---|---|
format | string | antler-search-export-v1 |
workspace, region | string | The workspace and its region; optional |
created_at, created_by | string | When and by whom it was made |
filters | list of strings | The names of the search's filters, sorted; never their values |
window | object | {from, to}: the window searched |
origin | string | The origin of the workspace's log |
checkpoint | string | The signed checkpoint note every proof checks against |
checkpoint_key | string | The checkpoint key, as a verifier key |
keys | object | The published keys |
fingerprints | map | The current keys' fingerprints; optional, never pinned from here |
chain_record | object | The inclusion proof, in checkpoint, of the export's own record in the server's chain |
truncated | boolean | Set when the search matched more records than the export holds |
A record line
{"line":"<the record line>","public_key":"<base64>","server_redacted":false,"proof":{"record_hash":"…","leaf":5,"tree_size":25,"hashes":["…"]}}
line is the record's line as a string; public_key the key that signed it (the audit key for a record of the server's chain); proof its inclusion proof in the header's checkpoint. server_redacted is present only when true. A record line is decoded strictly.
The trailer
{"signed":{"purpose":"report","key_id":"…","payload":"…","sig":"…"}}
a signed object of purpose report whose payload is, in order:
{"format":"antler-search-export-summary-v1","header_sha256":"<hex>","records":4,"lines_sha256":"<hex>","tree_size":25}
header_sha256is the SHA-256 of the header line without its line feed;recordsis the number of record lines;lines_sha256is the SHA-256 of the record lines as written, each with its line feed, in order;tree_sizeis the size of the header's checkpoint.
Checking a search export
- Pin the keys among the header's
keysandcheckpoint_key. - Verify
checkpointwith the checkpoint key underorigin. chain_recordmust be for a tree of the checkpoint's size and check against its root.- For each record line: parse
linestrictly as a record. For a device record,public_keymust derive itsdevice_id; for a record of the server's chain,public_keymust be the pinned audit key.proofmust be for the record's hash and the checkpoint's size, and check against its root. Then check the record's hash and signature, unless the line is markedserver_redacted. - Verify the trailer's signed object with the pinned report key, decode its payload strictly, and compare all four members with the file. A file with no trailer was cut.
Any failure makes the export invalid. It is valid when everything checks with pinned keys, and at best unproven without them. A search export holds records, not chains: it shows that each record is in the log the workspace signed, and that the workspace signed this set of them; it does not show that a device's chain has no other records.