docs: document the doc-sync tombstone propagation for Purge
Build Debug APK / build (push) Failing after 11m48s
Build Debug APK / build (push) Failing after 11m48s
Records the ALPN bump to /syn/doc-sync/3, the new DocumentTombstoneRecord/ DocumentTombstoneBatch wire messages, and the resurrection-guard step in apply_records, alongside a dedicated section explaining how this durable path relates to the gossip PurgeIntent fast path.
This commit is contained in:
+57
-1
@@ -35,11 +35,13 @@ Uses iroh's existing `Endpoint` — already set up with the N0 preset, NAT
|
||||
traversal, and pkarr-based peer discovery. Nodes find each other via the gossip
|
||||
ensemble discovery that is already implemented.
|
||||
|
||||
**ALPN:** `/syn/doc-sync/2`
|
||||
**ALPN:** `/syn/doc-sync/3`
|
||||
|
||||
The version suffix in the ALPN string provides protocol-level version
|
||||
negotiation at the transport layer; we don't need a version field in every
|
||||
message envelope. Version 2 adds the reverse-pull phase (see Session flow).
|
||||
Version 3 adds document-tombstone propagation (see "Document tombstones
|
||||
(Purge)" below).
|
||||
|
||||
**Stream model:** Each sync session uses two bidirectional QUIC streams on
|
||||
the same connection. Stream 0 is opened by the requester for the pull phase;
|
||||
@@ -334,6 +336,22 @@ BlobRefRecord {
|
||||
vault_id: [u8; 16],
|
||||
size_bytes: u64,
|
||||
}
|
||||
|
||||
/// Announces that a document was purged, so the deletion propagates and a
|
||||
/// stale copy of it does not resurrect from this node. See "Document
|
||||
/// tombstones (Purge)" below.
|
||||
DocumentTombstoneRecord {
|
||||
id: [u8; 16],
|
||||
deleted_at: i64,
|
||||
origin_node: [u8; 32],
|
||||
}
|
||||
|
||||
/// All of the sender's document tombstones matching the request's `since`
|
||||
/// filter, sent in one shot right after SyncHeader — unpaginated, since
|
||||
/// tombstone volume is expected to be small for a personal-scale store.
|
||||
DocumentTombstoneBatch {
|
||||
tombstones: Vec<DocumentTombstoneRecord>,
|
||||
}
|
||||
```
|
||||
|
||||
## Session flow
|
||||
@@ -345,6 +363,7 @@ Requester (A) Responder (B)
|
||||
|-- SyncRequest ──────────────────────> | (filter + optional resume cursor)
|
||||
| | [COUNT query, build estimates]
|
||||
| <────────────────────── SyncHeader ---| (total count + blob/artifact bytes)
|
||||
| <──────────── DocumentTombstoneBatch --| (all purged ids since `since`) → A applies them
|
||||
| | [page 1 query]
|
||||
| <──────────────── DocumentChunk(1) ---| (documents + cursor, is_final=false)
|
||||
| <──────────────── DocumentChunk(2) ---|
|
||||
@@ -354,6 +373,7 @@ Requester (A) Responder (B)
|
||||
| ── stream 1: B pulls from A ───────── |
|
||||
| <────────────────── SyncRequest ---- | (B opens stream 1, requests A's docs)
|
||||
|-- SyncHeader ───────────────────────> |
|
||||
|-- DocumentTombstoneBatch ───────────> | (B applies them)
|
||||
|-- DocumentChunk(1) ─────────────────> |
|
||||
| ... |
|
||||
|-- DocumentChunk(M) ─────────────────> | (is_final=true) → B promotes session
|
||||
@@ -386,6 +406,9 @@ When the chunk with `is_final=true` arrives, the receiver calls
|
||||
parent within the batch. Records whose parent is not in the batch (already in
|
||||
the live DB, or a root document) are treated as roots and emitted first.
|
||||
3. **Apply** each record in sorted order:
|
||||
- **Resurrection guard**: skip the record entirely if its id (or its
|
||||
`parent_id`) has a `document_tombstones` row — see "Document tombstones
|
||||
(Purge)" below.
|
||||
- `add_document` (INSERT OR IGNORE). Parents are guaranteed to exist before
|
||||
children, so no FK violation can occur.
|
||||
- `sync_receive_property` for each `PropertyRecord`. Keeps whichever row has
|
||||
@@ -395,6 +418,39 @@ When the chunk with `is_final=true` arrives, the receiver calls
|
||||
- `mark_present(doc_id, sender_pubkey)` and `mark_present(doc_id, local_node)`.
|
||||
4. **Mark session complete** and delete the staging rows.
|
||||
|
||||
## Document tombstones (Purge)
|
||||
|
||||
Purging a document (see "Removing a Document vs. Purging" in
|
||||
`docs/data-model.md`) deletes its row locally and records a
|
||||
`document_tombstones` row (id, deleted_at, origin_node) so the deletion
|
||||
sticks — this protocol is the durable propagation path for peers who were
|
||||
offline when the purge's gossip broadcast (`p2p::gossip::PurgeIntent`)
|
||||
happened, or missed it entirely. Whichever path arrives first wins; the other
|
||||
is a no-op.
|
||||
|
||||
Unlike artifact tombstones, which ride along inside the owning
|
||||
`DocumentRecord` (see `TombstoneRecord`), a document tombstone has no living
|
||||
document to be a member of, so it can't be embedded that way. Instead
|
||||
`build_document_tombstone_batch` sends every local tombstone matching the
|
||||
request's `since` filter as one `DocumentTombstoneBatch`, immediately after
|
||||
`SyncHeader` and before any `DocumentChunk` — in both pull directions, since
|
||||
both sides call the same `serve_pull_response`.
|
||||
|
||||
The receiver applies the batch (`apply_document_tombstone_batch`) before
|
||||
processing chunks, via `DocumentStore::apply_tombstone`, which:
|
||||
- Recursively purges any **locally known** children of the tombstoned id, not
|
||||
just the ids the sender happened to list. This self-heals a divergent tree
|
||||
view — a receiver with children the sender never learned about still ends
|
||||
up with them purged too.
|
||||
- Is idempotent: re-applying a tombstone for an already-purged id just keeps
|
||||
the newest `deleted_at`.
|
||||
|
||||
Applying tombstones before chunks matters: the resurrection guard in
|
||||
`apply_records` (step 3 above) checks `document_tombstones` before inserting,
|
||||
so a same-session `DocumentChunk` that (from a stale sender's point of view)
|
||||
still contains the purged document is correctly dropped instead of
|
||||
resurrecting it.
|
||||
|
||||
## Sender authentication
|
||||
|
||||
The sender's `NodePubkey` is known from `Connection::remote_id()`, mapped
|
||||
|
||||
Reference in New Issue
Block a user