The Manifest

What is a “pointer”?

A thing that references something in store, and a little bit of extra unique data about that item’s existence.

The “Store”s

Why are we using stores

• It’s a DAG so so that we can have easy deduplication while maintaining performance
• It allows for lazy loading of data by storing stuff across different stores
• We can easily define the data storage contracts of a given store. E.g. append only store

Backends

Goal

The “backend”

If you think about, every storage mechanism type has some shared functionality:

• You can allocate and dereference pointers to underlying data
• You can “cluster” stuff together so that you can load it faster

In a simple binary file, a pointer is an offset, and a cluster of stuff is a contiguous range (so we can easily load an entire page worth of items at once). But it doesn’t have to be so! In IPFS, a pointer is a content hash, and a cluster of stuff is a set of content hashes as an IPFS merkle DAG.

So why not make this generic?

The “vec” backend

pub struct VecBackend {
    page_store: PageStoreImpl,
    item_store: ItemStoreImpl,
    data_store: DataStoreImpl,
    signature_store: SignatureStoreImpl,
}

pub trait Backend: Sized {
    type Types: BackendTypes;
}

pub struct VecTypes;

impl BackendTypes for VecTypes {
    type Single<S: StoreTypes> = VecSinglePointer<S>;
    type Group<S: StoreTypes> = VecGroupPointer<S>;
}

pub struct VecSinglePointer<S: StoreTypes> {
    pub index: usize,
    pub unique: S::Unique,
}

pub struct VecGroupPointer<S: StoreTypes> {
    range: VecRange,
    uniques: Vec<S::Unique>,
}

The “frontend”

How the backend itself is accessed using backend-specific methods. This allows for, for example, easy interning.

The renderer

We iterate and draw! So simple!

let reader = DummyTDFBuilder::default()
    .add_page(vec![
        (
            ItemPrimitive::Shape(Shape {
                kind: ShapeKind::Circle,
            }),
            ItemUnique {
                position: Position { x: 1, y: 2 },
                ..Default::default()
            },
        ),
        (
            ItemPrimitive::TextBox(TextBox {
                content: "hi".into(),
                font: None,
            }),
            ItemUnique {
                position: Position { x: 3, y: 4 },
                ..Default::default()
            },
        ),
    ])
    .build();

items (primitive, unique):
  - Shape: Circle
      pos: (1, 2)
      tags: {}

  - TextBox: "hi"
      font: (comic_sans)
      pos: (3, 4)
      tags: {}

Signatures

The signature store is append-only: you can only add to the end, never reorder or delete. Each signature hashes over everything before it — so when you verify signature N, you have a guarantee that all prior content existed at the time it was signed.

graph LR
  subgraph chain["append only →"]
    I0["Visual Signature 1"] --> I1["Visual Signature 2"] --> SA(["Sig A"]) --> I2["Visual Signature 3"] --> I3["Visual Signature 4"] --> SB(["Sig B"])
  end
  I0 & I1 -.->|"hash 0"| SA
  SA & I2 & I3 -.->|"hash 1"| SB

Extra extras

Tags

• Flowing text and selecting text across pages
• Accessibility features like alt text
• Mention being able to easily manipulate a TDF whereas PDFs are hard to manipulate and you have to resave the entire thing
• Chunked compression via the backend