docs-site/src/pages/en/specs/protocol-app.md

---
title: App Protocol
description: Specification of the App Protocol
layout: ../../../layouts/MainLayout.astro
---

**All our protocols and formats use the binary codec called [BARE](https://baremessages.org/)**.

The App Protocol lets the Application talk with the Verifier.

This protocol exchanges content that isn't encrypted.

In the native apps, this protocol runs between the front-end part of the App and the back-end part that is running inside Tauri from Rust compiled into the app binary. Both sides of the protocol are in the same process, but there is an interface between them provided by Tauri (which is made of JSON objects), but this is transparent to the App developer.

In the web-app, the same happens, but the "back-end" part runs inside the same JS context, and is some WASM code compiled from Rust. The passing of messages doesn't involve JSON, as JS POJOs are directly exchanged at the interface.

If the App opens a session with a remote Verifier, the protocol is ran inside a WebSocket, with Noise encryption.

This protocol is available directly from the JS and Rust APIs and you do not need to implement it again or send messages manually. New bindings are always welcomed, for any contributor who wants to add them.

All the protocol messages are embedded inside an `AppRequest` message, and are replied by the Verifier with an `AppResponse`.

Some Requests will respond with a stream of AppResponses, while others will respond with only one.

Streamed responses :

- Fetch(Subscribe)
- FileGet

A new session needs to be opened with the LocalBroker methods, not documented here (it will be documented in the APIs reference). Once the session is opened, the session_id needs to be passed in every request.

For a reference of the common types, please refer to the [Repo format documentation](/en/specs/format-repo/#common-types)

### AppRequest

#### Request

```rust
enum AppRequest {
    V0(AppRequestV0),
}

struct AppRequestV0 {
    command: AppRequestCommandV0,

    nuri: NuriV0,

    payload: Option<AppRequestPayload>,

    session_id: u64,
}

enum AppRequestCommandV0 {
    Fetch(AppFetchContentV0),
    Pin,
    UnPin,
    Delete,
    Create,
    FileGet,
    FilePut,
}

enum AppFetchContentV0 {
    Get, // without subscribing
    Subscribe,
    Update,
    ReadQuery,
    WriteQuery,
    RdfDump,
    History,
    SignatureStatus,
    SignatureRequest,
    SignedSnapshotRequest,
}

enum AppRequestPayload {
    V0(AppRequestPayloadV0),
}

enum AppRequestPayloadV0 {
    Create(DocCreate),
    Query(DocQuery),
    Update(DocUpdate),
    AddFile(DocAddFile),
    Delete(DocDelete),
    SmallFilePut(SmallFile),
    /// content_type (IANA media type)
    RandomAccessFilePut(String),
    /// an empty Vec ends the upload
    RandomAccessFilePutChunk((u32, Vec<u8>)),
}

struct NuriV0 {

    identity: Option<UserId>,
    target: NuriTargetV0,
    entire_store: bool,

    objects: Vec<ObjectRef>,
    signature: Option<ObjectRef>,

    branch: Option<TargetBranchV0>,
    overlay: Option<OverlayLink>,

    access: Vec<NgAccessV0>,
    topic: Option<TopicId>,
    locator: Option<Locator>,
}

enum NuriTargetV0 {
    /// targets the whole DataSet of the user
    UserSite,

    PublicStore,
    ProtectedStore,
    PrivateStore,
    AllDialogs,
    /// shortname of a Dialog
    Dialog(String),
    AllGroups,
    /// shortname of a Group
    Group(String),

    Repo(RepoId),

    None,
}

enum TargetBranchV0 {
    Chat,
    Stream,
    Comments,
    BackLinks,
    Context,
    BranchId(BranchId),
    /// named branch or commit
    Named(String),
    Commits(Vec<ObjectId>),
}

enum OverlayLink {
    Outer(Digest),
    InnerLink(InnerOverlayLink),
    Inner(Digest),
    Inherit,
    Public(PubKey),
    Global,
}

struct InnerOverlayLink {
    /// overlay public key ID
    id: StoreOverlay,
    store_overlay_readcap: ReadCap,
}

enum NgAccessV0 {
    ReadCap(ReadCap),
    Token(Digest),
    ExtRequest(Vec<u8>),
    Key(BlockKey),
    Inbox(PubKey),
}

enum Locator {
    V0(LocatorV0),
}

type LocatorV0 = Vec<BrokerServer>;

struct BrokerServer {
    content: BrokerServerContentV0,

    /// peerId of the server
    peer_id: PubKey,

    /// optional signature over content by peer_id
    sig: Option<Sig>,
}

struct BrokerServerContentV0 {
    servers: Vec<BrokerServerTypeV0>,
    version: u32,
}

enum BrokerServerTypeV0 {
    /// optional port number, defaults to 1440
    Localhost(u16),
    BoxPrivate(Vec<BindAddress>),
    Public(Vec<BindAddress>),
    BoxPublicDyn(Vec<BindAddress>),
    Domain(String),
}
```

- NuriV0.identity : None for personal identity

- NuriV0.entire_store : If it is a store, will include all the docs belonging to the store. not used otherwise

- NuriV0.objects : used only for FileGet.

- NuriV0.branch : if None, the main branch is chosen

- InnerOverlayLink.store_overlay_readcap : The store has a special branch called `Overlay` that is used to manage access to the InnerOverlay. Only the ReadCapSecret is needed to access the InnerOverlay. The full readcap of this branch is needed in order to subscribe to the topic and decrypt the events, and hence be able to subscribe to refreshes of the InnerOverlay. The branchId can be found in the branch Definition. It can be useful to subscribe to this topic if the user is a member of the store's repo, so it will be notified of BranchCapRefresh on the overlay. To the contrary, if the user is an external user to the store, they won't be able to subscribe and they will loose access to the InnerOverlay after a BranchCapRefresh of the overlay branch of the store.

- BrokerServerTypeV0::Domain : accepts an optional trailing ":`port`" number

#### Response

```rust
enum AppResponse {
    V0(AppResponseV0),
}

enum AppResponseV0 {
    SessionStart(AppSessionStartResponse),
    TabInfo(AppTabInfo),
    State(AppState),
    Patch(AppPatch),
    History(AppHistory),
    SignatureStatus(Vec<(String, Option<String>, bool)>),
    Text(String),
    FileUploading(u32),
    FileUploaded(ObjectRef),
    FileBinary(Vec<u8>),
    FileMeta(FileMetaV0),
    QueryResult(Vec<u8>),
    Graph(Vec<u8>),
    Ok,
    True,
    False,
    Error(String),
    EndOfStream,
    Nuri(String),
}

enum AppSessionStartResponse {
    V0(AppSessionStartResponseV0),
}

struct AppSessionStartResponseV0 {
    private_store: RepoId,
    protected_store: RepoId,
    public_store: RepoId,
}

```

### DocCreate

Creates a new Document.

#### Request

```rust
// example AppRequestV0
AppRequestV0 {
    ...,
    command : AppRequestCommandV0::Create,
    nuri: NuriV0::new_empty(),
    payload: Some(AppRequestPayload::V0(AppRequestPayloadV0::Create(
        DocCreate {
            ...
        },
    ))),
}

struct DocCreate {
    store: StoreRepo,
    class: BranchCrdt,
    destination: DocCreateDestination,
}

enum DocCreateDestination {
    Store,
    Stream,
    MagicCarpet,
}
```

#### Response

replies with an `AppResponseV0::Nuri(string)` containing the string representation of the Document's Nuri, of the form `did:ng:o:[repo_id]:v:[overlay_id]`.

### DocUpdate

Updates the **graph** or **discrete** nature of the Document, or both.

Replied with `AppResponseV0::Ok`.

If set, the `NuriV0.branch` should be of the variant `BranchId(_)`. If None, the main branch will be used.

#### Request

```rust
// example AppRequestV0
AppRequestV0 {
    ...,
    command : AppRequestCommandV0::Fetch(AppFetchContentV0::Update),
    nuri: NuriV0 {
        target: NuriTargetV0::Repo(repo_id),
        overlay: Some(overlay_id),
        ... // all the rest empty
    },
    payload: Some(AppRequestPayload::V0(AppRequestPayloadV0::Update(
        DocUpdate {
            ...
        },
    ))),
}

struct DocUpdate {
    heads: Vec<ObjectId>,
    graph: Option<GraphUpdate>,
    discrete: Option<DiscreteUpdate>,
}

struct GraphUpdate {
    // serialization of Vec<Triple>
    inserts: Vec<u8>,
    // serialization of Vec<Triple>
    removes: Vec<u8>,
}

enum DiscreteUpdate {
    /// A yrs::Update
    YMap(Vec<u8>),
    YArray(Vec<u8>),
    YXml(Vec<u8>),
    YText(Vec<u8>),
    /// An automerge::Change.raw_bytes()
    Automerge(Vec<u8>),
}
```

- `GraphUpdate` is not implemented for now.

### WriteQuery

A SPARQL Update query. Can span multiple Documents (by indicating a `GRAPH <Nuri>` or `WITH <Nuri>`, and optional `USING <Nuri>` with a Nuri of the form `did:ng:o:v`).

The `AppRequestV0.nuri` is mandatory and will represent the default graph.

Replied with a `AppResponseV0::Ok`.

The `NuriV0.target` cannot be a `UserSite`, `AllDialogs` nor `AllGroups`.

The `NuriV0.branch` cannot be a `Commits(_)`.

```rust
// example AppRequestV0
AppRequestV0 {
    ...,
    command : AppRequestCommandV0::Fetch(AppFetchContentV0::WriteQuery),
    nuri: NuriV0 {
        target: NuriTargetV0::Repo(repo_id),
        overlay: Some(overlay_id),
        ... // all the rest empty
    },
    payload: Some(AppRequestPayload::V0(AppRequestPayloadV0::Query(
        DocQuery::V0 {
            ...
        },
    ))),
}

enum DocQuery {
    V0 {
        sparql: String,
        base: Option<String>,
    },
}
```

- DocQuery::V0.base : an optional base to resolve all your relative URIs of resources in the SPARQL Update query.

- DocQuery::V0.sparql : the text of your SPARQL Update

### FilePut

Uploads a binary file into the repository (Document). The API is composed of several calls that should be made sequentially.

The first call must be an `AppRequestPayloadV0::RandomAccessFilePut` that will return an upload_id.

Immediately followed by one or more `AppRequestPayloadV0::RandomAccessFilePutChunk`.

And eventually finished with one call to `AppRequestPayloadV0::AddFile`.

#### first Request : RandomAccessFilePut

```rust
// example AppRequestV0
AppRequestV0 {
    ...,
    command : AppRequestCommandV0::FilePut,
    nuri: NuriV0 {
        target: NuriTargetV0::Repo(repo_id),
        overlay: Some(overlay_id),
        ... // all the rest empty
    },
    payload: Some(AppRequestPayload::V0(AppRequestPayloadV0::RandomAccessFilePut(
        content_type // a string representing an IANA media type
    ))),
}
```

#### first Response

```rust
AppResponseV0::FileUploading(upload_id) // a u32
```

#### one or more chunk put Request(s) : RandomAccessFilePutChunk

upload your file in chunks of maximum 1 048 564 bytes, for best efficiency.

Repeat this call until you finished upload all your file.

Add another extra call at the end, with a size of zero. This will indicate that you are done with uploading chunks.

Replied with a `AppResponseV0::Ok` for each chunk that is non-empty.

The last call with an empty chunk is replied with an `AppResponseV0::FileUploaded(reference)` containing an `ObjectRef` to the uploaded file, that you should use in the next call `AppRequestPayloadV0::AddFile`.

```rust
// example AppRequestV0
AppRequestV0 {
    ...,
    command : AppRequestCommandV0::FilePut,
    nuri: NuriV0::new_empty(),
    payload: Some(AppRequestPayload::V0(
        AppRequestPayloadV0::RandomAccessFilePutChunk(
            (upload_id, chunk) // chunk is a Vec<u8>
        )
    )),
}
```

- nuri : can be omitted in those calls, unlike the 2 other type of calls

#### last Request : AddFile

Finally you make an `AppRequestPayloadV0::AddFile` call that will attach the Object to your document's branch.

```rust
// example AppRequestV0
AppRequestV0 {
    ...,
    command : AppRequestCommandV0::FilePut,
    nuri: NuriV0 {
        target: NuriTargetV0::Repo(repo_id),
        overlay: Some(overlay_id),
        branch: // can be omitted (defaults to main branch)
        ... // all the rest empty
    },
    payload: Some(AppRequestPayload::V0(
        AppRequestPayloadV0::AddFile(
            DocAddFile {
                ...
            }
        )
    )),
}

struct DocAddFile {
    filename: Option<String>,
    object: ObjectRef,
}
```

- DocAddFile.object : must be the reference you obtained from the last call to `RandomAccessFilePutChunk`.

- DocAddFile.filename : an optional filename. usually it is the original filename on the filesystem when selecting the binary file to upload.

#### final Response

Finally, you get an `AppResponseV0::Ok` if everything went well.

### FileGet

Downloads a binary file

The request nuri needs to have both a `target` and one, and only one `objects`.

Replied with a **stream** of `AppResponseV0`.

#### Request

```rust
// example AppRequestV0
AppRequestV0 {
    ...,
    command : AppRequestCommandV0::FileGet,
    nuri: NuriV0 {
        target: NuriTargetV0::Repo(repo_id),
        overlay: Some(overlay_id),
        branch: // can be omitted (defaults to main branch)
        objects: [object_ref], // the requested object/file reference
        ... // all the rest empty
    },
    payload: None,
}
```

#### first Response : FileMeta

The first response in the stream is always an `AppResponseV0::FileMeta(FileMetaV0)`.

```rust
struct FileMetaV0 {
    content_type: String,
    size: u64,
}
```

#### more Responses : FileBinary

Then a series of `AppResponseV0::FileBinary(chunk)` containing all the chunks (`Vec<u8>`) of the file, are send in the streamed response.

#### final response : EndOfStream

And eventually, once all the data has been transferred, an `AppResponseV0::EndOfStream` is sent in the stream.

### Subscribe

Subscribes to all the events/commits of a branch.

The response is streamed and long-lived, until cancelled.

The first response contains some meta-data about the branch called `TabInfo`. It is used mostly by the App to display the document header and instantiate the document's viewer.

Then a second response contains the **materialized state** of the document as it is known by the local Verifier at the moment of the subscribe request.

Later on, new updates are added to the stream as they arrive on the local replica (including from the changes made locally by the user).

#### Request

```rust
// example AppRequestV0
AppRequestV0 {
    ...,
    command : AppRequestCommandV0::Fetch(AppFetchContentV0::Subscribe),
    nuri: NuriV0 {
        target: NuriTargetV0::Repo(repo_id),
        overlay: Some(overlay_id),
        branch: // can be omitted (defaults to main branch)
        ... // all the rest empty
    },
    payload: None,
}
```

#### first Response : AppTabInfo

The first response is a `AppResponseV0::TabInfo(AppTabInfo)`.

```rust
struct AppTabInfo {
    branch: Option<AppTabBranchInfo>,
    doc: Option<AppTabDocInfo>,
    store: Option<AppTabStoreInfo>,
}

struct AppTabBranchInfo {
    id: Option<String>,
    readcap: Option<String>,
    comment_branch: Option<String>, // not implemented
    class: Option<String>,
}

struct AppTabDocInfo {
    nuri: Option<String>,
    is_store: Option<bool>,
    is_member: Option<String>,
    title: Option<String>,
    icon: Option<String>,
    description: Option<String>,
    authors: Option<Vec<String>>, // not implemented
    inbox: Option<String>, // not implemented
    can_edit: Option<bool>,
}

struct AppTabStoreInfo {
    repo: Option<StoreRepo>,
    overlay: Option<String>,
    has_outer: Option<String>, // not implemented
    store_type: Option<String>,
    readcap: Option<String>,
    is_member: Option<String>,
    inner: Option<String>,
    title: Option<String>,
    icon: Option<String>,
    description: Option<String>,
}
```

#### second Response : AppState

The second response is a `AppResponseV0::State(AppState)`.

```rust
struct AppState {
    pub heads: Vec<ObjectId>,
    pub head_keys: Vec<ObjectKey>,
    pub graph: Option<GraphState>,
    pub discrete: Option<DiscreteState>,
    pub files: Vec<FileName>,
}

struct GraphState {
    // serialization of Vec<Triple>
    pub triples: Vec<u8>,
}

enum DiscreteState {
    /// A yrs::Update
    YMap(Vec<u8>),
    YArray(Vec<u8>),
    YXml(Vec<u8>),
    YText(Vec<u8>),
    // the output of Automerge::save()
    Automerge(Vec<u8>),
}

struct FileName {
    pub name: Option<String>,
    pub reference: ObjectRef,
    pub nuri: String,
}
```

#### followed by AppPatch

Then on every update done on the branch, a new `AppResponseV0::Patch(AppPatch)` is sent in the stream.

```rust
struct AppPatch {
    pub commit_id: String,
    pub commit_info: CommitInfoJs,
    // or graph, or discrete, or both, or other.
    pub graph: Option<GraphPatch>,
    pub discrete: Option<DiscretePatch>,
    pub other: Option<OtherPatch>,
}

struct CommitInfoJs {
    pub past: Vec<String>,
    pub key: String,
    pub signature: Option<String>,
    pub author: String,
    pub timestamp: String,
    pub final_consistency: bool,
    pub commit_type: CommitType,
    pub branch: Option<String>,
    pub x: u32,
    pub y: u32,
}

enum CommitType {
    TransactionGraph,
    TransactionDiscrete,
    TransactionBoth,
    FileAdd,
    FileRemove,
    Snapshot,
    Compact,
    AsyncSignature,
    SyncSignature,
    Branch,
    UpdateBranch,
    BranchCapRefresh,
    CapRefreshed,
    Other,
}

struct GraphPatch {
    // serialization of Vec<Triple>
    pub inserts: Vec<u8>,
    // serialization of Vec<Triple>
    pub removes: Vec<u8>,
}

enum DiscretePatch {
    /// A yrs::Update
    YMap(Vec<u8>),
    YArray(Vec<u8>),
    YXml(Vec<u8>),
    YText(Vec<u8>),
    /// An automerge::Change.raw_bytes() or a concatenation of several of them.
    Automerge(Vec<u8>),
}

enum OtherPatch {
    FileAdd(FileName),
    FileRemove(ObjectId),
    AsyncSignature((String, Vec<String>)),
    Snapshot(ObjectRef),
    Compact(ObjectRef),
    Other,
}
```

### ReadQuery

A SPARQL query (read-only).

#### Request

```rust
// example AppRequestV0
AppRequestV0 {
    ...,
    command : AppRequestCommandV0::Fetch(AppFetchContentV0::ReadQuery),
    nuri: NuriV0 {
        target: NuriTargetV0::Repo(repo_id),
        overlay: Some(overlay_id),
        branch: // not implemented yet
        ... // all the rest empty
    },
    payload: Some(AppRequestPayload::V0(
        AppRequestPayloadV0::Query(
            DocQuery::V0 {
                ...
            }
        )
    )),
}

enum DocQuery {
    V0 {
        sparql: String,
        base: Option<String>,
    },
}
```

- AppRequestV0.nuri.target : represents the default graph. can be NuriV0::UserSite or NuriV0::None and in those cases, the **union graph** of all the graphs is used as default graph.

- DocQuery::V0.base : an optional base to resolve all your relative URIs of resources in the SPARQL Query.

- DocQuery::V0.sparql : the text of your SPARQL Query

#### Response

Depending on the type of query, the response differs.

- for SELECT queries: an `AppResponseV0::QueryResult(buffer)` where buffer is a `Vec<u8>` containing a UTF-8 serialization of a JSON string representing a JSON Sparql Query Result. see [SPARQL Query Results JSON Format](https://www.w3.org/TR/sparql11-results-json/).

- for CONSTRUCT an `AppResponseV0::Graph(buffer)` where buffer is a `Vec<u8>` containing a BARE serialization of a `Vec<Triple>`.

- for ASK queries: `AppResponseV0::True` or `AppResponseV0::False`

### RdfDump

Gets the full dump of all the quads contained in the UserSite (whole local DataSet of the User).

#### Request

```rust
// example AppRequestV0
AppRequestV0 {
    ...,
    command : AppRequestCommandV0::Fetch(AppFetchContentV0::RdfDump),
    nuri: NuriV0::new_empty()
    payload: None,
}
```

#### Response

```rust
AppResponseV0::Text(string)
```

- `string` being a Turtle output of all the quads.

### History

Fetches the current (HEADs) history of commits for this branch.

#### Request

```rust
// example AppRequestV0
AppRequestV0 {
    ...,
    command : AppRequestCommandV0::Fetch(AppFetchContentV0::History),
    nuri: NuriV0 {
        target: NuriTargetV0::Repo(repo_id),
        overlay: Some(overlay_id),
        branch: //not implemented. defaults to main branch
        ... // all the rest empty
    },
    payload: None
}
```

#### Response

The response is a `AppResponseV0::History(AppHistory)`

```rust
struct AppHistory {
    pub history: Vec<(ObjectId, CommitInfo)>,
    pub swimlane_state: Vec<Option<ObjectId>>,
}

struct CommitInfo {
    pub past: Vec<ObjectId>,
    pub key: ObjectKey,
    pub signature: Option<ObjectRef>,
    pub author: String,
    pub timestamp: Timestamp,
    pub final_consistency: bool,
    pub commit_type: CommitType,
    pub branch: Option<ObjectId>,
    pub x: u32,
    pub y: u32,
}
```

- AppHistory.history : in order, with the newest commits first. The first part of the tuple (ObjectId) is the CommitID.

- AppHistory.swimlane_state : can be discarded. it is only used by our GUI representation of the history in the Apps. Same with the x and y values in CommitInfo.

### SignatureStatus

Fetches the Signature status of the branch, at the HEADs.

#### Request

```rust
// example AppRequestV0
AppRequestV0 {
    ...,
    command : AppRequestCommandV0::Fetch(AppFetchContentV0::SignatureStatus),
    nuri: NuriV0 {
        target: NuriTargetV0::Repo(repo_id),
        overlay: Some(overlay_id),
        branch: //not implemented. defaults to main branch
        ... // all the rest empty
    },
    payload: None
}
```

#### Response

The response is a `AppResponseV0::SignatureStatus(Vec<(String, Option<String>, bool)>)`

Which is a list of commits at the HEAD. Each commit is represented by a tuple that contains :

- the commit ID printed as a string (44 characters)
- an optional string that is present only if the commit is a signature. in this case, the string represents a list of the signed commits `c:[commit_id]:k:[commit_key]` joined with `:` (at least one commit is present), followed by partial Nuri for the signature object `:s:[signature_object_id]:k:[signature_object_key]`.
- a boolean that indicates if the commit is a snapshot. (in this case, only one commit is present in the HEADs)

### SignatureRequest

Requests that the HEADs commit(s) be asynchronously signed.

#### Request

```rust
// example AppRequestV0
AppRequestV0 {
    ...,
    command : AppRequestCommandV0::Fetch(AppFetchContentV0::SignatureRequest),
    nuri: NuriV0 {
        target: NuriTargetV0::Repo(repo_id),
        overlay: Some(overlay_id),
        branch: //not implemented. defaults to main branch
        ... // all the rest empty
    },
    payload: None
}
```

#### Response

If the signature is immediately available, the response is `AppResponseV0::True`. The new HEAD containing the signature can be immediately fetched with a call to `SignatureStatus`.

If the signature is not immediately available (because the quorum needs to sign it and it can take some time), the response is `AppResponseV0::False`. A notification will be sent to the user once the signature is ready (not implemented yet).

### SignedSnapshotRequest

Requests that a snapshot be taken from the current materialized state at the HEADs, and that this new commit containing the snapshot, be signed asynchronously.

#### Request

```rust
// example AppRequestV0
AppRequestV0 {
    ...,
    command : AppRequestCommandV0::Fetch(AppFetchContentV0::SignedSnapshotRequest),
    nuri: NuriV0 {
        target: NuriTargetV0::Repo(repo_id),
        overlay: Some(overlay_id),
        branch: //not implemented. defaults to main branch
        ... // all the rest empty
    },
    payload: None
}
```

#### Response

If the signature is immediately available, the response is `AppResponseV0::True`. The newly created snapshot and its signature can be immediately fetched with a call to `SignatureStatus`.

If the signature is not immediately available (because the quorum needs to sign it and it can take some time), the response is `AppResponseV0::False`. A notification will be sent to the user once the signature is ready (not implemented yet). The snapshot is already available as the last commit in HEAD, but it isn't signed yet.