@ -22,7 +22,7 @@ For the developer of an App, this means that the data is accessed and manipulate
Developers of modern apps today often use some front-end frameworks like **React** and **Svelte**.
The data they manipulate is often stored in a **reactive store** (Runes in Svelte, probably useContext/useState in React) that needs to be configured and plugged to a backend system with some APIs that can range from WebSocket and GraphQL to HTTP/REST APIs to a MYSQL and so on...
The data they manipulate is often stored in a **reactive store** (Runes in Svelte, probably useContext/useState/Redux in React) that needs to be configured and plugged to a backend system with some APIs that can range from WebSocket and GraphQL to HTTP/REST APIs to a MYSQL and so on...
> With NextGraph, we provide the developer with a reactive store (for React, Svelte, and Deno/Node) and that's all they have to worry about. NextGraph transparently synchronizes, encrypts and deals with permissions for you.
**All our protocols and formats use the binary codec called [BARE](https://baremessages.org/)**.
TBD
The App Protocol let's the Application talk with the Verifier.
This protocol exchanges content that isn't encrypted.
In the native apps, this protocol runs between the front-end part of the App and the back-end part that is running inside Tauri from Rust compiled into the app binary. Both sides of the protocol are in the same process, but there is an interface between them provided by Tauri (which is made of JSON objects), but this is transparent to the App developer.
In the web-app, the same happens, but the "back-end" part runs inside the same JS context, and is some WASM code compiled from Rust. The passing of messages doesn't involve JSON, as JS POJOs are directly exchanged at the interface.
If the App opens a session with a remote Verifier, the protocol is ran inside a WebSocket, with Noise encryption.
This protocol is available directly from the JS and Rust APIs and you do not need to implement it again or send messages manually. New bindings are always welcomed, for any contributor who wants to add them.
All the protocol messages are embedded inside an `AppRequest` message, and are replied by the Verifier with an `AppResponse`.
Some Requests will respond will a stream of AppResponses, while others will respond with only one.
Streamed responses :
- Fetch(Subscribe)
- FileGet
A new session needs to be opened with the LocalBroker methods, not documented here (it will be documented in the APIs reference). Once the session is opened, the session_id needs to be passed in every request.
### AppRequest
#### Request
```rust
enum AppRequest {
V0(AppRequestV0),
}
struct AppRequestV0 {
command: AppRequestCommandV0,
nuri: NuriV0,
payload: Option<AppRequestPayload>,
session_id: u64,
}
enum AppRequestCommandV0 {
Fetch(AppFetchContentV0),
Pin,
UnPin,
Delete,
Create,
FileGet,
FilePut,
}
enum AppFetchContentV0 {
Get, // without subscribing
Subscribe,
Update,
ReadQuery,
WriteQuery,
RdfDump,
History,
SignatureStatus,
SignatureRequest,
SignedSnapshotRequest,
}
enum AppRequestPayload {
V0(AppRequestPayloadV0),
}
enum AppRequestPayloadV0 {
Create(DocCreate),
Query(DocQuery),
Update(DocUpdate),
AddFile(DocAddFile),
Delete(DocDelete),
SmallFilePut(SmallFile),
/// content_type (IANA media type)
RandomAccessFilePut(String),
/// an empty Vec ends the upload
RandomAccessFilePutChunk((u32, Vec<u8>)),
}
struct NuriV0 {
identity: Option<UserId>,
target: NuriTargetV0,
entire_store: bool,
objects: Vec<ObjectRef>,
signature: Option<ObjectRef>,
branch: Option<TargetBranchV0>,
overlay: Option<OverlayLink>,
access: Vec<NgAccessV0>,
topic: Option<TopicId>,
locator: Option<Locator>,
}
enum NuriTargetV0 {
/// targets the whole DataSet of the user
UserSite,
PublicStore,
ProtectedStore,
PrivateStore,
AllDialogs,
/// shortname of a Dialog
Dialog(String),
AllGroups,
/// shortname of a Group
Group(String),
Repo(RepoId),
None,
}
enum TargetBranchV0 {
Chat,
Stream,
Comments,
BackLinks,
Context,
BranchId(BranchId),
/// named branch or commit
Named(String),
Commits(Vec<ObjectId>),
}
enum OverlayLink {
Outer(Digest),
InnerLink(InnerOverlayLink),
Inner(Digest),
Inherit,
Public(PubKey),
Global,
}
struct InnerOverlayLink {
/// overlay public key ID
id: StoreOverlay,
store_overlay_readcap: ReadCap,
}
enum NgAccessV0 {
ReadCap(ReadCap),
Token(Digest),
ExtRequest(Vec<u8>),
Key(BlockKey),
Inbox(PubKey),
}
enum Locator {
V0(LocatorV0),
}
type LocatorV0 = Vec<BrokerServer>;
struct BrokerServer {
content: BrokerServerContentV0,
/// peerId of the server
peer_id: PubKey,
/// optional signature over content by peer_id
sig: Option<Sig>,
}
struct BrokerServerContentV0 {
servers: Vec<BrokerServerTypeV0>,
version: u32,
}
enum BrokerServerTypeV0 {
/// optional port number, defaults to 1440
Localhost(u16),
BoxPrivate(Vec<BindAddress>),
Public(Vec<BindAddress>),
BoxPublicDyn(Vec<BindAddress>),
Domain(String),
}
```
- NuriV0.identity : None for personal identity
- NuriV0.entire_store : If it is a store, will include all the docs belonging to the store. not used otherwise
- NuriV0.objects : used only for FileGet.
- NuriV0.branch : if None, the main branch is chosen
- InnerOverlayLink.store_overlay_readcap : The store has a special branch called `Overlay` that is used to manage access to the InnerOverlay. Only the ReadCapSecret is needed to access the InnerOverlay. The full readcap of this branch is needed in order to subscribe to the topic and decrypt the events, and hence be able to subscribe to refreshes of the InnerOverlay. The branchId can be found in the branch Definition. It can be useful to subscribe to this topic if the user is a member of the store's repo, so it will be notified of BranchCapRefresh on the overlay. To the contrary, if the user is an external user to the store, they won't be able to subscribe and they will loose access to the InnerOverlay after a BranchCapRefresh of the overlay branch of the store.
- BrokerServerTypeV0::Domain : accepts an optional trailing ":`port`" number
replies with an `AppResponseV0::Nuri(string)` containing the string representation of the Document's Nuri, of the form `did:ng:o:[repo_id]:v:[overlay_id]`.
### DocUpdate
Updates the **graph** or **discrete** nature of the Document, or both.
Replied with `AppResponseV0::Ok`.
If set, the `NuriV0.branch` should be of the variant `BranchId(_)`. If None, the main branch will be used.
A SPARQL Update query. Can span multiple Documents (by indicating a `GRAPH <Nuri>` or `WITH <Nuri>`, and optional `USING <Nuri>` with a Nuri of the form `did:ng:o:v`).
The `AppRequestV0.nuri` is mandatory and will represent the default graph.
Replied with a `AppResponseV0::Ok`.
The `NuriV0.target` cannot be a `UserSite`, `AllDialogs` nor `AllGroups`.
content_type // a string representing an IANA media type
))),
}
```
#### first Response
```rust
AppResponseV0::FileUploading(upload_id) // a u32
```
#### one or more chunk put Request(s) : RandomAccessFilePutChunk
upload your file in chunks of maximum 1 048 564 bytes, for best efficiency.
Repeat this call until you finished upload all your file.
Add another extra call at the end, with a size of zero. This will indicate that you are done with uploading chunks.
Replied with a `AppResponseV0::Ok` for each chunk that is non-empty.
The last call with an empty chunk is replied with an `AppResponseV0::FileUploaded(reference)` containing an `ObjectRef` to the uploaded file, that you should use in the next call `AppRequestPayloadV0::AddFile`.
```rust
// example AppRequestV0
AppRequestV0 {
...,
command : AppRequestCommandV0::FilePut,
nuri: NuriV0::new_empty(),
payload: Some(AppRequestPayload::V0(
AppRequestPayloadV0::RandomAccessFilePutChunk(
(upload_id, chunk) // chunk is a Vec<u8>
)
)),
}
```
- nuri : can be omitted in those calls, unlike the 2 other type of calls
#### last Request : AddFile
Finally you make an `AppRequestPayloadV0::AddFile` call that will attach the Object to your document's branch.
```rust
// example AppRequestV0
AppRequestV0 {
...,
command : AppRequestCommandV0::FilePut,
nuri: NuriV0 {
target: NuriTargetV0::Repo(repo_id),
overlay: Some(overlay_id),
branch: // can be omitted (defaults to main branch)
... // all the rest empty
},
payload: Some(AppRequestPayload::V0(
AppRequestPayloadV0::AddFile(
DocAddFile {
...
}
)
)),
}
struct DocAddFile {
filename: Option<String>,
object: ObjectRef,
}
```
- DocAddFile.object : must be the reference you obtained from the last call to `RandomAccessFilePutChunk`.
- DocAddFile.filename : an optional filename. usually it is the original filename on the filesystem when selecting the binary file to upload.
#### final Response
Finally, you get an `AppResponseV0::Ok` if everything went well.
### FileGet
Downloads a binary file
The request nuri needs to have both a `target` and one, and only one `objects`.
Replied with a **stream** of `AppResponseV0`.
#### Request
```rust
// example AppRequestV0
AppRequestV0 {
...,
command : AppRequestCommandV0::FileGet,
nuri: NuriV0 {
target: NuriTargetV0::Repo(repo_id),
overlay: Some(overlay_id),
branch: // can be omitted (defaults to main branch)
objects: [object_ref], // the requested object/file reference
... // all the rest empty
},
payload: None,
}
```
#### first Response : FileMeta
The first response in the stream is always an `AppResponseV0::FileMeta(FileMetaV0)`.
```rust
struct FileMetaV0 {
content_type: String,
size: u64,
}
```
#### more Responses : FileBinary
Then a series of `AppResponseV0::FileBinary(chunk)` containing all the chunks (`Vec<u8>`) of the file, are send in the streamed response.
#### final response : EndOfStream
And eventually, once all the data has been transferred, an `AppResponseV0::EndOfStream` is sent in the stream.
### Subscribe
Subscribes to all the events/commits of a branch.
The response is streamed and long-lived, until cancelled.
The first response contains some meta-data about the branch called `TabInfo`. It is used mostly bt the App to display the document header and instantiate the document's viewer.
Then a second response contains the **materialized state** of the document as it is known by the local Verifier.
Later on, new updates are added to the stream as they arrive on the local replica (including from the changes made locally by the user).
- AppRequestV0.nuri.target : represents the default graph. can be NuriV0::UserSite or NuriV0::None and in those cases, the **union graph** of all the graphs is used as default graph.
- DocQuery::V0.base : an optional base to resolve all your relative URIs of resources in the SPARQL Query.
- DocQuery::V0.sparql : the text of your SPARQL Query
#### Response
Depending on the type of query, the response differs.
- for SELECT queries: an `AppResponseV0::QueryResult(buffer)` where buffer is a `Vec<u8>` containing a UTF-8 serialization of a JSON string representing a JSON Sparql Query Result. see [SPARQL Query Results JSON Format](https://www.w3.org/TR/sparql11-results-json/).
- for CONSTRUCT an `AppResponseV0::Graph(buffer)` where buffer is a `Vec<u8>` containing a BARE serialization of a `Vec<Triple>`.
- for ASK queries: `AppResponseV0::True` or `AppResponseV0::False`
### RdfDump
Gets the full dump of all the quads contained in the UserSite (whole local DataSet of the User).
branch: //not implemented. defaults to main branch
... // all the rest empty
},
payload: None
}
```
#### Response
The response is a `AppResponseV0::History(AppHistory)`
```rust
struct AppHistory {
pub history: Vec<(ObjectId, CommitInfo)>,
pub swimlane_state: Vec<Option<ObjectId>>,
}
struct CommitInfo {
pub past: Vec<ObjectId>,
pub key: ObjectKey,
pub signature: Option<ObjectRef>,
pub author: String,
pub timestamp: Timestamp,
pub final_consistency: bool,
pub commit_type: CommitType,
pub branch: Option<ObjectId>,
pub x: u32,
pub y: u32,
}
```
- AppHistory.history : is order with the newest commits first.
- AppHistory.swimlane_state : can be discarded. it is only used by our GUI representation of the history in the Apps. Same with the x and y values in CommitInfo.
### SignatureStatus
Fetches the Signature status of the branch, at the HEADs.
branch: //not implemented. defaults to main branch
... // all the rest empty
},
payload: None
}
```
#### Response
The response is a `AppResponseV0::SignatureStatus(Vec<(String, Option<String>, bool)>)`
Which is a list of commits at the HEAD. Each commit is represented by a tuple that contains :
- the commit ID printed as a string (44 characters)
- an optional string that is present only if the commit is a signature. in this case, the string represents a list of the signed commits `c:[commit_id]:k:[commit_key]` joined with `:` (at least one commit is present), followed by partial Nuri for the signature object `:s:[signature_object_id]:k:[signature_object_key]`.
- a boolean that indicates if the commit is a snapshot. (in this case, only one commit is present in the HEADs)
### SignatureRequest
Requests that the HEADs commit(s) be asynchronously signed.
branch: //not implemented. defaults to main branch
... // all the rest empty
},
payload: None
}
```
#### Response
If the signature is immediately available, the response is `AppResponseV0::True`. The new HEADs containing the signature can be immediately fetched with a call to `SignatureStatus`.
If the signature is not immediately available (because the quorum needs to sign it and it can take some time), the response is `AppResponseV0::False`. A notification will be sent to the user once the signature is ready (not implemented yet).
### SignedSnapshotRequest
Requests that a snapshot be taken from the current materialized state at the HEADs, and that this new commit containing the snapshot, be signed asynchronously.
branch: //not implemented. defaults to main branch
... // all the rest empty
},
payload: None
}
```
#### Response
If the signature is immediately available, the response is `AppResponseV0::True`. The newly created snapshot and its signature can be immediately fetched with a call to `SignatureStatus`.
If the signature is not immediately available (because the quorum needs to sign it and it can take some time), the response is `AppResponseV0::False`. A notification will be sent to the user once the signature is ready (not implemented yet). The snapshot is already available as the last commit in HEAD, but it isn't signed yet.