--- title: Client Protocol description: Specification of the Client Protocol layout: ../../../layouts/MainLayout.astro --- **All our protocols and formats use the binary codec called [BARE](https://baremessages.org/)**. The Client Protocol is used by the Verifier in order to contact the Broker of the User. It maintains this connection throughout the session that was opened by the User (by opening the wallet in the app, by example). From this connection, the Verifier gets all the pushed updates (called Events), after it subscribed to some topics. The verifier also sends the updates that it wants to publish, in the form of an Event to the Pub/Sub Topic, and the Broker deals with forwarding this Event to all the other devices and users that have subscribed to this topic. The communication on the Client Protocol is using a WebSocket, encrypted from within with the Noise Protocol. In addition, all the Events, that are sent and received with this protocol, are also encrypted end-to-end. For now, the Verifier only connects to one Broker, but for redundancy and failsafe purposes, it will be possible in the future that it tries to connect to several Brokers. But one rule should always be followed: for any given Overlay, a User can only participate in this Overlay from one and only one Broker at the same time. Let's dive into the format of the messages and actions/commands that can be exchanged on the Client Protocol. The initiation of the connection is common to all protocols, and involves some Noise handshake. It isn't detailed here, please refer to the code for now. We will provide more documentation on that part later on. For a reference of the common types, please refer to the [Repo format documentation](/en/specs/format-repo/#common-types) ### ClientMessage All the subsequent messages sent and receive on this protocol, are encapsulated inside a `ClientMessage`. The `ClientRequestV0.id` is set by the requester in an incremental way. Request IDs must be unique by session. They should start from 1 after every start of a new session. This ID is present in the response, in order to match requests and responses. ```rust enum ClientMessage { V0(ClientMessageV0), } struct ClientMessageV0 { overlay: OverlayId, content: ClientMessageContentV0, /// Optional padding padding: Vec, } enum ClientMessageContentV0 { ClientRequest(ClientRequest), ClientResponse(ClientResponse), ForwardedEvent(Event), ForwardedBlock(Block), } enum ClientRequest { V0(ClientRequestV0), } struct ClientRequestV0 { /// Request ID id: i64, /// Request content content: ClientRequestContentV0, } enum ClientRequestContentV0 { OpenRepo(OpenRepo), PinRepo(PinRepo), UnpinRepo(UnpinRepo), RepoPinStatusReq(RepoPinStatusReq), // once repo is opened or pinned: TopicSub(TopicSub), TopicUnsub(TopicUnsub), BlocksExist(BlocksExist), BlocksGet(BlocksGet), CommitGet(CommitGet), TopicSyncReq(TopicSyncReq), // For Pinned Repos only : ObjectPin(ObjectPin), ObjectUnpin(ObjectUnpin), ObjectDel(ObjectDel), // For InnerOverlay's only : BlocksPut(BlocksPut), PublishEvent(PublishEvent), WalletPutExport(WalletPutExport), } enum ClientResponse { V0(ClientResponseV0), } struct ClientResponseV0 { /// Request ID id: i64, /// Result (including but not limited to ServerError) result: u16, /// Response content content: ClientResponseContentV0, } enum ClientResponseContentV0 { EmptyResponse, Block(Block), RepoOpened(RepoOpened), TopicSubRes(TopicSubRes), TopicSyncRes(TopicSyncRes), BlocksFound(BlocksFound), RepoPinStatus(RepoPinStatus), } ``` - ClientResponseV0.result : - 0 means success - 1 means PartialContent (the response is a stream. Each element in the stream will have this result code) - 2 means EndOfStream (that's the last response in the stream) - 3 means False - 4 and above are errors. For the list, see `ng-repo/src/errors.rs` starting at line 265. When an error occurs (result >= 3), the content is of the type ClientResponseContentV0::EmptyResponse - ClientMessageContentV0::ForwardedEvent(Event) is used by the Broker when it wants to **push** the events it received from other brokers or devices, down to the Client. ### BlocksGet Request a Block by ID. Can be used to retrieve the content of a ReadCap, or any Commit that didn't arrive from the Pub/Sub already. Images and Binary files also use BlocksGet when opened, read and downloaded. ReadCaps are preferably retrieved with CommitGet though. `commit_header_key` is always set to None in the reply when request is made on OuterOverlay of Protected or Group overlays. #### Request ```rust enum BlocksGet { V0(BlocksGetV0), } struct BlocksGetV0 { /// Block IDs to request ids: Vec, /// Whether or not to include all children recursively include_children: bool, topic: Option, } ``` - topic : (optional) Topic the object is referenced from, if it is known by the requester. Can be used by the Broker to do a BlockSearchTopic in the core overlay, in case the block is not present locally in the Broker. #### Response The response is a stream of `Block`s. ### CommitGet Request a Commit by ID Replied with a stream of `Block`s. `commit_header_key` of the replied blocks is always set to None when request is made on OuterOverlay of Protected or Group overlays. The difference with BlocksGet is that the Broker will try to return all the commit blocks as they were sent in the Pub/Sub Event, if it has the event. This will help in receiving all the blocks (including the header and body blocks) in one ClientProtocol message, while a BlocksGet would inevitably return only the blocks of the ObjectContent, and not the header nor the body. And the load() would fail with CommitLoadError::MissingBlocks. That's what happens when the Commit is not present in the pubsub, and then we need to default to using BlocksGet instead. #### Request ```rust enum CommitGet { V0(CommitGetV0), } struct CommitGetV0 { /// Commit ID to request id: ObjectId, /// Topic the commit is referenced from, if it is known by the requester. /// can be used to do a BlockSearchTopic in the core overlay. topic: Option, } ``` ### RepoPinStatus Request the pinning status for a repo on the broker (for the current user's session). Returns an error code if not pinned, otherwise returns a RepoPinStatusV0. The overlay entered in ClientMessage is important. Id it is the outer, only outer pinning will be checked. If it is the inner overlay, only the inner pinning will be checked. #### Request ```rust struct RepoPinStatusReqV0 { /// Repo Hash hash: RepoHash, } ``` #### Response ```rust enum RepoPinStatus { V0(RepoPinStatusV0), } struct RepoPinStatusV0 { /// Repo Hash hash: RepoHash, /// only possible for Inner overlays expose_outer: bool, /// list of topics that are subscribed to topics: Vec, } enum TopicSubRes { V0(TopicSubResV0), } struct TopicSubResV0 { /// Topic subscribed topic: TopicId, /// current HEADS at the broker known_heads: Vec, /// was the topics subscribed as publisher publisher: bool, commits_nbr: u64, } ``` - commits_nbr : used for a subsequent TopicSyncReq in order to properly size the bloomfilter ### PinRepo Request to pin a repo on the broker. When client will disconnect, the subscriptions and publisherAdvert of the topics will remain active on the broker. Replied with a RepoOpened #### Request ```rust enum PinRepo { V0(PinRepoV0), } struct PinRepoV0 { /// Repo Hash hash: RepoHash, overlay: OverlayAccess, overlay_root_topic: Option, expose_outer: bool, peers: Vec, max_peer_count: u16, ro_topics: Vec, rw_topics: Vec, } enum PublisherAdvert { V0(PublisherAdvertV0), } struct PublisherAdvertV0 { content: PublisherAdvertContentV0, /// Signature over content by topic key sig: Sig, } struct PublisherAdvertContentV0 { /// Topic public key topic: TopicId, /// Peer public key peer: DirectPeerId, } enum OverlayAccess { ReadOnly(OverlayId), ReadWrite((OverlayId, OverlayId)), WriteOnly(OverlayId), } ``` - OverlayAccess::ReadOnly : The repo will be accessed on the Outer Overlay in Read Only mode. This can be used for Public, Protected or Group overlays. Value should be an OverlayId::Outer - OverlayAccess::ReadWrite : The repo will be accessed on the Inner Overlay in Write mode, and the associated Outer overlay is also given. This is used for Public, Protected and Group overlays. First value in tuple should be the OverlayId::Inner, second the OverlayId::Outer. The overlay that should be used in the ClientMessageV0 is the InnerOverlay - OverlayAccess::WriteOnly : The repo will be accessed on the Inner Overlay in Write mode, and it doesn't have an Outer overlay. This is used for Private and Dialog overlays. Value should be an OverlayId::Inner - PinRepoV0.overlay_root_topic : Root topic of the overlay, used to listen to overlay refreshes. Only set for inner overlays (RW or WO) overlays. Not implemented yet - PinRepoV0.expose_outer : only possible for RW overlays. Not allowed for private or dialog overlay. Not implemented yet - PinRepoV0.peers : Broker peers to connect to in order to join the overlay. If the repo has previously been opened (during the same session) or if it is a private overlay, then peers info can be omitted. If there are no known peers in the overlay yet, vector is left empty (creation of a store, or repo in a store that is owned by user). - PinRepoV0.max_peer_count : Maximum number of peers to connect to for this overlay (only valid for an inner (RW/WO) overlay). Not implemented yet - PinRepoV0.ro_topics : list of topics that should be subscribed to. If the repo has previously been opened (during the same session) and the list of RO topics does not need to be modified, then ro_topics info can be omitted - PinRepoV0.rw_topics : list of topics for which the client will be a publisher. Only possible with inner (RW or WO) overlays. If the repo has previously been opened (during the same session) then rw_topics info can be omitted #### Response ```rust type RepoOpened = Vec; ``` for more details about TopicSubRes see above in [RepoPinStatus](#repopinstatus). ### TopicSub Request subscription to a `Topic` of an already opened or pinned Repo replied with a TopicSubRes containing the current heads that should be used to do a TopicSync #### Request ```rust enum TopicSub { V0(TopicSubV0), } struct TopicSubV0 { /// Topic to subscribe to topic: TopicId, /// Hash of the repo that was previously opened or pinned repo_hash: RepoHash, /// Publisher needs to provide a signed `PublisherAdvert` // for the PeerId of the broker publisher: Option, } ``` #### Response A `TopicSubRes`. See above in [RepoPinStatus](#repopinstatus) for more details. ### TopicSyncReq Topic synchronization request The broker will send all the missing events (or commits if it cannot find events) that are absent from the DAG of the Client. The events/commits are ordered respecting their causal relation one with another. Replied with a stream of `TopicSyncRes` #### Request ```rust enum TopicSyncReq { V0(TopicSyncReqV0), } struct TopicSyncReqV0 { /// Topic public key topic: TopicId, /// Fully synchronized until these commits known_heads: Vec, /// Stop synchronizing when these commits are met. /// if empty, the local HEAD at the responder is used instead target_heads: Vec, /// optional Bloom filter of all the commit IDs present locally known_commits: Option, } ``` #### Response ```rust enum TopicSyncRes { V0(TopicSyncResV0), } enum TopicSyncResV0 { Event(Event), Block(Block), } ``` ### BlocksExist Request to know if some blocks are present locally on the responder used by a Client before publishing an event with FILES, to know what to push, and save bandwidth if the blocks are already present on the Broker (content deduplication). Commits without FILES cannot be deduplicated because they are unique, due to their unique position in the DAG, and the embedded BranchId. Replied with `BlocksFound` #### Request ```rust enum BlocksExist { V0(BlocksExistV0), } struct BlocksExistV0 { /// Ids of Blocks to check blocks: Vec, } ``` #### Response ```rust enum BlocksFound { V0(BlocksFoundV0), } struct BlocksFoundV0 { /// Ids of Blocks that were found locally /// this might be deprecated soon (as it is useless) found: Vec, /// Ids of Blocks that were missing locally missing: Vec, } ``` ### BlocksPut Request to store one or more blocks on the broker Replied with an `EmptyResponse` #### Request ```rust enum BlocksPut { V0(BlocksPutV0), } struct BlocksPutV0 { /// Blocks to store blocks: Vec, } ``` ### PublishEvent Request to publish an event in pub/sub Replied with an `EmptyResponse` #### Request ```rust struct PublishEvent(Event); ```