Adds links to the code documentation

pull/46/head
Tpt 4 years ago
parent 961f465703
commit a0c5cf7286
  1. 7
      README.md
  2. 2
      lib/src/io/format.rs
  3. 8
      lib/src/io/read.rs
  4. 16
      lib/src/lib.rs
  5. 2
      lib/src/model/blank_node.rs
  6. 4
      lib/src/model/vocab.rs
  7. 5
      lib/src/sparql/error.rs
  8. 7
      lib/src/sparql/mod.rs
  9. 2
      lib/src/sparql/model.rs
  10. 44
      lib/src/store/memory.rs
  11. 66
      lib/src/store/rocksdb.rs
  12. 78
      lib/src/store/sled.rs

@ -7,19 +7,22 @@ Oxigraph
Oxigraph is a work in progress graph database implementing the [SPARQL](https://www.w3.org/TR/sparql11-overview/) standard. Oxigraph is a work in progress graph database implementing the [SPARQL](https://www.w3.org/TR/sparql11-overview/) standard.
There is no released version yet. There is no released version yet.
The storage format is not stable yet and may be brocken at any time.
Its goal is to provide a compliant, safe and fast graph database based on the [RocksDB](https://rocksdb.org/) key-value store. Its goal is to provide a compliant, safe and fast graph database based on the [RocksDB](https://rocksdb.org/) and [Sled](https://sled.rs/) key-value stores.
It is written in Rust. It is written in Rust.
It also provides a set of utility functions for reading, writing and processing RDF files.
It is split into multiple parts: It is split into multiple parts:
* The `lib` directory contains the database written as a Rust library. * The `lib` directory contains the database written as a Rust library.
* The `python` directory contains bindings to use Oxigraph in Python. See [its README](https://github.com/oxigraph/oxigraph/blob/master/python/README.md) for the Python bindings documentation. * The `python` directory contains bindings to use Oxigraph in Python. See [its README](https://github.com/oxigraph/oxigraph/blob/master/python/README.md) for the Python bindings documentation.
* The `js` directory contains bindings to use Oxigraph in JavaScript with the help of WebAssembly. See [its README](https://github.com/oxigraph/oxigraph/blob/master/js/README.md) for the JS bindings documentation. * The `js` directory contains bindings to use Oxigraph in JavaScript with the help of WebAssembly. See [its README](https://github.com/oxigraph/oxigraph/blob/master/js/README.md) for the JS bindings documentation.
* The `server` directory contains a stand-alone binary of a web server implementing the [SPARQL 1.1 Protocol](https://www.w3.org/TR/sparql11-protocol/). * The `server` directory contains a stand-alone binary of a web server implementing the [SPARQL 1.1 Protocol](https://www.w3.org/TR/sparql11-protocol/). It uses the [RocksDB](https://rocksdb.org/) key-value store.
* The `wikibase` directory contains a stand-alone binary of a web server able to synchronize with a [Wikibase instance](https://wikiba.se/). * The `wikibase` directory contains a stand-alone binary of a web server able to synchronize with a [Wikibase instance](https://wikiba.se/).
Are currently implemented: Are currently implemented:
* [SPARQL 1.1 Query](https://www.w3.org/TR/sparql11-query/) except `FROM` and `FROM NAMED`. * [SPARQL 1.1 Query](https://www.w3.org/TR/sparql11-query/) except `FROM` and `FROM NAMED`.
* [SPARQL 1.1 Federated Query](https://www.w3.org/TR/sparql11-federated-query/).
* [Turtle](https://www.w3.org/TR/turtle/), [TriG](https://www.w3.org/TR/trig/), [N-Triples](https://www.w3.org/TR/n-triples/), [N-Quads](https://www.w3.org/TR/n-quads/) and [RDF XML](https://www.w3.org/TR/rdf-syntax-grammar/) RDF serialization formats for both data ingestion and retrieval using the [Rio library](https://github.com/oxigraph/rio). * [Turtle](https://www.w3.org/TR/turtle/), [TriG](https://www.w3.org/TR/trig/), [N-Triples](https://www.w3.org/TR/n-triples/), [N-Quads](https://www.w3.org/TR/n-quads/) and [RDF XML](https://www.w3.org/TR/rdf-syntax-grammar/) RDF serialization formats for both data ingestion and retrieval using the [Rio library](https://github.com/oxigraph/rio).
* [SPARQL Query Results XML Format](http://www.w3.org/TR/rdf-sparql-XMLres/) and [SPARQL Query Results JSON Format](https://www.w3.org/TR/sparql11-results-json/). * [SPARQL Query Results XML Format](http://www.w3.org/TR/rdf-sparql-XMLres/) and [SPARQL Query Results JSON Format](https://www.w3.org/TR/sparql11-results-json/).

@ -1,6 +1,6 @@
/// A file serialization format. /// A file serialization format.
/// ///
/// Is implemented by `GraphFormat` for graph files and `DatasetFormat` for dataset files. /// Is implemented by [`GraphFormat`](../enum.GraphFormat.html) for graph files and [`DatasetFormat`](../enum.DatasetFormat.html) for dataset files.
#[deprecated(note = "Use directly the methods on the implementing types")] #[deprecated(note = "Use directly the methods on the implementing types")]
pub trait FileSyntax: Sized { pub trait FileSyntax: Sized {
/// Its canonical IRI according to the [Unique URIs for file formats registry](https://www.w3.org/ns/formats/). /// Its canonical IRI according to the [Unique URIs for file formats registry](https://www.w3.org/ns/formats/).

@ -12,7 +12,7 @@ use std::collections::HashMap;
use std::io; use std::io;
use std::io::BufRead; use std::io::BufRead;
/// A reader for RDF graph serialization formats. /// Parsers for RDF graph serialization formats.
/// ///
/// It currently supports the following formats: /// It currently supports the following formats:
/// * [N-Triples](https://www.w3.org/TR/n-triples/) (`GraphFormat::NTriples`) /// * [N-Triples](https://www.w3.org/TR/n-triples/) (`GraphFormat::NTriples`)
@ -87,7 +87,7 @@ impl GraphParser {
} }
/// Allows reading triples. /// Allows reading triples.
/// Could be built using a `GraphParser`. /// Could be built using a [`GraphParser`](struct.GraphParser.html).
/// ///
/// ``` /// ```
/// use oxigraph::io::{GraphFormat, GraphParser}; /// use oxigraph::io::{GraphFormat, GraphParser};
@ -170,7 +170,7 @@ impl<R: BufRead> TripleReader<R> {
} }
} }
/// A reader for RDF dataset serialization formats. /// A parser for RDF dataset serialization formats.
/// ///
/// It currently supports the following formats: /// It currently supports the following formats:
/// * [N-Quads](https://www.w3.org/TR/n-quads/) (`DatasetFormat::NQuads`) /// * [N-Quads](https://www.w3.org/TR/n-quads/) (`DatasetFormat::NQuads`)
@ -241,7 +241,7 @@ impl DatasetParser {
} }
/// Allows reading quads. /// Allows reading quads.
/// Could be built using a `DatasetParser`. /// Could be built using a [`DatasetParser`](struct.DatasetParser.html).
/// ///
/// ``` /// ```
/// use oxigraph::io::{DatasetFormat, DatasetParser}; /// use oxigraph::io::{DatasetFormat, DatasetParser};

@ -2,17 +2,19 @@
//! //!
//! Its goal is to provide a compliant, safe and fast graph database. //! Its goal is to provide a compliant, safe and fast graph database.
//! //!
//! It currently provides three `Store` implementation providing [SPARQL 1.1 query](https://www.w3.org/TR/sparql11-query/) capability: //! It currently provides three store implementations providing [SPARQL 1.1 query](https://www.w3.org/TR/sparql11-query/) capability:
//! * `MemoryStore`: a simple in memory implementation. //! * [`MemoryStore`](store/memory/struct.MemoryStore.html): a simple in memory implementation.
//! * `RocksDbStore`: a file system implementation based on the [RocksDB](https://rocksdb.org/) key-value store. //! * [`RocksDbStore`](store/rocksdb/struct.RocksDbStore.html): a file system implementation based on the [RocksDB](https://rocksdb.org/) key-value store.
//! It requires the `"rocksdb"` feature to be activated. //! It requires the `"rocksdb"` feature to be activated.
//! It also requires the [clang](https://clang.llvm.org/) compiler to be installed. //! The `"rocksdb"` requires the [clang](https://clang.llvm.org/) compiler to be installed.
//! * `SledStore`: another file system implementation based on the [Sled](https://sled.rs/) key-value store. //! * [`SledStore`](store/sled/struct.SledStore.html): another file system implementation based on the [Sled](https://sled.rs/) key-value store.
//! It requires the `"sled"` feature to be activated. //! It requires the `"sled"` feature to be activated.
//! Sled is much faster to build than RockDB and does not require a C++ compiler. //! Sled is much faster to build than RockDB and does not require a C++ compiler.
//! However, Sled is still in developpment, less tested and data load seems much slower than RocksDB. //! However, Sled is still in developpment, less tested and data load seems much slower than RocksDB.
//! //!
//! Usage example with the `MemoryStore`: //! It also provides a set of utility functions for reading, writing and processing RDF files.
//!
//! Usage example with the [`MemoryStore`](store/memory/struct.MemoryStore.html):
//! //!
//! ``` //! ```
//! use oxigraph::MemoryStore; //! use oxigraph::MemoryStore;
@ -46,6 +48,8 @@
unsafe_code, unsafe_code,
unused_qualifications unused_qualifications
)] )]
#![doc(html_favicon_url = "https://raw.githubusercontent.com/oxigraph/oxigraph/master/logo.svg")]
#![doc(html_logo_url = "https://raw.githubusercontent.com/oxigraph/oxigraph/master/logo.svg")]
#![warn( #![warn(
clippy::unimplemented, clippy::unimplemented,
clippy::cast_lossless, clippy::cast_lossless,

@ -322,7 +322,7 @@ fn to_integer_id(id: &str) -> Option<u128> {
Some(value) Some(value)
} }
/// An error raised during `BlankNode` validation. /// An error raised during [`BlankNode`](struct.BlankNode.html) IDs validation.
#[allow(missing_copy_implementations)] #[allow(missing_copy_implementations)]
#[derive(Debug)] #[derive(Debug)]
pub struct BlankNodeIdParseError {} pub struct BlankNodeIdParseError {}

@ -1,4 +1,4 @@
//! Provides ready to use `NamedNode`s for basic RDF vocabularies //! Provides ready to use [`NamedNodeRef`s](struct.NamedNodeRef.html) for basic RDF vocabularies
pub mod rdf { pub mod rdf {
//! [RDF 1.1](https://www.w3.org/TR/rdf11-concepts/) vocabulary //! [RDF 1.1](https://www.w3.org/TR/rdf11-concepts/) vocabulary
@ -109,7 +109,7 @@ pub mod rdfs {
} }
pub mod xsd { pub mod xsd {
//! `NamedNode`s for [RDF compatible XSD datatypes](https://www.w3.org/TR/rdf11-concepts/#dfn-rdf-compatible-xsd-types) //! [RDF compatible XSD datatypes](https://www.w3.org/TR/rdf11-concepts/#dfn-rdf-compatible-xsd-types)
use crate::model::named_node::NamedNodeRef; use crate::model::named_node::NamedNodeRef;
/// true, false /// true, false

@ -6,10 +6,7 @@ use std::error;
use std::fmt; use std::fmt;
use std::io; use std::io;
/// SPARQL evaluation error. /// A SPARQL evaluation error.
///
/// The `wrap` method allows us to make this type wrap any implementation of `std::error::Error`.
/// This type also avoids heap allocations for the most common cases of evaluation errors.
#[derive(Debug)] #[derive(Debug)]
#[non_exhaustive] #[non_exhaustive]
pub enum EvaluationError { pub enum EvaluationError {

@ -169,7 +169,7 @@ impl QueryOptions {
self self
} }
/// Use a given `ServiceHandler` to execute SPARQL SERVICE calls /// Use a given [`ServiceHandler`](trait.ServiceHandler.html) to execute [SPARQL 1.1 Federated Query](https://www.w3.org/TR/sparql11-federated-query/) SERVICE calls.
pub fn with_service_handler(mut self, service_handler: impl ServiceHandler + 'static) -> Self { pub fn with_service_handler(mut self, service_handler: impl ServiceHandler + 'static) -> Self {
self.service_handler = Rc::new(ErrorConversionServiceHandler { self.service_handler = Rc::new(ErrorConversionServiceHandler {
handler: service_handler, handler: service_handler,
@ -178,9 +178,10 @@ impl QueryOptions {
} }
} }
/// Handler for SPARQL SERVICEs. /// Handler for [SPARQL 1.1 Federated Query](https://www.w3.org/TR/sparql11-federated-query/) SERVICE.
/// ///
/// Might be used to implement [SPARQL 1.1 Federated Query](https://www.w3.org/TR/sparql11-federated-query/) /// Should be given to [`QueryOptions`](struct.QueryOptions.html#method.with_service_handler)
/// before evaluating a SPARQL query that uses SERVICE calls.
/// ///
/// ``` /// ```
/// use oxigraph::MemoryStore; /// use oxigraph::MemoryStore;

@ -298,7 +298,7 @@ pub struct QuerySolution {
} }
impl QuerySolution { impl QuerySolution {
/// Returns a value for a given position in the tuple (`usize`) or a given variable name (`&str` or `Variable`) /// Returns a value for a given position in the tuple ([`usize`](https://doc.rust-lang.org/std/primitive.usize.html)) or a given variable name ([`&str`](https://doc.rust-lang.org/std/primitive.str.html) or [`Variable`](struct.Variable.html))
/// ///
/// ```ignore /// ```ignore
/// let foo = solution.get("foo"); // Get the value of the variable ?foo if it exists /// let foo = solution.get("foo"); // Get the value of the variable ?foo if it exists

@ -24,7 +24,7 @@ use std::{fmt, io};
/// In-memory store. /// In-memory store.
/// It encodes a [RDF dataset](https://www.w3.org/TR/rdf11-concepts/#dfn-rdf-dataset) and allows to query and update it using SPARQL. /// It encodes a [RDF dataset](https://www.w3.org/TR/rdf11-concepts/#dfn-rdf-dataset) and allows to query and update it using SPARQL.
/// It is cheap to build using the `MemoryStore::new()` method. /// It is cheap to build using the [`MemoryStore::new()`](#method.new) method.
/// ///
/// Usage example: /// Usage example:
/// ``` /// ```
@ -77,7 +77,7 @@ impl Default for MemoryStore {
} }
impl MemoryStore { impl MemoryStore {
/// Constructs a new `MemoryStore` /// Constructs a new [`MemoryStore`]()
pub fn new() -> Self { pub fn new() -> Self {
Self { Self {
indexes: Arc::new(RwLock::default()), indexes: Arc::new(RwLock::default()),
@ -212,10 +212,10 @@ impl MemoryStore {
self.indexes().spog.is_empty() self.indexes().spog.is_empty()
} }
/// Executes a transaction. /// Executes an ACID transaction.
/// ///
/// The transaction is executed if the given closure returns `Ok`. /// The transaction is executed if the given closure returns `Ok`.
/// Nothing is done if the clusre returns `Err`. /// The transaction if rollbacked if the closure returns `Err`.
/// ///
/// Usage example: /// Usage example:
/// ``` /// ```
@ -283,8 +283,12 @@ impl MemoryStore {
/// # Result::<_,Box<dyn std::error::Error>>::Ok(()) /// # Result::<_,Box<dyn std::error::Error>>::Ok(())
/// ``` /// ```
/// ///
/// Errors related to parameter validation like the base IRI use the `INVALID_INPUT` error kind. /// Warning: This functions saves the triples during the parsing.
/// Errors related to a bad syntax in the loaded file use the `INVALID_DATA` error kind. /// If the parsing fails in the middle of the file, the triples read before stay in the store.
/// Use a (memory greedy) [transaction](#method.transaction) if you do not want that.
///
/// Errors related to parameter validation like the base IRI use the [`InvalidInput`](https://doc.rust-lang.org/std/io/enum.ErrorKind.html#variant.InvalidInput) error kind.
/// Errors related to a bad syntax in the loaded file use the [`InvalidData`](https://doc.rust-lang.org/std/io/enum.ErrorKind.html#variant.InvalidData) error kind.
pub fn load_graph<'a>( pub fn load_graph<'a>(
&self, &self,
reader: impl BufRead, reader: impl BufRead,
@ -318,8 +322,12 @@ impl MemoryStore {
/// # Result::<_,Box<dyn std::error::Error>>::Ok(()) /// # Result::<_,Box<dyn std::error::Error>>::Ok(())
/// ``` /// ```
/// ///
/// Errors related to parameter validation like the base IRI use the `INVALID_INPUT` error kind. /// Warning: This functions saves the quads during the parsing.
/// Errors related to a bad syntax in the loaded file use the `INVALID_DATA` error kind. /// If the parsing fails in the middle of the file, the quads read before stay in the store.
/// Use a (memory greedy) [transaction](#method.transaction) if you do not want that.
///
/// Errors related to parameter validation like the base IRI use the [`InvalidInput`](https://doc.rust-lang.org/std/io/enum.ErrorKind.html#variant.InvalidInput) error kind.
/// Errors related to a bad syntax in the loaded file use the [`InvalidData`](https://doc.rust-lang.org/std/io/enum.ErrorKind.html#variant.InvalidData) error kind.
pub fn load_dataset( pub fn load_dataset(
&self, &self,
reader: impl BufRead, reader: impl BufRead,
@ -376,8 +384,8 @@ impl MemoryStore {
/// # std::io::Result::Ok(()) /// # std::io::Result::Ok(())
/// ``` /// ```
/// ///
/// Errors related to parameter validation like the base IRI use the `INVALID_INPUT` error kind. /// Errors related to parameter validation like the base IRI use the [`InvalidInput`](https://doc.rust-lang.org/std/io/enum.ErrorKind.html#variant.InvalidInput) error kind.
/// Errors related to a bad syntax in the loaded file use the `INVALID_DATA` error kind. /// Errors related to a bad syntax in the loaded file use the [`InvalidData`](https://doc.rust-lang.org/std/io/enum.ErrorKind.html#variant.InvalidData) error kind.
pub fn dump_graph<'a>( pub fn dump_graph<'a>(
&self, &self,
writer: impl Write, writer: impl Write,
@ -410,8 +418,8 @@ impl MemoryStore {
/// # std::io::Result::Ok(()) /// # std::io::Result::Ok(())
/// ``` /// ```
/// ///
/// Errors related to parameter validation like the base IRI use the `INVALID_INPUT` error kind. /// Errors related to parameter validation like the base IRI use the [`InvalidInput`](https://doc.rust-lang.org/std/io/enum.ErrorKind.html#variant.InvalidInput) error kind.
/// Errors related to a bad syntax in the loaded file use the `INVALID_DATA` error kind. /// Errors related to a bad syntax in the loaded file use the [`InvalidData`](https://doc.rust-lang.org/std/io/enum.ErrorKind.html#variant.InvalidData) error kind.
pub fn dump_dataset(&self, writer: impl Write, format: DatasetFormat) -> Result<(), io::Error> { pub fn dump_dataset(&self, writer: impl Write, format: DatasetFormat) -> Result<(), io::Error> {
dump_dataset( dump_dataset(
self.quads_for_pattern(None, None, None, None).map(Ok), self.quads_for_pattern(None, None, None, None).map(Ok),
@ -930,7 +938,7 @@ fn quad_map_flatten<'a, T: Copy>(gspo: &'a QuadMap<T>) -> impl Iterator<Item = (
}) })
} }
/// A prepared [SPARQL query](https://www.w3.org/TR/sparql11-query/) for the `MemoryStore`. /// A prepared [SPARQL query](https://www.w3.org/TR/sparql11-query/) for the [`MemoryStore`](struct.MemoryStore.html).
pub struct MemoryPreparedQuery(SimplePreparedQuery<MemoryStore>); pub struct MemoryPreparedQuery(SimplePreparedQuery<MemoryStore>);
impl MemoryPreparedQuery { impl MemoryPreparedQuery {
@ -940,7 +948,7 @@ impl MemoryPreparedQuery {
} }
} }
/// Allows to insert and delete quads during a transaction with the `MemoryStore`. /// Allows to insert and delete quads during an ACID transaction with the [`MemoryStore`](struct.MemoryStore.html).
pub struct MemoryTransaction { pub struct MemoryTransaction {
ops: Vec<TransactionOp>, ops: Vec<TransactionOp>,
} }
@ -973,6 +981,10 @@ impl MemoryTransaction {
/// assert_eq!(vec![Quad::new(ex.clone(), ex.clone(), ex.clone(), None)], results); /// assert_eq!(vec![Quad::new(ex.clone(), ex.clone(), ex.clone(), None)], results);
/// # Result::<_, oxigraph::sparql::EvaluationError>::Ok(()) /// # Result::<_, oxigraph::sparql::EvaluationError>::Ok(())
/// ``` /// ```
///
/// If the file parsing fails in the middle of the file, the triples read before are still
/// considered by the transaction. Rollback the transaction by making the transaction closure
/// return an error if you don't want that.
pub fn load_graph<'a>( pub fn load_graph<'a>(
&mut self, &mut self,
reader: impl BufRead, reader: impl BufRead,
@ -1014,6 +1026,10 @@ impl MemoryTransaction {
/// assert_eq!(vec![Quad::new(ex.clone(), ex.clone(), ex.clone(), Some(ex.into()))], results); /// assert_eq!(vec![Quad::new(ex.clone(), ex.clone(), ex.clone(), Some(ex.into()))], results);
/// # Result::<_, oxigraph::sparql::EvaluationError>::Ok(()) /// # Result::<_, oxigraph::sparql::EvaluationError>::Ok(())
/// ``` /// ```
///
/// If the file parsing fails in the middle of the file, the quads read before are still
/// considered by the transaction. Rollback the transaction by making the transaction closure
/// return an error if you don't want that.
pub fn load_dataset( pub fn load_dataset(
&mut self, &mut self,
reader: impl BufRead, reader: impl BufRead,

@ -81,7 +81,7 @@ const COLUMN_FAMILIES: [&str; 7] = [
const MAX_TRANSACTION_SIZE: usize = 1024; const MAX_TRANSACTION_SIZE: usize = 1024;
impl RocksDbStore { impl RocksDbStore {
/// Opens a `RocksDbStore` /// Opens a [`RocksDbStore`]()
pub fn open(path: impl AsRef<Path>) -> Result<Self, io::Error> { pub fn open(path: impl AsRef<Path>) -> Result<Self, io::Error> {
let mut options = Options::default(); let mut options = Options::default();
options.create_if_missing(true); options.create_if_missing(true);
@ -95,7 +95,7 @@ impl RocksDbStore {
/// Executes a [SPARQL 1.1 query](https://www.w3.org/TR/sparql11-query/). /// Executes a [SPARQL 1.1 query](https://www.w3.org/TR/sparql11-query/).
/// ///
/// See `MemoryStore` for a usage example. /// See [`MemoryStore`](../memory/struct.MemoryStore.html#method.query) for a usage example.
pub fn query( pub fn query(
&self, &self,
query: impl TryInto<Query, Error = impl Into<EvaluationError>>, query: impl TryInto<Query, Error = impl Into<EvaluationError>>,
@ -107,7 +107,7 @@ impl RocksDbStore {
/// Prepares a [SPARQL 1.1 query](https://www.w3.org/TR/sparql11-query/) and returns an object that could be used to execute it. /// Prepares a [SPARQL 1.1 query](https://www.w3.org/TR/sparql11-query/) and returns an object that could be used to execute it.
/// It is useful if you want to execute multiple times the same SPARQL query. /// It is useful if you want to execute multiple times the same SPARQL query.
/// ///
/// See `MemoryStore` for a usage example. /// See [`MemoryStore`](../memory/struct.MemoryStore.html#method.prepare_query) for a usage example.
pub fn prepare_query( pub fn prepare_query(
&self, &self,
query: impl TryInto<Query, Error = impl Into<EvaluationError>>, query: impl TryInto<Query, Error = impl Into<EvaluationError>>,
@ -122,7 +122,7 @@ impl RocksDbStore {
/// Retrieves quads with a filter on each quad component /// Retrieves quads with a filter on each quad component
/// ///
/// See `MemoryStore` for a usage example. /// See [`MemoryStore`](../memory/struct.MemoryStore.html#method.quads_for_pattern) for a usage example.
pub fn quads_for_pattern( pub fn quads_for_pattern(
&self, &self,
subject: Option<NamedOrBlankNodeRef<'_>>, subject: Option<NamedOrBlankNodeRef<'_>>,
@ -164,12 +164,14 @@ impl RocksDbStore {
.is_none() .is_none()
} }
/// Executes a transaction. /// Executes an ACID transaction.
/// ///
/// The transaction is executed if the given closure returns `Ok`. /// The transaction is executed if the given closure returns `Ok`.
/// Nothing is done if the closure returns `Err`. /// The transaction is rollbacked if the closure returns `Err`.
/// ///
/// See `MemoryStore` for a usage example. /// The transaction data are stored in memory while the transaction is not committed or rollbacked.
///
/// See [`MemoryStore`](../memory/struct.MemoryStore.html#method.transaction) for a usage example.
pub fn transaction<'a, E: From<io::Error>>( pub fn transaction<'a, E: From<io::Error>>(
&'a self, &'a self,
f: impl FnOnce(&mut RocksDbTransaction<'a>) -> Result<(), E>, f: impl FnOnce(&mut RocksDbTransaction<'a>) -> Result<(), E>,
@ -187,12 +189,12 @@ impl RocksDbStore {
/// Loads a graph file (i.e. triples) into the store /// Loads a graph file (i.e. triples) into the store
/// ///
/// Warning: This functions saves the triples in batch. If the parsing fails in the middle of the file, /// Warning: This functions saves the triples in batch. If the parsing fails in the middle of the file,
/// only a part of it may be written. Use a (memory greedy) transaction if you do not want that. /// only a part of it may be written. Use a (memory greedy) [transaction](#method.transaction) if you do not want that.
/// ///
/// See `MemoryStore` for a usage example. /// See [`MemoryStore`](../memory/struct.MemoryStore.html#method.load_graph) for a usage example.
/// ///
/// Errors related to parameter validation like the base IRI use the `INVALID_INPUT` error kind. /// Errors related to parameter validation like the base IRI use the [`InvalidInput`](https://doc.rust-lang.org/std/io/enum.ErrorKind.html#variant.InvalidInput) error kind.
/// Errors related to a bad syntax in the loaded file use the `INVALID_DATA` error kind. /// Errors related to a bad syntax in the loaded file use the [`InvalidData`](https://doc.rust-lang.org/std/io/enum.ErrorKind.html#variant.InvalidData) error kind.
/// Errors related to data loading into the store use the other error kinds. /// Errors related to data loading into the store use the other error kinds.
pub fn load_graph<'a>( pub fn load_graph<'a>(
&self, &self,
@ -215,12 +217,12 @@ impl RocksDbStore {
/// Loads a dataset file (i.e. quads) into the store. /// Loads a dataset file (i.e. quads) into the store.
/// ///
/// Warning: This functions saves the quads in batch. If the parsing fails in the middle of the file, /// Warning: This functions saves the quads in batch. If the parsing fails in the middle of the file,
/// only a part of it may be written. Use a (memory greedy) transaction if you do not want that. /// only a part of it may be written. Use a (memory greedy) [transaction](#method.transaction) if you do not want that.
/// ///
/// See `MemoryStore` for a usage example. /// See [`MemoryStore`](../memory/struct.MemoryStore.html#method.load_dataset) for a usage example.
/// ///
/// Errors related to parameter validation like the base IRI use the `INVALID_INPUT` error kind. /// Errors related to parameter validation like the base IRI use the [`InvalidInput`](https://doc.rust-lang.org/std/io/enum.ErrorKind.html#variant.InvalidInput) error kind.
/// Errors related to a bad syntax in the loaded file use the `INVALID_DATA` error kind. /// Errors related to a bad syntax in the loaded file use the [`InvalidData`](https://doc.rust-lang.org/std/io/enum.ErrorKind.html#variant.InvalidData) error kind.
/// Errors related to data loading into the store use the other error kinds. /// Errors related to data loading into the store use the other error kinds.
pub fn load_dataset( pub fn load_dataset(
&self, &self,
@ -234,6 +236,7 @@ impl RocksDbStore {
} }
/// Adds a quad to this store. /// Adds a quad to this store.
/// This operation is atomic and could not leave the store in a bad state.
pub fn insert<'a>(&self, quad: impl Into<QuadRef<'a>>) -> Result<(), io::Error> { pub fn insert<'a>(&self, quad: impl Into<QuadRef<'a>>) -> Result<(), io::Error> {
let mut transaction = self.auto_batch_writer(); let mut transaction = self.auto_batch_writer();
let quad = transaction.encode_quad(quad.into())?; let quad = transaction.encode_quad(quad.into())?;
@ -242,6 +245,7 @@ impl RocksDbStore {
} }
/// Removes a quad from this store. /// Removes a quad from this store.
/// This operation is atomic and could not leave the store in a bad state.
pub fn remove<'a>(&self, quad: impl Into<QuadRef<'a>>) -> Result<(), io::Error> { pub fn remove<'a>(&self, quad: impl Into<QuadRef<'a>>) -> Result<(), io::Error> {
if let Some(quad) = self.get_encoded_quad(quad.into())? { if let Some(quad) = self.get_encoded_quad(quad.into())? {
let mut transaction = self.auto_batch_writer(); let mut transaction = self.auto_batch_writer();
@ -254,7 +258,7 @@ impl RocksDbStore {
/// Dumps a store graph into a file. /// Dumps a store graph into a file.
/// ///
/// See `MemoryStore` for a usage example. /// See [`MemoryStore`](../memory/struct.MemoryStore.html#method.dump_graph) for a usage example.
pub fn dump_graph<'a>( pub fn dump_graph<'a>(
&self, &self,
writer: impl Write, writer: impl Write,
@ -271,7 +275,7 @@ impl RocksDbStore {
/// Dumps the store dataset into a file. /// Dumps the store dataset into a file.
/// ///
/// See `MemoryStore` for a usage example. /// See [`MemoryStore`](../memory/struct.MemoryStore.html#method.dump_dataset) for a usage example.
pub fn dump_dataset(&self, writer: impl Write, syntax: DatasetFormat) -> Result<(), io::Error> { pub fn dump_dataset(&self, writer: impl Write, syntax: DatasetFormat) -> Result<(), io::Error> {
dump_dataset( dump_dataset(
self.quads_for_pattern(None, None, None, None), self.quads_for_pattern(None, None, None, None),
@ -579,7 +583,7 @@ impl ReadableEncodedStore for RocksDbStore {
} }
} }
/// A prepared [SPARQL query](https://www.w3.org/TR/sparql11-query/) for the `RocksDbStore`. /// A prepared [SPARQL query](https://www.w3.org/TR/sparql11-query/) for the [`RocksDbStore`](struct.RocksDbStore.html).
pub struct RocksDbPreparedQuery(SimplePreparedQuery<RocksDbStore>); pub struct RocksDbPreparedQuery(SimplePreparedQuery<RocksDbStore>);
impl RocksDbPreparedQuery { impl RocksDbPreparedQuery {
@ -683,7 +687,7 @@ impl WritableEncodedStore for AutoBatchWriter<'_> {
} }
} }
/// Allows inserting and deleting quads during a transaction with the `RocksDbStore`. /// Allows inserting and deleting quads during an ACID transaction with the [`RocksDbStore`](struct.RocksDbStore.html).
pub struct RocksDbTransaction<'a> { pub struct RocksDbTransaction<'a> {
store: &'a RocksDbStore, store: &'a RocksDbStore,
batch: WriteBatch, batch: WriteBatch,
@ -695,13 +699,17 @@ impl RocksDbTransaction<'_> {
/// Loads a graph file (i.e. triples) into the store during the transaction. /// Loads a graph file (i.e. triples) into the store during the transaction.
/// ///
/// Warning: Because the load happens during a transaction, /// Warning: Because the load happens during a transaction,
/// the full file content might be temporarily stored in main memory. /// the full file content is temporarily stored in main memory.
/// Do not use for big files. /// Do not use for big files.
/// ///
/// See `MemoryTransaction` for a usage example. /// See [`MemoryTransaction`](../memory/struct.MemoryTransaction.html#method.load_graph) for a usage example.
///
/// If the file parsing fails in the middle of the file, the triples read before are still
/// considered by the transaction. Rollback the transaction by making the transaction closure
/// return an error if you don't want that.
/// ///
/// Errors related to parameter validation like the base IRI use the `INVALID_INPUT` error kind. /// Errors related to parameter validation like the base IRI use the [`InvalidInput`](https://doc.rust-lang.org/std/io/enum.ErrorKind.html#variant.InvalidInput) error kind.
/// Errors related to a bad syntax in the loaded file use the `INVALID_DATA` error kind. /// Errors related to a bad syntax in the loaded file use the [`InvalidData`](https://doc.rust-lang.org/std/io/enum.ErrorKind.html#variant.InvalidData) error kind.
pub fn load_graph<'a>( pub fn load_graph<'a>(
&mut self, &mut self,
reader: impl BufRead, reader: impl BufRead,
@ -716,13 +724,17 @@ impl RocksDbTransaction<'_> {
/// Loads a dataset file (i.e. quads) into the store. into the store during the transaction. /// Loads a dataset file (i.e. quads) into the store. into the store during the transaction.
/// ///
/// Warning: Because the load happens during a transaction, /// Warning: Because the load happens during a transaction,
/// the full file content might be temporarily stored in main memory. /// the full file content is temporarily stored in main memory.
/// Do not use for big files. /// Do not use for big files.
/// ///
/// See `MemoryTransaction` for a usage example. /// See [`MemoryTransaction`](../memory/struct.MemoryTransaction.html#method.load_dataset) for a usage example.
///
/// If the file parsing fails in the middle of the file, the quads read before are still
/// considered by the transaction. Rollback the transaction by making the transaction closure
/// return an error if you don't want that.
/// ///
/// Errors related to parameter validation like the base IRI use the `INVALID_INPUT` error kind. /// Errors related to parameter validation like the base IRI use the [`InvalidInput`](https://doc.rust-lang.org/std/io/enum.ErrorKind.html#variant.InvalidInput) error kind.
/// Errors related to a bad syntax in the loaded file use the `INVALID_DATA` error kind. /// Errors related to a bad syntax in the loaded file use the [`InvalidData`](https://doc.rust-lang.org/std/io/enum.ErrorKind.html#variant.InvalidData) error kind.
pub fn load_dataset( pub fn load_dataset(
&mut self, &mut self,
reader: impl BufRead, reader: impl BufRead,

@ -29,7 +29,7 @@ use std::{fmt, io, str};
/// ///
/// To use it, the `"sled"` feature needs to be activated. /// To use it, the `"sled"` feature needs to be activated.
/// ///
/// Warning: quad insertions and deletions are not (yet) atomic. /// Warning: Sled is not stable yet and might break its storage format.
/// ///
/// Usage example: /// Usage example:
/// ``` /// ```
@ -77,12 +77,12 @@ type EncodedQuad = crate::store::numeric_encoder::EncodedQuad<StrHash>;
//TODO: indexes for the default graph and indexes for the named graphs (no more Optional and space saving) //TODO: indexes for the default graph and indexes for the named graphs (no more Optional and space saving)
impl SledStore { impl SledStore {
/// Opens a temporary `SledStore` that will be deleted after drop. /// Creates a temporary [`SledStore`]() that will be deleted after drop.
pub fn new() -> Result<Self, io::Error> { pub fn new() -> Result<Self, io::Error> {
Self::do_open(&Config::new().temporary(true)) Self::do_open(&Config::new().temporary(true))
} }
/// Opens a `SledStore` /// Opens a [`SledStore`]() and creates it if it does not exist yet.
pub fn open(path: impl AsRef<Path>) -> Result<Self, io::Error> { pub fn open(path: impl AsRef<Path>) -> Result<Self, io::Error> {
Self::do_open(&Config::new().path(path)) Self::do_open(&Config::new().path(path))
} }
@ -97,7 +97,7 @@ impl SledStore {
/// Executes a [SPARQL 1.1 query](https://www.w3.org/TR/sparql11-query/). /// Executes a [SPARQL 1.1 query](https://www.w3.org/TR/sparql11-query/).
/// ///
/// See `MemoryStore` for a usage example. /// See [`MemoryStore`](../memory/struct.MemoryStore.html#method.query) for a usage example.
pub fn query( pub fn query(
&self, &self,
query: impl TryInto<Query, Error = impl Into<EvaluationError>>, query: impl TryInto<Query, Error = impl Into<EvaluationError>>,
@ -108,7 +108,7 @@ impl SledStore {
/// Prepares a [SPARQL 1.1 query](https://www.w3.org/TR/sparql11-query/) and returns an object that could be used to execute it. /// Prepares a [SPARQL 1.1 query](https://www.w3.org/TR/sparql11-query/) and returns an object that could be used to execute it.
/// ///
/// See `MemoryStore` for a usage example. /// See [`MemoryStore`](../memory/struct.MemoryStore.html#method.prepare_query) for a usage example.
pub fn prepare_query( pub fn prepare_query(
&self, &self,
query: impl TryInto<Query, Error = impl Into<EvaluationError>>, query: impl TryInto<Query, Error = impl Into<EvaluationError>>,
@ -123,7 +123,7 @@ impl SledStore {
/// Retrieves quads with a filter on each quad component /// Retrieves quads with a filter on each quad component
/// ///
/// See `MemoryStore` for a usage example. /// See [`MemoryStore`](../memory/struct.MemoryStore.html#method.quads_for_pattern) for a usage example.
pub fn quads_for_pattern( pub fn quads_for_pattern(
&self, &self,
subject: Option<NamedOrBlankNodeRef<'_>>, subject: Option<NamedOrBlankNodeRef<'_>>,
@ -160,10 +160,10 @@ impl SledStore {
self.quads.is_empty() self.quads.is_empty()
} }
/// Executes a transaction. /// Executes an ACID transaction.
/// ///
/// The transaction is executed if the given closure returns `Ok`. /// The transaction is executed if the given closure returns `Ok`.
/// Nothing is done if the closure returns `Err`. /// The transaction is rollbacked if the closure returns `Err`.
/// ///
/// Usage example: /// Usage example:
/// ``` /// ```
@ -197,13 +197,16 @@ impl SledStore {
/// Loads a graph file (i.e. triples) into the store /// Loads a graph file (i.e. triples) into the store
/// ///
/// Warning: This functions saves the triples in batch. If the parsing fails in the middle of the file, /// Warning: This functions saves the triples in a not atomic way. If the parsing fails in the middle of the file,
/// only a part of it may be written. Use a (memory greedy) transaction if you do not want that. /// only a part of it may be written to the store.
/// Also, this method is optimized for performances and is not atomic.
/// It might leave the store in a bad state if a crash happens during a triple insertion.
/// Use a (memory greedy) [transaction](#method.transaction) if you do not want that.
/// ///
/// See `MemoryStore` for a usage example. /// See [`MemoryStore`](../memory/struct.MemoryStore.html#method.load_graph) for a usage example.
/// ///
/// Errors related to parameter validation like the base IRI use the `INVALID_INPUT` error kind. /// Errors related to parameter validation like the base IRI use the [`InvalidInput`](https://doc.rust-lang.org/std/io/enum.ErrorKind.html#variant.InvalidInput) error kind.
/// Errors related to a bad syntax in the loaded file use the `INVALID_DATA` error kind. /// Errors related to a bad syntax in the loaded file use the [`InvalidData`](https://doc.rust-lang.org/std/io/enum.ErrorKind.html#variant.InvalidData) error kind.
/// Errors related to data loading into the store use the other error kinds. /// Errors related to data loading into the store use the other error kinds.
pub fn load_graph<'a>( pub fn load_graph<'a>(
&self, &self,
@ -219,13 +222,16 @@ impl SledStore {
/// Loads a dataset file (i.e. quads) into the store. /// Loads a dataset file (i.e. quads) into the store.
/// ///
/// Warning: This functions saves the quads in batch. If the parsing fails in the middle of the file, /// Warning: This functions saves the triples in a not atomic way. If the parsing fails in the middle of the file,
/// only a part of it may be written. Use a (memory greedy) transaction if you do not want that. /// only a part of it may be written to the store.
/// Also, this method is optimized for performances and is not atomic.
/// It might leave the store in a bad state if a crash happens during a quad insertion.
/// Use a (memory greedy) [transaction](#method.transaction) if you do not want that.
/// ///
/// See `MemoryStore` for a usage example. /// See [`MemoryStore`](../memory/struct.MemoryStore.html#method.load_dataset) for a usage example.
/// ///
/// Errors related to parameter validation like the base IRI use the `INVALID_INPUT` error kind. /// Errors related to parameter validation like the base IRI use the [`InvalidInput`](https://doc.rust-lang.org/std/io/enum.ErrorKind.html#variant.InvalidInput) error kind.
/// Errors related to a bad syntax in the loaded file use the `INVALID_DATA` error kind. /// Errors related to a bad syntax in the loaded file use the [`InvalidData`](https://doc.rust-lang.org/std/io/enum.ErrorKind.html#variant.InvalidData) error kind.
/// Errors related to data loading into the store use the other error kinds. /// Errors related to data loading into the store use the other error kinds.
pub fn load_dataset( pub fn load_dataset(
&self, &self,
@ -239,6 +245,10 @@ impl SledStore {
} }
/// Adds a quad to this store. /// Adds a quad to this store.
///
/// This method is optimized for performances and is not atomic.
/// It might leave the store in a bad state if a crash happens during the insertion.
/// Use a (memory greedy) [transaction](#method.transaction) if you do not want that.
pub fn insert<'a>(&self, quad: impl Into<QuadRef<'a>>) -> Result<(), io::Error> { pub fn insert<'a>(&self, quad: impl Into<QuadRef<'a>>) -> Result<(), io::Error> {
let mut this = self; let mut this = self;
let quad = this.encode_quad(quad.into())?; let quad = this.encode_quad(quad.into())?;
@ -246,6 +256,10 @@ impl SledStore {
} }
/// Removes a quad from this store. /// Removes a quad from this store.
///
/// This method is optimized for performances and is not atomic.
/// It might leave the store in a bad state if a crash happens during the removal.
/// Use a (memory greedy) [transaction](#method.transaction) if you do not want that.
pub fn remove<'a>(&self, quad: impl Into<QuadRef<'a>>) -> Result<(), io::Error> { pub fn remove<'a>(&self, quad: impl Into<QuadRef<'a>>) -> Result<(), io::Error> {
if let Some(quad) = self.get_encoded_quad(quad.into())? { if let Some(quad) = self.get_encoded_quad(quad.into())? {
let mut this = self; let mut this = self;
@ -257,7 +271,7 @@ impl SledStore {
/// Dumps a store graph into a file. /// Dumps a store graph into a file.
/// ///
/// See `MemoryStore` for a usage example. /// See [`MemoryStore`](../memory/struct.MemoryStore.html#method.dump_graph) for a usage example.
pub fn dump_graph<'a>( pub fn dump_graph<'a>(
&self, &self,
writer: impl Write, writer: impl Write,
@ -274,7 +288,7 @@ impl SledStore {
/// Dumps the store dataset into a file. /// Dumps the store dataset into a file.
/// ///
/// See `MemoryStore` for a usage example. /// See [`MemoryStore`](../memory/struct.MemoryStore.html#method.dump_dataset) for a usage example.
pub fn dump_dataset(&self, writer: impl Write, format: DatasetFormat) -> Result<(), io::Error> { pub fn dump_dataset(&self, writer: impl Write, format: DatasetFormat) -> Result<(), io::Error> {
dump_dataset( dump_dataset(
self.quads_for_pattern(None, None, None, None), self.quads_for_pattern(None, None, None, None),
@ -584,7 +598,7 @@ impl<'a> WritableEncodedStore for &'a SledStore {
} }
} }
/// Allows inserting and deleting quads during a transaction with the `SeldStore`. /// Allows inserting and deleting quads during an ACID transaction with the [`SledStore`](struct.SledStore.html).
pub struct SledTransaction<'a> { pub struct SledTransaction<'a> {
quads: &'a TransactionalTree, quads: &'a TransactionalTree,
id2str: &'a TransactionalTree, id2str: &'a TransactionalTree,
@ -597,10 +611,14 @@ impl SledTransaction<'_> {
/// the full file content might be temporarily stored in main memory. /// the full file content might be temporarily stored in main memory.
/// Do not use for big files. /// Do not use for big files.
/// ///
/// See `MemoryTransaction` for a usage example. /// See [`MemoryTransaction`](../memory/struct.MemoryTransaction.html#method.load_graph) for a usage example.
/// ///
/// Errors related to parameter validation like the base IRI use the `INVALID_INPUT` error kind. /// If the file parsing fails in the middle of the file, the triples read before are still
/// Errors related to a bad syntax in the loaded file use the `INVALID_DATA` error kind. /// considered by the transaction. Rollback the transaction by making the transaction closure
/// return an error if you don't want that.
///
/// Errors related to parameter validation like the base IRI use the [`InvalidInput`](https://doc.rust-lang.org/std/io/enum.ErrorKind.html#variant.InvalidInput) error kind.
/// Errors related to a bad syntax in the loaded file use the [`InvalidData`](https://doc.rust-lang.org/std/io/enum.ErrorKind.html#variant.InvalidData) error kind.
pub fn load_graph<'a>( pub fn load_graph<'a>(
&self, &self,
reader: impl BufRead, reader: impl BufRead,
@ -619,10 +637,14 @@ impl SledTransaction<'_> {
/// the full file content might be temporarily stored in main memory. /// the full file content might be temporarily stored in main memory.
/// Do not use for big files. /// Do not use for big files.
/// ///
/// See `MemoryTransaction` for a usage example. /// See [`MemoryTransaction`](../memory/struct.MemoryTransaction.html#method.load_dataset) for a usage example.
///
/// If the file parsing fails in the middle of the file, the quads read before are still
/// considered by the transaction. Rollback the transaction by making the transaction closure
/// return an error if you don't want that.
/// ///
/// Errors related to parameter validation like the base IRI use the `INVALID_INPUT` error kind. /// Errors related to parameter validation like the base IRI use the [`InvalidInput`](https://doc.rust-lang.org/std/io/enum.ErrorKind.html#variant.InvalidInput) error kind.
/// Errors related to a bad syntax in the loaded file use the `INVALID_DATA` error kind. /// Errors related to a bad syntax in the loaded file use the [`InvalidData`](https://doc.rust-lang.org/std/io/enum.ErrorKind.html#variant.InvalidData) error kind.
pub fn load_dataset( pub fn load_dataset(
&self, &self,
reader: impl BufRead, reader: impl BufRead,
@ -903,7 +925,7 @@ impl<T> From<SledConflictableTransactionError<T>> for ConflictableTransactionErr
} }
} }
/// A prepared [SPARQL query](https://www.w3.org/TR/sparql11-query/) for the `SledStore`. /// A prepared [SPARQL query](https://www.w3.org/TR/sparql11-query/) for the [`SledStore`](struct.SledStore.html).
pub struct SledPreparedQuery(SimplePreparedQuery<SledStore>); pub struct SledPreparedQuery(SimplePreparedQuery<SledStore>);
impl SledPreparedQuery { impl SledPreparedQuery {

Loading…
Cancel
Save