NextGraph
/
oxigraph
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Activity

Removes the MemoryStore

Browse Source
pull/171/head
Tpt 3 years ago
parent 3726d2cbfc
commit 1662bd7e57
9 changed files with 34 additions and 1921 deletions
Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL
Show Stats Download Patch File Download Diff File
  1. 17
      lib/README.md
  2. 22
      lib/benches/store.rs
  3. 18
      lib/src/lib.rs
  4. 2
      lib/src/sparql/mod.rs
  5. 24
      lib/src/sparql/model.rs
  6. 15
      lib/src/sparql/service.rs
  7. 1739
      lib/src/store/memory.rs
  8. 2
      lib/src/store/mod.rs
  9. 116
      lib/src/store/sophia.rs

17
lib/README.md
Unescape View File

@ -9,14 +9,9 @@ Oxigraph
Oxigraph is a graph database library implementing the [SPARQL](https://www.w3.org/TR/sparql11-overview/) standard.
Its goal is to provide a compliant, safe and fast graph database.
Its goal is to provide a compliant, safe and fast on-disk graph database.
It also provides a set of utility functions for reading, writing, and processing RDF files.
It currently provides three store implementations providing [SPARQL](https://www.w3.org/TR/sparql11-overview/) capability:
* `MemoryStore`: a simple in memory implementation.
* `SledStore`: a file system implementation based on the [Sled](https://sled.rs/) key-value store.
It requires the `"sled"` feature to be activated.
Oxigraph is in heavy development and SPARQL query evaluation has not been optimized yet.
The disabled by default `"sophia"` feature provides [`sophia_api`](https://docs.rs/sophia_api/) traits implementation on Oxigraph terms and stores.
@ -31,22 +26,22 @@ Oxigraph implements the following specifications:
A preliminary benchmark [is provided](../bench/README.md).
Usage example with the `MemoryStore`:
Usage example:
```rust
use oxigraph::MemoryStore;
use oxigraph::SledStore;
use oxigraph::model::*;
use oxigraph::sparql::QueryResults;
let store = MemoryStore::new();
let store = SledStore::new()?;
// insertion
let ex = NamedNode::new("http://example.com")?;
let quad = Quad::new(ex.clone(), ex.clone(), ex.clone(), None);
store.insert(quad.clone());
store.insert(&quad)?;
// quad filter
let results: Vec<Quad> = store.quads_for_pattern(Some(ex.as_ref().into()), None, None, None).collect();
let results = store.quads_for_pattern(Some(ex.as_ref().into()), None, None, None).collect::<Result<Vec<Quad>,_>>()?;
assert_eq!(vec![quad], results);
// SPARQL query

22
lib/benches/store.rs
Unescape View File

@ -1,6 +1,6 @@
use criterion::{criterion_group, criterion_main, BenchmarkId, Criterion, Throughput};
use oxigraph::model::{Dataset, Graph, NamedNode, Quad, Triple};
use oxigraph::{MemoryStore, SledStore};
use oxigraph::SledStore;
use rand::random;
use std::iter::FromIterator;
@ -8,7 +8,6 @@ criterion_group!(
store_load,
graph_load_bench,
dataset_load_bench,
memory_load_bench,
sled_load_bench
);
@ -46,25 +45,6 @@ fn dataset_load_bench(c: &mut Criterion) {
group.finish();
}
fn memory_load_bench(c: &mut Criterion) {
let mut group = c.benchmark_group("memory");
group.nresamples(10);
group.sample_size(10);
for size in [100, 1_000, 10_000].iter() {
group.throughput(Throughput::Elements(*size as u64));
let quads = create_quads(*size);
group.bench_function(BenchmarkId::from_parameter(size), |b| {
b.iter(|| {
let store = MemoryStore::new();
for quad in &quads {
store.insert(quad.clone());
}
});
});
}
group.finish();
}
fn sled_load_bench(c: &mut Criterion) {
let mut group = c.benchmark_group("sled");
group.nresamples(10);

18
lib/src/lib.rs
Unescape View File

@ -1,35 +1,28 @@
//! Oxigraph is a graph database library implementing the [SPARQL](https://www.w3.org/TR/sparql11-overview/) standard.
//!
//! Its goal is to provide a compliant, safe and fast graph database.
//! Its goal is to provide a compliant, safe and fast graph on-disk database.
//! It also provides a set of utility functions for reading, writing, and processing RDF files.
//!
//! It currently provides three store implementations providing [SPARQL](https://www.w3.org/TR/sparql11-overview/) capability:
//! * [`MemoryStore`](store::memory::MemoryStore): a simple in memory implementation.
//! * [`SledStore`](store::sled::SledStore): another file system implementation based on the [Sled](https://sled.rs/) key-value store.
//! It requires the `"sled"` feature to be activated.
//!
//! Oxigraph is in heavy development and SPARQL query evaluation has not been optimized yet.
//!
//! The disabled by default `"sophia"` feature provides [`sophia_api`](https://docs.rs/sophia_api/) traits implemention on Oxigraph terms and stores.
//!
//! Oxigraph also provides [a standalone HTTP server](https://crates.io/crates/oxigraph_server) based on this library.
//!
//! Usage example with the [`MemoryStore`](store::memory::MemoryStore):
//!
//! ```
//! use oxigraph::MemoryStore;
//! use oxigraph::SledStore;
//! use oxigraph::model::*;
//! use oxigraph::sparql::QueryResults;
//!
//! let store = MemoryStore::new();
//! let store = SledStore::new()?;
//!
//! // insertion
//! let ex = NamedNode::new("http://example.com")?;
//! let quad = Quad::new(ex.clone(), ex.clone(), ex.clone(), None);
//! store.insert(quad.clone());
//! store.insert(&quad)?;
//!
//! // quad filter
//! let results: Vec<Quad> = store.quads_for_pattern(Some(ex.as_ref().into()), None, None, None).collect();
//! let results = store.quads_for_pattern(Some(ex.as_ref().into()), None, None, None).collect::<Result<Vec<Quad>,_>>()?;
//! assert_eq!(vec![quad], results);
//!
//! // SPARQL query
@ -122,6 +115,5 @@ pub mod model;
pub mod sparql;
pub mod store;
pub use crate::store::memory::MemoryStore;
#[cfg(feature = "sled")]
pub use crate::store::sled::SledStore;

2
lib/src/sparql/mod.rs
Unescape View File

@ -1,6 +1,6 @@
//! [SPARQL](https://www.w3.org/TR/sparql11-overview/) implementation.
//!
//! Stores execute SPARQL. See [`MemoryStore`](super::store::memory::MemoryStore::query()) for an example.
//! Stores execute SPARQL. See [`SledStore`](super::store::memory::SledStore::query()) for an example.
pub mod algebra;
mod csv_results;

24
lib/src/sparql/model.rs
Unescape View File

@ -45,13 +45,13 @@ impl QueryResults {
/// This method fails if it is called on the `Graph` results
///
/// ```
/// use oxigraph::MemoryStore;
/// use oxigraph::SledStore;
/// use oxigraph::model::*;
/// use oxigraph::sparql::QueryResultsFormat;
///
/// let store = MemoryStore::new();
/// let ex = NamedNode::new("http://example.com")?;
/// store.insert(Quad::new(ex.clone(), ex.clone(), ex.clone(), None));
/// let store = SledStore::new()?;
/// let ex = NamedNodeRef::new("http://example.com")?;
/// store.insert(QuadRef::new(ex, ex, ex, GraphNameRef::DefaultGraph))?;
///
/// let mut results = Vec::new();
/// store.query("SELECT ?s WHERE { ?s ?p ?o }")?.write(&mut results, QueryResultsFormat::Json)?;
@ -76,14 +76,14 @@ impl QueryResults {
/// This method fails if it is called on the `Solution` or `Boolean` results
///
/// ```
/// use oxigraph::MemoryStore;
/// use oxigraph::SledStore;
/// use oxigraph::io::GraphFormat;
/// use oxigraph::model::*;
/// use std::io::Cursor;
///
/// let graph = "<http://example.com> <http://example.com> <http://example.com> .\n".as_bytes();
///
/// let store = MemoryStore::new();
/// let store = SledStore::new()?;
/// store.load_graph(Cursor::new(graph), GraphFormat::NTriples, &GraphName::DefaultGraph, None)?;
///
/// let mut results = Vec::new();
@ -217,10 +217,10 @@ impl QueryResultsFormat {
/// An iterator over [`QuerySolution`]s
///
/// ```
/// use oxigraph::MemoryStore;
/// use oxigraph::SledStore;
/// use oxigraph::sparql::QueryResults;
///
/// let store = MemoryStore::new();
/// let store = SledStore::new()?;
/// if let QueryResults::Solutions(solutions) = store.query("SELECT ?s WHERE { ?s ?p ?o }")? {
/// for solution in solutions {
/// println!("{:?}", solution?.get("s"));
@ -244,10 +244,10 @@ impl QuerySolutionIter {
/// The variables used in the solutions
///
/// ```
/// use oxigraph::MemoryStore;
/// use oxigraph::SledStore;
/// use oxigraph::sparql::{QueryResults, Variable};
///
/// let store = MemoryStore::new();
/// let store = SledStore::new()?;
/// if let QueryResults::Solutions(solutions) = store.query("SELECT ?s ?o WHERE { ?s ?p ?o }")? {
/// assert_eq!(solutions.variables(), &[Variable::new("s")?, Variable::new("o")?]);
/// }
@ -360,10 +360,10 @@ impl VariableSolutionIndex for Variable {
/// An iterator over the triples that compose a graph solution
///
/// ```
/// use oxigraph::MemoryStore;
/// use oxigraph::SledStore;
/// use oxigraph::sparql::QueryResults;
///
/// let store = MemoryStore::new();
/// let store = SledStore::new()?;
/// if let QueryResults::Graph(triples) = store.query("CONSTRUCT WHERE { ?s ?p ?o }")? {
/// for triple in triples {
/// println!("{}", triple?);

15
lib/src/sparql/service.rs
Unescape View File

@ -15,13 +15,12 @@ use std::error::Error;
/// before evaluating a SPARQL query that uses SERVICE calls.
///
/// ```
/// use oxigraph::MemoryStore;
/// use oxigraph::SledStore;
/// use oxigraph::model::*;
/// use oxigraph::sparql::{QueryOptions, QueryResults, ServiceHandler, Query, EvaluationError};
///
/// #[derive(Default)]
/// struct TestServiceHandler {
/// store: MemoryStore
/// store: SledStore
/// }
///
/// impl ServiceHandler for TestServiceHandler {
@ -36,10 +35,12 @@ use std::error::Error;
/// }
/// }
///
/// let store = MemoryStore::new();
/// let service = TestServiceHandler::default();
/// let ex = NamedNode::new("http://example.com")?;
/// service.store.insert(Quad::new(ex.clone(), ex.clone(), ex.clone(), None));
/// let store = SledStore::new()?;
/// let service = TestServiceHandler {
/// store: SledStore::new()?
/// };
/// let ex = NamedNodeRef::new("http://example.com")?;
/// service.store.insert(QuadRef::new(ex, ex, ex, GraphNameRef::DefaultGraph))?;
///
/// if let QueryResults::Solutions(mut solutions) = store.query_opt(
/// "SELECT ?s WHERE { SERVICE <http://example.com/service> { ?s ?p ?o } }",

1739
lib/src/store/memory.rs
View File

File diff suppressed because it is too large Load Diff

2
lib/src/store/mod.rs
Unescape View File

@ -2,7 +2,6 @@
#[cfg(feature = "sled")]
mod binary_encoder;
pub mod memory;
pub(crate) mod numeric_encoder;
#[cfg(feature = "sled")]
pub mod sled;
@ -10,7 +9,6 @@ pub(crate) mod small_string;
#[cfg(feature = "sophia")]
mod sophia;
pub use crate::store::memory::MemoryStore;
#[cfg(feature = "sled")]
pub use crate::store::sled::SledStore;

116
lib/src/store/sophia.rs
Unescape View File

@ -5,9 +5,8 @@ use crate::store::*;
use sophia_api::dataset::*;
use sophia_api::quad::stream::{QuadSource, StreamResult};
use sophia_api::quad::streaming_mode::{ByValue, StreamedQuad};
use sophia_api::term::{TTerm, TermKind, TryCopyTerm};
use sophia_api::term::{TTerm, TermKind};
use std::collections::HashSet;
use std::convert::Infallible;
use std::hash::Hash;
use std::iter::empty;
@ -412,80 +411,6 @@ macro_rules! impl_dataset {
};
}
mod memory {
use super::*;
impl_dataset!(
MemoryStore,
Infallible,
infallible_quad_map,
infallible_err_map
);
impl MutableDataset for MemoryStore {
type MutationError = Infallible;
fn insert<TS, TP, TO, TG>(
&mut self,
s: &TS,
p: &TP,
o: &TO,
g: Option<&TG>,
) -> MDResult<Self, bool>
where
TS: TTerm + ?Sized,
TP: TTerm + ?Sized,
TO: TTerm + ?Sized,
TG: TTerm + ?Sized,
{
let quad = match convert_quad(s, p, o, g) {
Some(quad) => quad,
None => return Ok(false),
};
MemoryStore::insert(self, quad);
Ok(true)
}
fn remove<TS, TP, TO, TG>(
&mut self,
s: &TS,
p: &TP,
o: &TO,
g: Option<&TG>,
) -> MDResult<Self, bool>
where
TS: TTerm + ?Sized,
TP: TTerm + ?Sized,
TO: TTerm + ?Sized,
TG: TTerm + ?Sized,
{
let mut buf_s = String::new();
let mut buf_p = String::new();
let mut buf_o = String::new();
let mut buf_g = String::new();
let quadref =
match convert_quadref(s, p, o, g, &mut buf_s, &mut buf_p, &mut buf_o, &mut buf_g) {
Some(quad) => quad,
None => return Ok(false),
};
MemoryStore::remove(self, quadref);
Ok(true)
}
}
impl CollectibleDataset for MemoryStore {
fn from_quad_source<QS: QuadSource>(
quads: QS,
) -> StreamResult<Self, QS::Error, Self::Error> {
let mut d = MemoryStore::new();
d.insert_all(quads)?;
Ok(d)
}
}
#[cfg(test)]
sophia_api::test_dataset_impl!(test, MemoryStore, false, false);
}
#[cfg(feature = "sled")]
mod sled {
use super::*;
@ -561,16 +486,6 @@ mod sled {
}
// helper functions
#[allow(clippy::unnecessary_wraps)]
fn infallible_quad_map<'a>(q: Quad) -> Result<StreamedSophiaQuad<'a>, Infallible> {
let q: SophiaQuad = q.into();
Ok(StreamedQuad::by_value(q))
}
fn infallible_err_map(_: EvaluationError) -> Infallible {
panic!("Unexpected error")
}
#[cfg(feature = "sled")]
fn io_quad_map<'a>(
res: Result<Quad, std::io::Error>,
@ -676,35 +591,6 @@ fn convert_iri_raw<'a>(
NamedNodeRef::new_unchecked(iri)
}
fn convert_quad<TS, TP, TO, TG>(s: &TS, p: &TP, o: &TO, g: Option<&TG>) -> Option<Quad>
where
TS: TTerm + ?Sized,
TP: TTerm + ?Sized,
TO: TTerm + ?Sized,
TG: TTerm + ?Sized,
{
let s = match NamedOrBlankNode::try_copy(s) {
Ok(s) => s,
Err(_) => return None,
};
let p = match NamedNode::try_copy(p) {
Ok(p) => p,
Err(_) => return None,
};
let o = match Term::try_copy(o) {
Ok(o) => o,
Err(_) => return None,
};
let g = match g {
None => GraphName::DefaultGraph,
Some(g) => match NamedOrBlankNode::try_copy(g) {
Ok(g) => g.into(),
Err(_) => return None,
},
};
Some(Quad::new(s, p, o, g))
}
fn convert_quadref<'a, TS, TP, TO, TG>(
s: &'a TS,
p: &'a TP,

Write Preview
Loading…
Cancel
Save