604d1bbe2e
BulkLoader: rename set_* methods to with_* methods
12 months ago
4084acb9b8
Renames dump_dataset and dump_graph to dump_to_write and dump_graph_to_write
12 months ago
efd5eec65d
Introduces load_from_read instead of load_graph and load_dataset
1 year ago
108721624f
Improves bulk loader doc comments
1 year ago
1eaa77ad93
Uses new rustdoc <div class="warning"> style
1 year ago
f10e5a40a3
Enables more Clippy lints
...
Auto-enable all Clippy lints that are not in nursery but a blacklist
1 year ago
c6e55c706a
RDF serialization: removes implicit flush
...
Allows to the user to control flushing
1 year ago
f183196859
Adopt new I/O API for parsing
1 year ago
217abaf7ee
Adopt new I/O API for serialization
1 year ago
922023b1da
Parsers do not need BufRead anymore
1 year ago
69d8ce6b4e
Migrates RDF/XML parser from Rio
1 year ago
001b6e07b7
Enforces "return self not must use" lint
2 years ago
71b1768d28
New N3/Turtle/TriG/N-Triple/N-Quad parsers and serializers
...
- Compatible with async IO
- Turtle/TriG parser recovery on simple errors
2 years ago
7175784356
Improves Clippy lint list
2 years ago
60ffd99ad8
Exposes the query profiler to the CLI app
2 years ago
13976014e7
Bulk loader: reduces default system parameters
...
Uses by default targets of 1GB of RAM and 2 threads.
Data parsing is in most of the case slower than ingestion so no more than 2 threads are used anyway.
2 years ago
fbcbd60c0e
Python: adds Store.bulk_extend
2 years ago
26f4e2dc98
Allows to have secondary instances in memory
2 years ago
855c39146d
Replaces Store::open_with_options with explicit variants
...
Trades enums and structs for methods
2 years ago
f7637ee5a5
Add option to open database in read-only mode
...
With read-only it's not possible to modify the data. Updates to the data
are possible via a primary instance of oxigraph, but will not be
reflected. The data is frozen at the time the read-only server is
started.
2 years ago
aa9476b9cc
Add option to open rocksdb in secondary mode
...
The database can be opened once in primary mode, but may be opened
multiple times in secondary mode. It's not possible to write data to the
database in secondary mode.
Secondary mode does not support checkpoints so there might be data
inconsistencies when data is changed while said data is queried. It
might happen that the result data mixes both elements before and after
the change that can not happen in primary mode.
2 years ago
1fa0633db3
Library: adds basic WASI support
...
Adds tests for WASM+JS and WASI to the CI
2 years ago
eedc4b3a71
Removes unnecessary qualified paths
2 years ago
839375c2a4
Adds an option to avoid aborting bulk loading on syntax error
3 years ago
54c6f2c19f
Bulk load: use megabytes as unit
3 years ago
9e969cd8eb
Adds methods to configure bulk loader system consumption
3 years ago
adea1899e5
Bulk loader: rewrite API and add hook to display progress indicators
3 years ago
ac64b58825
Fixes wasm build
3 years ago
822dd60596
Fixes invalid database generated from bulk load
...
Adds an invariant validation test
3 years ago
ad4bfe35c2
Fixes typos in the documentation
3 years ago
4efd193708
Improves documentation
3 years ago
0d14159b6e
Adds the Store.backup function to pyoxigraph
3 years ago
203bd4c080
Adds a backup system
...
RocksDB provides a great cheap backup feature thanks to the immutable SST storage.
Sadly it is not compatible with in-memory databases which do not rely on the SST files.
3 years ago
045fb8cbf6
Adds a public API for transaction
...
Very similar to the one of the previous SledStore
3 years ago
34a944036f
Simplifies UpdateOptions API
...
Keeps only useful methods
3 years ago
ec5c7c86be
Improves RDF I/O ParseError
3 years ago
a33dbb6d06
Avoids abusing std::io::Error
...
Introduces new specific errors and make them implement Into<io::Error> for compatibility
3 years ago
44d1a5f04c
Makes bulk load partial insertions atomic
...
Ensures that the stores does not end up in an inconsistent state.
3 years ago
f64001906c
Minor documentation improvements
3 years ago
d6d88b074c
Improves pyoxigraph documentation
3 years ago
8d20f65890
Adds back in memory WASM support
3 years ago
a3e40556da
Automatically retries transactions on failure
3 years ago
e99d6f4ad0
Makes bulk loader multithreaded
3 years ago
6f44a5956b
Simplifies bulk load API
3 years ago
607aa0b0dd
WIP: Transactions
3 years ago
196d6d6576
Adds bulk load by directly write RocksDB SSTs
3 years ago
569000b5ea
Storage: adds write batch
...
Atomic insertions and huge performance improvements for RocksDB
3 years ago
986d3e60bb
RocksDB: bulk load
...
TODO: we do a lot of lookups during load for GC, so the option is not very useful
3 years ago
98f9a307b8
Exposes a function to compact the database
...
Use it after load
3 years ago
bf0f178c41
Adds a ColumnFamilyDefinition struct to store column family options
3 years ago
Tpt