cc4e3f8b0f
Bulk loader: avoids to write duplicated values
3 years ago
42f316f7db
Limits the number of opened file by RocksDB
...
This was generating too many open file errors
3 years ago
3cbde014a6
Limits the bulk loader thread count to 4
...
Allows writing bigger files
3 years ago
b5611445eb
Fixes ARM compilation
3 years ago
d0b3d76bf1
Bulk loader: use as much memory as possible
3 years ago
c5f12f10f6
Better RocksDB error Debug implementation
3 years ago
54c6f2c19f
Bulk load: use megabytes as unit
3 years ago
9e969cd8eb
Adds methods to configure bulk loader system consumption
3 years ago
adea1899e5
Bulk loader: rewrite API and add hook to display progress indicators
3 years ago
ac64b58825
Fixes wasm build
3 years ago
822dd60596
Fixes invalid database generated from bulk load
...
Adds an invariant validation test
3 years ago
ad4bfe35c2
Fixes typos in the documentation
3 years ago
203bd4c080
Adds a backup system
...
RocksDB provides a great cheap backup feature thanks to the immutable SST storage.
Sadly it is not compatible with in-memory databases which do not rely on the SST files.
3 years ago
b9ef40df07
Avoids freeing null values
3 years ago
32d3f0782b
Fixes some new clippy warnings
3 years ago
7d58f451bd
Runs LLVM address sanitizer and fixes found bugs
3 years ago
fc26fa87a5
Makes a lot of dumb functions inline
3 years ago
ec5c7c86be
Improves RDF I/O ParseError
3 years ago
f5545f1948
Makes Oxigraph model standalone as oxrdf
...
Allows to reuse it in other crates like spargebra
3 years ago
4337349aad
Moves xsd module to root
...
Makes "model" module more stand-alone
3 years ago
a91ea89fff
Moves str2id outside of transactions
...
Allows avoiding conflicts and reducing transactional guarantees
3 years ago
72a17c4f72
Avoids using std::io::Error where possible
3 years ago
a33dbb6d06
Avoids abusing std::io::Error
...
Introduces new specific errors and make them implement Into<io::Error> for compatibility
3 years ago
18ec80c362
Exposes more of RocksDB internal Status struct
3 years ago
bada850284
Txn conflict: allows the OS to do something else before retrying
3 years ago
44d1a5f04c
Makes bulk load partial insertions atomic
...
Ensures that the stores does not end up in an inconsistent state.
3 years ago
f64001906c
Minor documentation improvements
3 years ago
8d20f65890
Adds back in memory WASM support
3 years ago
ed17e86853
Renames write_stt_files to insert_stt_files
3 years ago
a3e40556da
Automatically retries transactions on failure
3 years ago
eb4fb8254a
Lock the graph name when clearing a named graph
...
Makes sure we conflict if a quad is inserted in the graph while we clear it
3 years ago
e99d6f4ad0
Makes bulk loader multithreaded
3 years ago
b9d4f912dc
Bulk load: avoid not needed lookup
...
It's fine to have some extra inserts and avoid reading the DB state
3 years ago
7bb41e655e
SPARQL: we do not need to ensure strings exist anymore
3 years ago
e59c4612b2
RocksDB: Removes merge and compact operators
3 years ago
b7ee3a6767
Removes string usage counting
3 years ago
2c304aa29d
Share environment between RocksDB instances
...
Allow multiple RocksDB instance in the same process to share resources instead of fighting to get them.
3 years ago
6f44a5956b
Simplifies bulk load API
3 years ago
7b1c4e0ad5
Returns a clean error on transaction read after commit
3 years ago
607aa0b0dd
WIP: Transactions
3 years ago
9f414c13fd
Simplifies RocksDB SST API
3 years ago
367a1b4585
Reuse RocksDB transactional DB
...
First step for snapshots and transactions support
4 years ago
196d6d6576
Adds bulk load by directly write RocksDB SSTs
4 years ago
1d3108d27f
RocksDB: Uses the most recent storage format
4 years ago
569000b5ea
Storage: adds write batch
...
Atomic insertions and huge performance improvements for RocksDB
4 years ago
986d3e60bb
RocksDB: bulk load
...
TODO: we do a lot of lookups during load for GC, so the option is not very useful
4 years ago
98f9a307b8
Exposes a function to compact the database
...
Use it after load
4 years ago
8c0b4f5322
RocksDB: do not use transactiondb for now.
...
Not used yet
4 years ago
00ad6e634f
Cleaner storage backend architecture
4 years ago
73cd761229
Removes unused TermEncoder
4 years ago
Tpt