f183196859
Adopt new I/O API for parsing
1 year ago
217abaf7ee
Adopt new I/O API for serialization
1 year ago
b06d6506cb
Fixes "let else" formatting
1 year ago
001b6e07b7
Enforces "return self not must use" lint
2 years ago
cb9922379c
Uses "let else" syntax where relevant
2 years ago
a1cbfdf67d
Improves some code style details
2 years ago
76deca135c
BulkLoader: Uses thread::scope
2 years ago
7c0563cb1b
XSD type structs: use by-value instead of by-ref in methods
...
They are small types
2 years ago
c016116b09
Makes Clippy 1.65 happy
2 years ago
57d39cad24
Makes Clippy happy
2 years ago
7175784356
Improves Clippy lint list
2 years ago
1d02098b70
Fix missing GSPO compaction
2 years ago
a977adff91
Allows to collect RocksDB statistics
...
Hidden behind the "rocksdb_debug" features
2 years ago
6af6c9c0eb
Bulk loader: fixes the progress indicator display
2 years ago
7787be6e84
Bulk loader: pre-allocate some buffers
2 years ago
13976014e7
Bulk loader: reduces default system parameters
...
Uses by default targets of 1GB of RAM and 2 threads.
Data parsing is in most of the case slower than ingestion so no more than 2 threads are used anyway.
2 years ago
fbcbd60c0e
Python: adds Store.bulk_extend
2 years ago
26f4e2dc98
Allows to have secondary instances in memory
2 years ago
5f68cb3746
Cleans up RocksDB wrapper code and implements backup from read-only and secondary instances
2 years ago
b2385509a6
Improves read only and secondary test coverage
...
Adds the missing get method
2 years ago
855c39146d
Replaces Store::open_with_options with explicit variants
...
Trades enums and structs for methods
2 years ago
f7637ee5a5
Add option to open database in read-only mode
...
With read-only it's not possible to modify the data. Updates to the data
are possible via a primary instance of oxigraph, but will not be
reflected. The data is frozen at the time the read-only server is
started.
2 years ago
aa9476b9cc
Add option to open rocksdb in secondary mode
...
The database can be opened once in primary mode, but may be opened
multiple times in secondary mode. It's not possible to write data to the
database in secondary mode.
Secondary mode does not support checkpoints so there might be data
inconsistencies when data is changed while said data is queried. It
might happen that the result data mixes both elements before and after
the change that can not happen in primary mode.
2 years ago
1fa0633db3
Library: adds basic WASI support
...
Adds tests for WASM+JS and WASI to the CI
2 years ago
854e29ee38
Makes XSD datatypes a standalone crate
...
Allows other crates to reuse Oxigraph XSD datatypes implementation
2 years ago
808c9db007
String formatting: uses inline syntax
2 years ago
59aea75a30
Uses try_into for slice to array conversions
2 years ago
4cea628570
Upgrades to sysinfo 0.26
2 years ago
e9bbe8e036
Applies new Clippy suggestions
2 years ago
95fd439dc2
Updates dependencies
3 years ago
b04bdcceed
Makes the new Clippy happy
3 years ago
ad710dc825
Bulk loader: removes dead code
...
There were some dead code and key-values where deduplicated twice
3 years ago
c815782631
rlimits: be safe if the nice value is bigger than the hard one
3 years ago
cc4e3f8b0f
Bulk loader: avoids to write duplicated values
3 years ago
42f316f7db
Limits the number of opened file by RocksDB
...
This was generating too many open file errors
3 years ago
3cbde014a6
Limits the bulk loader thread count to 4
...
Allows writing bigger files
3 years ago
b5611445eb
Fixes ARM compilation
3 years ago
d0b3d76bf1
Bulk loader: use as much memory as possible
3 years ago
c5f12f10f6
Better RocksDB error Debug implementation
3 years ago
54c6f2c19f
Bulk load: use megabytes as unit
3 years ago
9e969cd8eb
Adds methods to configure bulk loader system consumption
3 years ago
adea1899e5
Bulk loader: rewrite API and add hook to display progress indicators
3 years ago
ac64b58825
Fixes wasm build
3 years ago
822dd60596
Fixes invalid database generated from bulk load
...
Adds an invariant validation test
3 years ago
ad4bfe35c2
Fixes typos in the documentation
3 years ago
203bd4c080
Adds a backup system
...
RocksDB provides a great cheap backup feature thanks to the immutable SST storage.
Sadly it is not compatible with in-memory databases which do not rely on the SST files.
3 years ago
b9ef40df07
Avoids freeing null values
3 years ago
32d3f0782b
Fixes some new clippy warnings
3 years ago
7d58f451bd
Runs LLVM address sanitizer and fixes found bugs
3 years ago
fc26fa87a5
Makes a lot of dumb functions inline
3 years ago
Tpt