fork of https://github.com/oxigraph/rocksdb and https://github.com/facebook/rocksdb for nextgraph and oxigraph

You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

Maysam Yabandeh f383641a1d Unordered Writes (#5218) ... Summary: Performing unordered writes in rocksdb when unordered_write option is set to true. When enabled the writes to memtable are done without joining any write thread. This offers much higher write throughput since the upcoming writes would not have to wait for the slowest memtable write to finish. The tradeoff is that the writes visible to a snapshot might change over time. If the application cannot tolerate that, it should implement its own mechanisms to work around that. Using TransactionDB with WRITE_PREPARED write policy is one way to achieve that. Doing so increases the max throughput by 2.2x without however compromising the snapshot guarantees. The patch is prepared based on an original by siying Existing unit tests are extended to include unordered_write option. Benchmark Results: ``` TEST_TMPDIR=/dev/shm/ ./db_bench_unordered --benchmarks=fillrandom --threads=32 --num=10000000 -max_write_buffer_number=16 --max_background_jobs=64 --batch_size=8 --writes=3000000 -level0_file_num_compaction_trigger=99999 --level0_slowdown_writes_trigger=99999 --level0_stop_writes_trigger=99999 -enable_pipelined_write=false -disable_auto_compactions --unordered_write=1 ``` With WAL - Vanilla RocksDB: 78.6 MB/s - WRITER_PREPARED with unordered_write: 177.8 MB/s (2.2x) - unordered_write: 368.9 MB/s (4.7x with relaxed snapshot guarantees) Without WAL - Vanilla RocksDB: 111.3 MB/s - WRITER_PREPARED with unordered_write: 259.3 MB/s MB/s (2.3x) - unordered_write: 645.6 MB/s (5.8x with relaxed snapshot guarantees) - WRITER_PREPARED with unordered_write disable concurrency control: 185.3 MB/s MB/s (2.35x) Limitations: - The feature is not yet extended to `max_successive_merges` > 0. The feature is also incompatible with `enable_pipelined_write` = true as well as with `allow_concurrent_memtable_write` = false. Pull Request resolved: https://github.com/facebook/rocksdb/pull/5218 Differential Revision: D15219029 Pulled By: maysamyabandeh fbshipit-source-id: 38f2abc4af8780148c6128acdba2b3227bc81759		5 years ago
..
cf_options.cc	Refresh snapshot list during long compactions (2nd attempt) (#5278)	5 years ago
cf_options.h	Refresh snapshot list during long compactions (2nd attempt) (#5278)	5 years ago
db_options.cc	Unordered Writes (#5218)	5 years ago
db_options.h	Unordered Writes (#5218)	5 years ago
options.cc	Refresh snapshot list during long compactions (2nd attempt) (#5278)	5 years ago
options_helper.cc	Unordered Writes (#5218)	5 years ago
options_helper.h	Option string/map/file can set env from object registry (#5237)	5 years ago
options_parser.cc	Add BlockBasedTableOptions::index_shortening (#5174)	5 years ago
options_parser.h	WritePrepared Txn: Optimize for recoverable state	7 years ago
options_sanity_check.cc	Change RocksDB License	7 years ago
options_sanity_check.h	Change RocksDB License	7 years ago
options_settable_test.cc	Unordered Writes (#5218)	5 years ago
options_test.cc	Refresh snapshot list during long compactions (2nd attempt) (#5278)	5 years ago