fork of https://github.com/oxigraph/rocksdb and https://github.com/facebook/rocksdb for nextgraph and oxigraph

You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

Akanksha Mahajan 04b2c16f9b Fix bug in rocksdb internal automatic prefetching (#9234) ... Summary: After introducing adaptive_readahead, the original flow got broken. Readahead size was set to 0 because of which rocksdb wasn't be able to do automatic prefetching which it enables after seeing sequential reads. This PR fixes it. ---------------------------------------------------------------------------------------------------- Before this patch: b_bench -use_existing_db=true -db=/tmp/prefix_scan -benchmarks="seekrandom" -key_size=32 -value_size=512 -num=5000000 -use_direct_reads=true -seek_nexts=327680 -duration=120 -ops_between_duration_checks=1 Initializing RocksDB Options from the specified file Initializing RocksDB Options from command-line flags RocksDB: version 6.27 Date: Tue Nov 30 11:56:50 2021 CPU: 24 * Intel Core Processor (Broadwell) CPUCache: 16384 KB Keys: 32 bytes each (+ 0 bytes user-defined timestamp) Values: 512 bytes each (256 bytes after compression) Entries: 5000000 Prefix: 0 bytes Keys per prefix: 0 RawSize: 2594.0 MB (estimated) FileSize: 1373.3 MB (estimated) Write rate: 0 bytes/second Read rate: 0 ops/second Compression: Snappy Compression sampling rate: 0 Memtablerep: SkipListFactory Perf Level: 1 WARNING: Assertions are enabled; benchmarks unnecessarily slow ------------------------------------------------ DB path: [/tmp/prefix_scan] seekrandom : 5356367.174 micros/op 0 ops/sec; 29.4 MB/s (23 of 23 found) ---------------------------------------------------------------------------------------------------- After the patch: ./db_bench -use_existing_db=true -db=/tmp/prefix_scan -benchmarks="seekrandom" -key_size=32 -value_size=512 -num=5000000 -use_direct_reads=true -seek_nexts=327680 -duration=120 -ops_between_duration_checks=1 Initializing RocksDB Options from the specified file Initializing RocksDB Options from command-line flags RocksDB: version 6.27 Date: Tue Nov 30 14:38:33 2021 CPU: 24 * Intel Core Processor (Broadwell) CPUCache: 16384 KB Keys: 32 bytes each (+ 0 bytes user-defined timestamp) Values: 512 bytes each (256 bytes after compression) Entries: 5000000 Prefix: 0 bytes Keys per prefix: 0 RawSize: 2594.0 MB (estimated) FileSize: 1373.3 MB (estimated) Write rate: 0 bytes/second Read rate: 0 ops/second Compression: Snappy Compression sampling rate: 0 Memtablerep: SkipListFactory Perf Level: 1 WARNING: Assertions are enabled; benchmarks unnecessarily slow ------------------------------------------------ DB path: [/tmp/prefix_scan] seekrandom : 456504.277 micros/op 2 ops/sec; 359.8 MB/s (264 of 264 found) Pull Request resolved: https://github.com/facebook/rocksdb/pull/9234 Test Plan: Ran ./db_bench -db=/data/mysql/rocksdb/prefix_scan -benchmarks="fillseq" -key_size=32 -value_size=512 -num=5000000 -use_d irect_io_for_flush_and_compaction=true -target_file_size_base=16777216 and then ./db_bench -use_existing_db=true -db=/data/mysql/rocksdb/prefix_scan -benchmarks="seekrandom" -key_size=32 -value_siz e=512 -num=5000000 -use_direct_reads=true -seek_nexts=327680 -duration=120 -ops_between_duration_checks=1 and compared the results. Reviewed By: anand1976 Differential Revision: D32743965 Pulled By: akankshamahajan15 fbshipit-source-id: b950fba68c91963b7deb5c20acdf471bc60251f5		3 years ago
..
delete_scheduler.cc	Skip directory fsync for filesystem btrfs (#8903)	3 years ago
delete_scheduler.h	Use SystemClock* instead of std::shared_ptr<SystemClock> in lower level routines (#8033)	4 years ago
delete_scheduler_test.cc	Use SST file manager to track blob files as well (#8037)	4 years ago
file_prefetch_buffer.cc	Fix bug in rocksdb internal automatic prefetching (#9234)	3 years ago
file_prefetch_buffer.h	Fix bug in rocksdb internal automatic prefetching (#9234)	3 years ago
file_util.cc	Support readahead during compaction for blob files (#9187)	3 years ago
file_util.h	Allow WAL dir to change with db dir (#8582)	3 years ago
filename.cc	Skip directory fsync for filesystem btrfs (#8903)	3 years ago
filename.h	Add (Live)FileStorageInfo API (#8968)	3 years ago
line_file_reader.cc	Replace Status with IOStatus in the backupable_db (#8820)	3 years ago
line_file_reader.h	Replace Status with IOStatus in the backupable_db (#8820)	3 years ago
prefetch_test.cc	Fix bug in rocksdb internal automatic prefetching (#9234)	3 years ago
random_access_file_reader.cc	Add listener API that notifies on IOError (#9177)	3 years ago
random_access_file_reader.h	Add listener API that notifies on IOError (#9177)	3 years ago
random_access_file_reader_test.cc	use the pointer directly (#8095)	4 years ago
read_write_util.cc	Move old files to warm tier in FIFO compactions (#8310)	3 years ago
read_write_util.h	Refactor: add LineFileReader and Status::MustCheck (#8026)	4 years ago
readahead_file_info.h	Reuse internal auto readhead_size at each Level (expect L0) for Iterations (#9056)	3 years ago
readahead_raf.cc	Make StringEnv, StringSink, StringSource use FS classes (#7786)	4 years ago
readahead_raf.h	Make StringEnv, StringSink, StringSource use FS classes (#7786)	4 years ago
sequence_file_reader.cc	Add file operation callbacks to SequentialFileReader (#8982)	3 years ago
sequence_file_reader.h	Add listener API that notifies on IOError (#9177)	3 years ago
sst_file_manager_impl.cc	Cleanup includes in dbformat.h (#8930)	3 years ago
sst_file_manager_impl.h	Use SST file manager to track blob files as well (#8037)	4 years ago
writable_file_writer.cc	Add listener API that notifies on IOError (#9177)	3 years ago
writable_file_writer.h	Add listener API that notifies on IOError (#9177)	3 years ago