fork of https://github.com/oxigraph/rocksdb and https://github.com/facebook/rocksdb for nextgraph and oxigraph

You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

Akanksha Mahajan 04b2c16f9b Fix bug in rocksdb internal automatic prefetching (#9234) ... Summary: After introducing adaptive_readahead, the original flow got broken. Readahead size was set to 0 because of which rocksdb wasn't be able to do automatic prefetching which it enables after seeing sequential reads. This PR fixes it. ---------------------------------------------------------------------------------------------------- Before this patch: b_bench -use_existing_db=true -db=/tmp/prefix_scan -benchmarks="seekrandom" -key_size=32 -value_size=512 -num=5000000 -use_direct_reads=true -seek_nexts=327680 -duration=120 -ops_between_duration_checks=1 Initializing RocksDB Options from the specified file Initializing RocksDB Options from command-line flags RocksDB: version 6.27 Date: Tue Nov 30 11:56:50 2021 CPU: 24 * Intel Core Processor (Broadwell) CPUCache: 16384 KB Keys: 32 bytes each (+ 0 bytes user-defined timestamp) Values: 512 bytes each (256 bytes after compression) Entries: 5000000 Prefix: 0 bytes Keys per prefix: 0 RawSize: 2594.0 MB (estimated) FileSize: 1373.3 MB (estimated) Write rate: 0 bytes/second Read rate: 0 ops/second Compression: Snappy Compression sampling rate: 0 Memtablerep: SkipListFactory Perf Level: 1 WARNING: Assertions are enabled; benchmarks unnecessarily slow ------------------------------------------------ DB path: [/tmp/prefix_scan] seekrandom : 5356367.174 micros/op 0 ops/sec; 29.4 MB/s (23 of 23 found) ---------------------------------------------------------------------------------------------------- After the patch: ./db_bench -use_existing_db=true -db=/tmp/prefix_scan -benchmarks="seekrandom" -key_size=32 -value_size=512 -num=5000000 -use_direct_reads=true -seek_nexts=327680 -duration=120 -ops_between_duration_checks=1 Initializing RocksDB Options from the specified file Initializing RocksDB Options from command-line flags RocksDB: version 6.27 Date: Tue Nov 30 14:38:33 2021 CPU: 24 * Intel Core Processor (Broadwell) CPUCache: 16384 KB Keys: 32 bytes each (+ 0 bytes user-defined timestamp) Values: 512 bytes each (256 bytes after compression) Entries: 5000000 Prefix: 0 bytes Keys per prefix: 0 RawSize: 2594.0 MB (estimated) FileSize: 1373.3 MB (estimated) Write rate: 0 bytes/second Read rate: 0 ops/second Compression: Snappy Compression sampling rate: 0 Memtablerep: SkipListFactory Perf Level: 1 WARNING: Assertions are enabled; benchmarks unnecessarily slow ------------------------------------------------ DB path: [/tmp/prefix_scan] seekrandom : 456504.277 micros/op 2 ops/sec; 359.8 MB/s (264 of 264 found) Pull Request resolved: https://github.com/facebook/rocksdb/pull/9234 Test Plan: Ran ./db_bench -db=/data/mysql/rocksdb/prefix_scan -benchmarks="fillseq" -key_size=32 -value_size=512 -num=5000000 -use_d irect_io_for_flush_and_compaction=true -target_file_size_base=16777216 and then ./db_bench -use_existing_db=true -db=/data/mysql/rocksdb/prefix_scan -benchmarks="seekrandom" -key_size=32 -value_siz e=512 -num=5000000 -use_direct_reads=true -seek_nexts=327680 -duration=120 -ops_between_duration_checks=1 and compared the results. Reviewed By: anand1976 Differential Revision: D32743965 Pulled By: akankshamahajan15 fbshipit-source-id: b950fba68c91963b7deb5c20acdf471bc60251f5		3 years ago
..
adaptive	Refactor: use TableBuilderOptions to reduce parameter lists (#8240)	4 years ago
block_based	Fix bug in rocksdb internal automatic prefetching (#9234)	3 years ago
cuckoo	Improve / clean up meta block code & integrity (#9163)	3 years ago
plain	Improve / clean up meta block code & integrity (#9163)	3 years ago
block_fetcher.cc	Support readahead during compaction for blob files (#9187)	3 years ago
block_fetcher.h	Improve / clean up meta block code & integrity (#9163)	3 years ago
block_fetcher_test.cc	Cleanup includes in dbformat.h (#8930)	3 years ago
cleanable_test.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	5 years ago
format.cc	Support readahead during compaction for blob files (#9187)	3 years ago
format.h	Improve / clean up meta block code & integrity (#9163)	3 years ago
get_context.cc	Support readahead during compaction for blob files (#9187)	3 years ago
get_context.h	Cleanup includes in dbformat.h (#8930)	3 years ago
internal_iterator.h	Reuse internal auto readhead_size at each Level (expect L0) for Iterations (#9056)	3 years ago
iter_heap.h	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	5 years ago
iterator.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	5 years ago
iterator_wrapper.h	Reuse internal auto readhead_size at each Level (expect L0) for Iterations (#9056)	3 years ago
merger_test.cc	Cleanup multiple implementations of VectorIterator (#8901)	3 years ago
merging_iterator.cc	MergingIterator: rearrange fields to reduce paddings (#9024)	3 years ago
merging_iterator.h	Cleanup includes in dbformat.h (#8930)	3 years ago
meta_blocks.cc	Fix backward compatibility with 2.5 through 2.7 (#9189)	3 years ago
meta_blocks.h	Improve / clean up meta block code & integrity (#9163)	3 years ago
mock_table.cc	Introduce a mechanism to dump out blocks from block cache and re-insert to secondary cache (#8912)	3 years ago
mock_table.h	Fix some minor issues in the Customizable infrastructure (#8566)	3 years ago
multiget_context.h	Batch blob read IO for MultiGet (#8699)	3 years ago
persistent_cache_helper.cc	Improve / clean up meta block code & integrity (#9163)	3 years ago
persistent_cache_helper.h	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	5 years ago
persistent_cache_options.h	Improve / clean up meta block code & integrity (#9163)	3 years ago
scoped_arena_iterator.h	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	5 years ago
sst_file_dumper.cc	Support readahead during compaction for blob files (#9187)	3 years ago
sst_file_dumper.h	Make ImmutableOptions struct that inherits from ImmutableCFOptions and ImmutableDBOptions (#8262)	4 years ago
sst_file_reader.cc	Make ImmutableOptions struct that inherits from ImmutableCFOptions and ImmutableDBOptions (#8262)	4 years ago
sst_file_reader_test.cc	Enable a few unit tests to use custom Env objects (#9087)	3 years ago
sst_file_writer.cc	Support timestamps in SstFileWriter (#8899)	3 years ago
sst_file_writer_collectors.h	Cleanup includes in dbformat.h (#8930)	3 years ago
table_builder.h	Partially revert the "apply subrange of table property collectors" change (#8465)	3 years ago
table_factory.cc	Add customizable_util.h to the public API (#8301)	3 years ago
table_properties.cc	Improve / clean up meta block code & integrity (#9163)	3 years ago
table_properties_internal.h	Improve / clean up meta block code & integrity (#9163)	3 years ago
table_reader.h	dedup ReadOptions in iterator hierarchy (#7210)	4 years ago
table_reader_bench.cc	Make it possible to apply only a subrange of table property collectors (#8298)	4 years ago
table_reader_caller.h	Fix and detect headers with missing dependencies (#8893)	3 years ago
table_test.cc	Support readahead during compaction for blob files (#9187)	3 years ago
two_level_iterator.cc	Clarify caching behavior for index and filter partitions (#9068)	3 years ago
two_level_iterator.h	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	5 years ago
unique_id.cc	Experimental support for SST unique IDs (#8990)	3 years ago
unique_id_impl.h	Experimental support for SST unique IDs (#8990)	3 years ago