fork of https://github.com/oxigraph/rocksdb and https://github.com/facebook/rocksdb for nextgraph and oxigraph

You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

Akanksha Mahajan 04b2c16f9b Fix bug in rocksdb internal automatic prefetching (#9234) ... Summary: After introducing adaptive_readahead, the original flow got broken. Readahead size was set to 0 because of which rocksdb wasn't be able to do automatic prefetching which it enables after seeing sequential reads. This PR fixes it. ---------------------------------------------------------------------------------------------------- Before this patch: b_bench -use_existing_db=true -db=/tmp/prefix_scan -benchmarks="seekrandom" -key_size=32 -value_size=512 -num=5000000 -use_direct_reads=true -seek_nexts=327680 -duration=120 -ops_between_duration_checks=1 Initializing RocksDB Options from the specified file Initializing RocksDB Options from command-line flags RocksDB: version 6.27 Date: Tue Nov 30 11:56:50 2021 CPU: 24 * Intel Core Processor (Broadwell) CPUCache: 16384 KB Keys: 32 bytes each (+ 0 bytes user-defined timestamp) Values: 512 bytes each (256 bytes after compression) Entries: 5000000 Prefix: 0 bytes Keys per prefix: 0 RawSize: 2594.0 MB (estimated) FileSize: 1373.3 MB (estimated) Write rate: 0 bytes/second Read rate: 0 ops/second Compression: Snappy Compression sampling rate: 0 Memtablerep: SkipListFactory Perf Level: 1 WARNING: Assertions are enabled; benchmarks unnecessarily slow ------------------------------------------------ DB path: [/tmp/prefix_scan] seekrandom : 5356367.174 micros/op 0 ops/sec; 29.4 MB/s (23 of 23 found) ---------------------------------------------------------------------------------------------------- After the patch: ./db_bench -use_existing_db=true -db=/tmp/prefix_scan -benchmarks="seekrandom" -key_size=32 -value_size=512 -num=5000000 -use_direct_reads=true -seek_nexts=327680 -duration=120 -ops_between_duration_checks=1 Initializing RocksDB Options from the specified file Initializing RocksDB Options from command-line flags RocksDB: version 6.27 Date: Tue Nov 30 14:38:33 2021 CPU: 24 * Intel Core Processor (Broadwell) CPUCache: 16384 KB Keys: 32 bytes each (+ 0 bytes user-defined timestamp) Values: 512 bytes each (256 bytes after compression) Entries: 5000000 Prefix: 0 bytes Keys per prefix: 0 RawSize: 2594.0 MB (estimated) FileSize: 1373.3 MB (estimated) Write rate: 0 bytes/second Read rate: 0 ops/second Compression: Snappy Compression sampling rate: 0 Memtablerep: SkipListFactory Perf Level: 1 WARNING: Assertions are enabled; benchmarks unnecessarily slow ------------------------------------------------ DB path: [/tmp/prefix_scan] seekrandom : 456504.277 micros/op 2 ops/sec; 359.8 MB/s (264 of 264 found) Pull Request resolved: https://github.com/facebook/rocksdb/pull/9234 Test Plan: Ran ./db_bench -db=/data/mysql/rocksdb/prefix_scan -benchmarks="fillseq" -key_size=32 -value_size=512 -num=5000000 -use_d irect_io_for_flush_and_compaction=true -target_file_size_base=16777216 and then ./db_bench -use_existing_db=true -db=/data/mysql/rocksdb/prefix_scan -benchmarks="seekrandom" -key_size=32 -value_siz e=512 -num=5000000 -use_direct_reads=true -seek_nexts=327680 -duration=120 -ops_between_duration_checks=1 and compared the results. Reviewed By: anand1976 Differential Revision: D32743965 Pulled By: akankshamahajan15 fbshipit-source-id: b950fba68c91963b7deb5c20acdf471bc60251f5		3 years ago
..
binary_search_index_reader.cc	Separate internal and user key comparators in `BlockIter` (#6944)	5 years ago
binary_search_index_reader.h	Extend Get/MultiGet deadline support to table open (#6982)	5 years ago
block.cc	Support timestamps in SstFileWriter (#8899)	3 years ago
block.h	Cleanup includes in dbformat.h (#8930)	3 years ago
block_based_filter_block.cc	Deallocate payload of BlockBasedTableBuilder::Rep::FilterBlockBuilder earlier for Full/PartitionedFilter (#9070)	3 years ago
block_based_filter_block.h	Deallocate payload of BlockBasedTableBuilder::Rep::FilterBlockBuilder earlier for Full/PartitionedFilter (#9070)	3 years ago
block_based_filter_block_test.cc	Add table properties for number of entries added to filters (#8323)	4 years ago
block_based_table_builder.cc	Fix backward compatibility with 2.5 through 2.7 (#9189)	3 years ago
block_based_table_builder.h	Some checksum code refactoring (#9113)	3 years ago
block_based_table_factory.cc	Account Bloom/Ribbon filter construction memory in global memory limit (#9073)	3 years ago
block_based_table_factory.h	Cleanup includes in dbformat.h (#8930)	3 years ago
block_based_table_iterator.cc	Reuse internal auto readhead_size at each Level (expect L0) for Iterations (#9056)	3 years ago
block_based_table_iterator.h	Fix bug in rocksdb internal automatic prefetching (#9234)	3 years ago
block_based_table_reader.cc	Support readahead during compaction for blob files (#9187)	3 years ago
block_based_table_reader.h	Support readahead during compaction for blob files (#9187)	3 years ago
block_based_table_reader_impl.h	Parallelize secondary cache lookup in MultiGet (#8405)	4 years ago
block_based_table_reader_test.cc	Cleanup includes in dbformat.h (#8930)	3 years ago
block_builder.cc	Improve data block construction performance (#9040)	3 years ago
block_builder.h	Improve data block construction performance (#9040)	3 years ago
block_like_traits.h	fix lru caching test and fix reference binding to null pointer (#8326)	4 years ago
block_prefetcher.cc	Improve / clean up meta block code & integrity (#9163)	3 years ago
block_prefetcher.h	Fix bug in rocksdb internal automatic prefetching (#9234)	3 years ago
block_prefix_index.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	5 years ago
block_prefix_index.h	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	5 years ago
block_test.cc	Improve / clean up meta block code & integrity (#9163)	3 years ago
block_type.h	Fix and detect headers with missing dependencies (#8893)	3 years ago
cachable_entry.h	Parallelize secondary cache lookup in MultiGet (#8405)	4 years ago
data_block_footer.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	5 years ago
data_block_footer.h	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	5 years ago
data_block_hash_index.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	5 years ago
data_block_hash_index.h	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	5 years ago
data_block_hash_index_test.cc	Make it possible to apply only a subrange of table property collectors (#8298)	4 years ago
filter_block.h	Account Bloom/Ribbon filter construction memory in global memory limit (#9073)	3 years ago
filter_block_reader_common.cc	Parallelize secondary cache lookup in MultiGet (#8405)	4 years ago
filter_block_reader_common.h	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	5 years ago
filter_policy.cc	Improve / clean up meta block code & integrity (#9163)	3 years ago
filter_policy_internal.h	Add Bloom/Ribbon hybrid API support (#8679)	3 years ago
flush_block_policy.cc	Improve / clean up meta block code & integrity (#9163)	3 years ago
flush_block_policy.h	Make FlushBlockPolicyFactory into a Customizable class (#8432)	3 years ago
full_filter_block.cc	Deallocate payload of BlockBasedTableBuilder::Rep::FilterBlockBuilder earlier for Full/PartitionedFilter (#9070)	3 years ago
full_filter_block.h	Account Bloom/Ribbon filter construction memory in global memory limit (#9073)	3 years ago
full_filter_block_test.cc	Add table properties for number of entries added to filters (#8323)	4 years ago
hash_index_reader.cc	Make ImmutableOptions struct that inherits from ImmutableCFOptions and ImmutableDBOptions (#8262)	4 years ago
hash_index_reader.h	Extend Get/MultiGet deadline support to table open (#6982)	5 years ago
index_builder.cc	Add (& fix) some simple source code checks (#8821)	3 years ago
index_builder.h	Make db_basic_test pass assert status checked (#7452)	4 years ago
index_reader_common.cc	Parallelize secondary cache lookup in MultiGet (#8405)	4 years ago
index_reader_common.h	Divide block_based_table_reader.cc (#6527)	5 years ago
mock_block_based_table.h	Make ImmutableOptions struct that inherits from ImmutableCFOptions and ImmutableDBOptions (#8262)	4 years ago
parsed_full_filter_block.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	5 years ago
parsed_full_filter_block.h	Use new Insert and Lookup APIs in table reader to support secondary cache (#8315)	4 years ago
partitioned_filter_block.cc	Improve / clean up meta block code & integrity (#9163)	3 years ago
partitioned_filter_block.h	Deallocate payload of BlockBasedTableBuilder::Rep::FilterBlockBuilder earlier for Full/PartitionedFilter (#9070)	3 years ago
partitioned_filter_block_test.cc	Deallocate payload of BlockBasedTableBuilder::Rep::FilterBlockBuilder earlier for Full/PartitionedFilter (#9070)	3 years ago
partitioned_index_iterator.cc	Reuse internal auto readhead_size at each Level (expect L0) for Iterations (#9056)	3 years ago
partitioned_index_iterator.h	Fix bug in rocksdb internal automatic prefetching (#9234)	3 years ago
partitioned_index_reader.cc	Improve / clean up meta block code & integrity (#9163)	3 years ago
partitioned_index_reader.h	Clarify caching behavior for index and filter partitions (#9068)	3 years ago
reader_common.cc	Improve / clean up meta block code & integrity (#9163)	3 years ago
reader_common.h	Bring the Configurable options together (#5753)	4 years ago
uncompression_dict_reader.cc	Cleanup includes in dbformat.h (#8930)	3 years ago
uncompression_dict_reader.h	Extend Get/MultiGet deadline support to table open (#6982)	5 years ago