fork of https://github.com/oxigraph/rocksdb and https://github.com/facebook/rocksdb for nextgraph and oxigraph

You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

Changyu Bi 7a95938899 Improve FragmentTombstones() speed by lazily initializing `seq_set_` (#10848) ... Summary: FragmentedRangeTombstoneList has a member variable `seq_set_` that contains the sequence numbers of all range tombstones in a set. The set is constructed in `FragmentTombstones()` and is used only in `FragmentedRangeTombstoneList::ContainsRange()` which only happens during compaction. This PR moves the initialization of `seq_set_` to `FragmentedRangeTombstoneList::ContainsRange()`. This should speed up `FragmentTombstones()` when the range tombstone list is used for read/scan requests. Microbench shows the speed improvement to be ~45%. Pull Request resolved: https://github.com/facebook/rocksdb/pull/10848 Test Plan: - Existing tests and stress test: `python3 tools/db_crashtest.py whitebox --simple --verify_iterator_with_expected_state_one_in=5`. - Microbench: update `range_del_aggregator_bench` to benchmark speed of `FragmentTombstones()`: ``` ./range_del_aggregator_bench --num_range_tombstones=1000 --tombstone_start_upper_bound=50000000 --num_runs=10000 --tombstone_width_mean=200 --should_deletes_per_run=100 --use_compaction_range_del_aggregator=true Before this PR: ========================= Fragment Tombstones: 270.286 us AddTombstones: 1.28933 us ShouldDelete (first): 0.525528 us ShouldDelete (rest): 0.0797519 us After this PR: time to fragment tombstones is pushed to AddTombstones() which only happen during compaction. ========================= Fragment Tombstones: 149.879 us AddTombstones: 102.131 us ShouldDelete (first): 0.565871 us ShouldDelete (rest): 0.0729444 us ``` - db_bench: this should improve speed for fragmenting range tombstones for mutable memtable: ``` ./db_bench --benchmarks=readwhilewriting --writes_per_range_tombstone=100 --max_write_buffer_number=100 --min_write_buffer_number_to_merge=100 --writes=500000 --reads=250000 --disable_auto_compactions --max_num_range_tombstones=100000 --finish_after_writes --write_buffer_size=1073741824 --threads=25 Before this PR: readwhilewriting : 18.301 micros/op 1310445 ops/sec 4.769 seconds 6250000 operations; 28.1 MB/s (41001 of 250000 found) After this PR: readwhilewriting : 16.943 micros/op 1439376 ops/sec 4.342 seconds 6250000 operations; 23.8 MB/s (28977 of 250000 found) ``` Reviewed By: ajkr Differential Revision: D40646227 Pulled By: cbi42 fbshipit-source-id: ea471667edb258f67d01cfd828588e80a89e4083		2 years ago
..
adaptive	More refactoring ahead of footer & meta changes (#9240)	3 years ago
block_based	Improve FragmentTombstones() speed by lazily initializing `seq_set_` (#10848)	2 years ago
cuckoo	Print stack traces on frozen tests in CI (#10828)	2 years ago
plain	Fix platform 10 build with folly (#10708)	2 years ago
block_fetcher.cc	Refactor to avoid confusing "raw block" (#10408)	2 years ago
block_fetcher.h	Refactor to avoid confusing "raw block" (#10408)	2 years ago
block_fetcher_test.cc	Make MemoryAllocator into a Customizable class (#8980)	3 years ago
cleanable_test.cc	Eliminate unnecessary (slow) block cache Ref()ing in MultiGet (#9899)	3 years ago
format.cc	Refactor to avoid confusing "raw block" (#10408)	2 years ago
format.h	Refactor to avoid confusing "raw block" (#10408)	2 years ago
get_context.cc	User-defined timestamp support for `DeleteRange()` (#10661)	2 years ago
get_context.h	User-defined timestamp support for `DeleteRange()` (#10661)	2 years ago
internal_iterator.h	Skip swaths of range tombstone covered keys in merging iterator (2022 edition) (#10449)	2 years ago
iter_heap.h	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	5 years ago
iterator.cc	Eliminate unnecessary (slow) block cache Ref()ing in MultiGet (#9899)	3 years ago
iterator_wrapper.h	Skip swaths of range tombstone covered keys in merging iterator (2022 edition) (#10449)	2 years ago
merger_test.cc	Print stack traces on frozen tests in CI (#10828)	2 years ago
merging_iterator.cc	User-defined timestamp support for `DeleteRange()` (#10661)	2 years ago
merging_iterator.h	Fix memtable-only iterator regression (#10705)	2 years ago
meta_blocks.cc	Refactor to avoid confusing "raw block" (#10408)	2 years ago
meta_blocks.h	Refactor to avoid confusing "raw block" (#10408)	2 years ago
mock_table.cc	Align compaction output file boundaries to the next level ones (#10655)	2 years ago
mock_table.h	Align compaction output file boundaries to the next level ones (#10655)	2 years ago
multiget_context.h	MultiGet async IO across multiple levels (#10535)	2 years ago
persistent_cache_helper.cc	Refactor to avoid confusing "raw block" (#10408)	2 years ago
persistent_cache_helper.h	Refactor to avoid confusing "raw block" (#10408)	2 years ago
persistent_cache_options.h	Use STATIC_AVOID_DESTRUCTION for static objects with non-trivial destructors (#9958)	3 years ago
scoped_arena_iterator.h	Remove code that only compiles for Visual Studio versions older than 2015 (#10065)	2 years ago
sst_file_dumper.cc	Remove customized naming from InternalKeyComparator (#10343)	2 years ago
sst_file_dumper.h	Support using ZDICT_finalizeDictionary to generate zstd dictionary (#9857)	2 years ago
sst_file_reader.cc	Remove range tombstone test code from sst_file_reader (#10847)	2 years ago
sst_file_reader_test.cc	Remove range tombstone test code from sst_file_reader (#10847)	2 years ago
sst_file_writer.cc	User-defined timestamp support for `DeleteRange()` (#10661)	2 years ago
sst_file_writer_collectors.h	Refactor to avoid confusing "raw block" (#10408)	2 years ago
table_builder.h	Always verify SST unique IDs on SST file open (#10532)	2 years ago
table_factory.cc	Restore Regex support for ObjectLibrary::Register, rename new APIs to allow old one to be deprecated in the future (#9362)	3 years ago
table_properties.cc	Add seqno to time mapping (#10338)	2 years ago
table_properties_internal.h	Improve / clean up meta block code & integrity (#9163)	3 years ago
table_reader.h	Refactor block cache tracing APIs (#10811)	2 years ago
table_reader_bench.cc	Add support for wide-column point lookups (#10540)	2 years ago
table_test.cc	Use kXXH3 as default checksum (CPU efficiency) (#10778)	2 years ago
two_level_iterator.cc	Clarify caching behavior for index and filter partitions (#9068)	3 years ago
two_level_iterator.h	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	5 years ago
unique_id.cc	Derive cache keys from SST unique IDs (#10394)	2 years ago
unique_id_impl.h	Derive cache keys from SST unique IDs (#10394)	2 years ago