fork of https://github.com/oxigraph/rocksdb and https://github.com/facebook/rocksdb for nextgraph and oxigraph

You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

Hui Xiao 1da9ac2363 Add UT to test BG read qps behavior during upgrade for pr11406 (#11522) ... Summary: **Context/Summary:** When db is upgrading to adopt [pr11406](https://github.com/facebook/rocksdb/pull/11406/), it's possible for RocksDB to infer a small tail size to prefetch for pre-upgrade files. Such small tail size would have caused 1 file read per index or filter partition if partitioned index or filer is used. This PR provides a UT to show this would not happen. Misc: refactor the related UTs a bit to make this new UT more readable. Pull Request resolved: https://github.com/facebook/rocksdb/pull/11522 Test Plan: - New UT If logic of upgrade is wrong e.g, ``` --- a/table/block_based/partitioned_index_reader.cc +++ b/table/block_based/partitioned_index_reader.cc @@ -166,7 +166,8 @@ Status PartitionIndexReader::CacheDependencies( uint64_t prefetch_len = last_off - prefetch_off; std::unique_ptr<FilePrefetchBuffer> prefetch_buffer; if (tail_prefetch_buffer == nullptr || !tail_prefetch_buffer->Enabled() || - tail_prefetch_buffer->GetPrefetchOffset() > prefetch_off) { + (false && tail_prefetch_buffer->GetPrefetchOffset() > prefetch_off)) { ``` , then the UT will fail like below ``` [ RUN ] PrefetchTailTest/PrefetchTailTest.UpgradeToTailSizeInManifest/0 file/prefetch_test.cc:461: Failure Expected: (db_open_file_read.count) < (num_index_partition), actual: 38 vs 33 Received signal 11 (Segmentation fault) ``` Reviewed By: pdillinger Differential Revision: D46546707 Pulled By: hx235 fbshipit-source-id: 9897b0a975e9055963edac5451fd1cd9d6c45d0e		1 year ago
..
adaptive	Remove RocksDB LITE (#11147)	2 years ago
block_based	Add UT to test BG read qps behavior during upgrade for pr11406 (#11522)	1 year ago
cuckoo	Group rocksdb.sst.read.micros stat by IOActivity flush and compaction (#11288)	2 years ago
plain	Group rocksdb.sst.read.micros stat by IOActivity flush and compaction (#11288)	2 years ago
block_fetcher.cc	Change internal headers with duplicate names (#11408)	2 years ago
block_fetcher.h	Change internal headers with duplicate names (#11408)	2 years ago
block_fetcher_test.cc	Record and use the tail size to prefetch table tail (#11406)	2 years ago
cleanable_test.cc	Eliminate unnecessary (slow) block cache Ref()ing in MultiGet (#9899)	3 years ago
compaction_merging_iterator.cc	Refactor AddRangeDels() + consider range tombstone during compaction file cutting (#11113)	2 years ago
compaction_merging_iterator.h	Refactor AddRangeDels() + consider range tombstone during compaction file cutting (#11113)	2 years ago
format.cc	Change internal headers with duplicate names (#11408)	2 years ago
format.h	Change internal headers with duplicate names (#11408)	2 years ago
get_context.cc	Change internal headers with duplicate names (#11408)	2 years ago
get_context.h	Merge operator failed subcode (#11231)	2 years ago
internal_iterator.h	remove unused InternalIteratorBase::is_mutable_ (#11104)	2 years ago
iter_heap.h	Format files under table/ by clang-format (#10852)	2 years ago
iterator.cc	Format files under table/ by clang-format (#10852)	2 years ago
iterator_wrapper.h	Format files under table/ by clang-format (#10852)	2 years ago
merger_test.cc	Print stack traces on frozen tests in CI (#10828)	2 years ago
merging_iterator.cc	Improve documentation for MergingIterator (#11161)	2 years ago
merging_iterator.h	Improve documentation for MergingIterator (#11161)	2 years ago
meta_blocks.cc	Record and use the tail size to prefetch table tail (#11406)	2 years ago
meta_blocks.h	Group rocksdb.sst.read.micros stat by IOActivity flush and compaction (#11288)	2 years ago
mock_table.cc	Group rocksdb.sst.read.micros stat by IOActivity flush and compaction (#11288)	2 years ago
mock_table.h	Align compaction output file boundaries to the next level ones (#10655)	2 years ago
multiget_context.h	Add a new MultiGetEntity API (#11222)	2 years ago
persistent_cache_helper.cc	Format files under table/ by clang-format (#10852)	2 years ago
persistent_cache_helper.h	Change internal headers with duplicate names (#11408)	2 years ago
persistent_cache_options.h	Change internal headers with duplicate names (#11408)	2 years ago
scoped_arena_iterator.h	Format files under table/ by clang-format (#10852)	2 years ago
sst_file_dumper.cc	Block per key-value checksum (#11287)	2 years ago
sst_file_dumper.h	Remove RocksDB LITE (#11147)	2 years ago
sst_file_reader.cc	Block per key-value checksum (#11287)	2 years ago
sst_file_reader_test.cc	Remove RocksDB LITE (#11147)	2 years ago
sst_file_writer.cc	validate SstFileWriter range tombstones cover positive ranges (#11322)	2 years ago
sst_file_writer_collectors.h	Refactor to avoid confusing "raw block" (#10408)	2 years ago
table_builder.h	Add support to strip / pad timestamp when creating / reading a block based table (#11495)	1 year ago
table_factory.cc	Remove FactoryFunc from LoadXXXObject (#11203)	2 years ago
table_properties.cc	Record and use the tail size to prefetch table tail (#11406)	2 years ago
table_properties_internal.h	Improve / clean up meta block code & integrity (#9163)	3 years ago
table_reader.h	Group rocksdb.sst.read.micros stat by IOActivity flush and compaction (#11288)	2 years ago
table_reader_bench.cc	Block per key-value checksum (#11287)	2 years ago
table_test.cc	Change internal headers with duplicate names (#11408)	2 years ago
two_level_iterator.cc	Format files under table/ by clang-format (#10852)	2 years ago
two_level_iterator.h	Format files under table/ by clang-format (#10852)	2 years ago
unique_id.cc	Derive cache keys from SST unique IDs (#10394)	2 years ago
unique_id_impl.h	Derive cache keys from SST unique IDs (#10394)	2 years ago