From 097f9f4425d78cd11ff1912a6ac5f8a2edd3f4e0 Mon Sep 17 00:00:00 2001 From: Andrew Kryczka Date: Mon, 21 Nov 2022 16:14:03 -0800 Subject: [PATCH] Fix CompactionIterator flag for penultimate level output (#10967) Summary: We were not resetting it in non-debug mode so it could be true once and then stay true for future keys where it should be false. This PR adds the reset logic. Pull Request resolved: https://github.com/facebook/rocksdb/pull/10967 Test Plan: - built `db_bench` with DEBUG_LEVEL=0 - ran benchmark: `TEST_TMPDIR=/dev/shm/prefix ./db_bench -benchmarks=fillrandom -compaction_style=1 -preserve_internal_time_seconds=100 -preclude_last_level_data_seconds=10 -write_buffer_size=1048576 -target_file_size_base=1048576 -subcompactions=8 -duration=120` - compared "output_to_penultimate_level: X bytes + last: Y bytes" lines in LOG output - Before this fix, Y was always zero - After this fix, Y gradually increased throughout the benchmark Reviewed By: riversand963 Differential Revision: D41417726 Pulled By: ajkr fbshipit-source-id: ace1e9a289e751a5b0c2fbaa8addd4eda5525329 --- HISTORY.md | 1 + db/compaction/compaction_iterator.cc | 4 +++- 2 files changed, 4 insertions(+), 1 deletion(-) diff --git a/HISTORY.md b/HISTORY.md index dc4b320a6..97fa172d5 100644 --- a/HISTORY.md +++ b/HISTORY.md @@ -10,6 +10,7 @@ * Fixed an issue where the `READ_NUM_MERGE_OPERANDS` ticker was not updated when the base key-value or tombstone was read from an SST file. * Fixed a memory safety bug when using a SecondaryCache with `block_cache_compressed`. `block_cache_compressed` no longer attempts to use SecondaryCache features. * Fixed a regression in scan for async_io. During seek, valid buffers were getting cleared causing a regression. +* Tiered Storage: fixed excessive keys written to penultimate level in non-debug builds. ### New Features * Add basic support for user-defined timestamp to Merge (#10819). diff --git a/db/compaction/compaction_iterator.cc b/db/compaction/compaction_iterator.cc index 788c8876c..9f54f7813 100644 --- a/db/compaction/compaction_iterator.cc +++ b/db/compaction/compaction_iterator.cc @@ -1105,7 +1105,9 @@ void CompactionIterator::DecideOutputLevel() { TEST_SYNC_POINT_CALLBACK("CompactionIterator::PrepareOutput.context", &context); output_to_penultimate_level_ = context.output_to_penultimate_level; -#endif /* !NDEBUG */ +#else + output_to_penultimate_level_ = false; +#endif // NDEBUG // if the key is newer than the cutoff sequence or within the earliest // snapshot, it should output to the penultimate level.