fork of https://github.com/oxigraph/rocksdb and https://github.com/facebook/rocksdb for nextgraph and oxigraph

You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

Changyu Bi e95cc1217d `CompactRange()` always compacts to bottommost level for leveled compaction (#11468) ... Summary: currently for leveled compaction, the max output level of a call to `CompactRange()` is pre-computed before compacting each level. This max output level is the max level whose key range overlaps with the manual compaction key range. However, during manual compaction, files in the max output level may be compacted down further by some background compaction. When this background compaction is a trivial move, there is a race condition and the manual compaction may not be able to compact all keys in the specified key range. This PR updates `CompactRange()` to always compact to the bottommost level to make this race condition more unlikely (it can still happen, see more in comment here: 796f58f42a/db/db_impl/db_impl_compaction_flush.cc (L1180C29-L1184)). This PR also changes the behavior of CompactRange() when `bottommost_level_compaction=kIfHaveCompactionFilter` (the default option). The old behavior is that, if a compaction filter is provided, CompactRange() always does an intra-level compaction at the final output level for all files in the manual compaction key range. The only exception when `first_overlapped_level = 0` and `max_overlapped_level = 0`. It’s awkward to maintain the same behavior after this PR since we do not compute max_overlapped_level anymore. So the new behavior is similar to kForceOptimized: always does intra-level compaction at the bottommost level, but not including new files generated during this manual compaction. Several unit tests are updated to work with this new manual compaction behavior. Pull Request resolved: https://github.com/facebook/rocksdb/pull/11468 Test Plan: Add new unit tests `DBCompactionTest.ManualCompactionCompactAllKeysInRange*` Reviewed By: ajkr Differential Revision: D46079619 Pulled By: cbi42 fbshipit-source-id: 19d844ba4ec8dc1a0b8af5d2f36ff15820c6e76f		1 year ago
..
compacted_db_impl.cc	Group rocksdb.sst.read.micros stat by IOActivity flush and compaction (#11288)	2 years ago
compacted_db_impl.h	Support Clip DB to KeyRange (#11379)	2 years ago
db_impl.cc	Support Clip DB to KeyRange (#11379)	2 years ago
db_impl.h	Flush option in WaitForCompact() (#11483)	1 year ago
db_impl_compaction_flush.cc	`CompactRange()` always compacts to bottommost level for leveled compaction (#11468)	1 year ago
db_impl_debug.cc	Add WaitForCompact with WaitForCompactOptions to public API (#11436)	1 year ago
db_impl_experimental.cc	Record and use the tail size to prefetch table tail (#11406)	2 years ago
db_impl_files.cc	Fix flaky test `DBTestUniversalManualCompactionOutputPathId.ManualCompactionOutputPathId` (#11412)	2 years ago
db_impl_open.cc	Logging timestamp size record in WAL and use it during recovery (#11471)	1 year ago
db_impl_readonly.cc	Group rocksdb.sst.read.micros stat by IOActivity flush and compaction (#11288)	2 years ago
db_impl_readonly.h	Support Clip DB to KeyRange (#11379)	2 years ago
db_impl_secondary.cc	Logging timestamp size record in WAL and use it during recovery (#11471)	1 year ago
db_impl_secondary.h	Add utils to use for handling user defined timestamp size record in WAL (#11451)	2 years ago
db_impl_write.cc	Logging timestamp size record in WAL and use it during recovery (#11471)	1 year ago