Ignore non-overlapping levels when determinig grandparent files (#9051)

Summary:
Right now, when picking a compaction, grand parent files are from output_level + 1. This usually works, but if the level doesn't have any overlapping file, it will be more efficient to go further down. This is because the files are likely to be trivial moved further and might create a violation of max_compaction_bytes. This situation can naturally happen and might happen even more with TTL compactions. There is no harm to fix it.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/9051

Test Plan: Run existing tests and see it passes. Also briefly run crash test.

Reviewed By: ajkr

Differential Revision: D31748829

fbshipit-source-id: 52b99ab4284dc816d22f34406d528a3c98ff6719
main
sdong 3 years ago committed by Facebook GitHub Bot
parent b234a3f569
commit f053851af6
  1. 12
      db/compaction/compaction_picker.cc

@ -554,10 +554,14 @@ void CompactionPicker::GetGrandparents(
InternalKey start, limit;
GetRange(inputs, output_level_inputs, &start, &limit);
// Compute the set of grandparent files that overlap this compaction
// (parent == level+1; grandparent == level+2)
if (output_level_inputs.level + 1 < NumberLevels()) {
vstorage->GetOverlappingInputs(output_level_inputs.level + 1, &start,
&limit, grandparents);
// (parent == level+1; grandparent == level+2 or the first
// level after that has overlapping files)
for (int level = output_level_inputs.level + 1; level < NumberLevels();
level++) {
vstorage->GetOverlappingInputs(level, &start, &limit, grandparents);
if (!grandparents->empty()) {
break;
}
}
}

Loading…
Cancel
Save