Deflake DBBloomFilterTest.OptimizeFiltersForHits (#11383)

Summary:
In CircleCI build-linux-arm-test-full job (https://app.circleci.com/pipelines/github/facebook/rocksdb/26462/workflows/a9d39d2c-c970-4b0f-9c10-7743beb9771b/jobs/591722), this test exhibited the following flaky failure:

```
db/db_bloom_filter_test.cc:2506: Failure
Expected: (TestGetTickerCount(options, BLOOM_FILTER_USEFUL)) > (65000 * 2), actual: 120558 vs 130000
```

I ssh'd to an instance and observed it cuts memtables at slightly different points across runs. Logging in `ConcurrentArena` pointed to `try_lock()` returning false at different points across runs.

This PR changes the approach to allow a fixed number of keys per memtable flush. I verified the bloom filter useful count is deterministic now even on the CircleCI ARM instance.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/11383

Reviewed By: cbi42

Differential Revision: D45036829

Pulled By: ajkr

fbshipit-source-id: b602dacb63955f1af09bf0ed409cde0552805a08
oxigraph-8.3.2
Andrew Kryczka 2 years ago committed by Facebook GitHub Bot
parent 226ee25d30
commit b8555ba470
  1. 15
      db/db_bloom_filter_test.cc

@ -2437,9 +2437,11 @@ TEST_F(DBBloomFilterTest, PrefixScan) {
}
TEST_F(DBBloomFilterTest, OptimizeFiltersForHits) {
const int kNumKeysPerFlush = 1000;
Options options = CurrentOptions();
options.write_buffer_size = 64 * 1024;
options.arena_block_size = 4 * 1024;
options.memtable_factory.reset(
test::NewSpecialSkipListFactory(kNumKeysPerFlush));
options.target_file_size_base = 64 * 1024;
options.level0_file_num_compaction_trigger = 2;
options.level0_slowdown_writes_trigger = 2;
@ -2475,8 +2477,13 @@ TEST_F(DBBloomFilterTest, OptimizeFiltersForHits) {
int num_inserted = 0;
for (int key : keys) {
ASSERT_OK(Put(1, Key(key), "val"));
if (++num_inserted % 1000 == 0) {
ASSERT_OK(dbfull()->TEST_WaitForFlushMemTable());
num_inserted++;
// The write after each `kNumKeysPerFlush` keys triggers a flush. Always
// wait for that flush and any follow-on compactions for deterministic LSM
// shape.
if (num_inserted > kNumKeysPerFlush &&
num_inserted % kNumKeysPerFlush == 1) {
ASSERT_OK(dbfull()->TEST_WaitForFlushMemTable(handles_[1]));
ASSERT_OK(dbfull()->TEST_WaitForCompact());
}
}

Loading…
Cancel
Save