Extend trace filtering to more operation types (#9335)

Summary:
- Extended trace filtering to cover `MultiGet()`, `Seek()`, and `SeekForPrev()`. Now all user ops that can be traced support filtering.
- Enabled the new filter masks in `db_stress` since it only cares to trace writes.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/9335

Test Plan:
- trace-heavy `db_stress` command reduced 30% elapsed time  (79.21 -> 55.47 seconds)

Benchmark command:
```
$ /usr/bin/time ./db_stress -ops_per_thread=100000 -sync_fault_injection=1 --db=/dev/shm/rocksdb_stress_db/ --expected_values_dir=/dev/shm/rocksdb_stress_expected/ --clear_column_family_one_in=0
```

- replay-heavy `db_stress` command reduced 12.4% elapsed time (23.69 -> 20.75 seconds)

Setup command:
```
$  ./db_stress -ops_per_thread=100000000 -sync_fault_injection=1 -db=/dev/shm/rocksdb_stress_db/ -expected_values_dir=/dev/shm/rocksdb_stress_expected --clear_column_family_one_in=0 & sleep 120; pkill -9 db_stress
```

Benchmark command:
```
$ /usr/bin/time ./db_stress -ops_per_thread=1 -reopen=0 -expected_values_dir=/dev/shm/rocksdb_stress_expected/ -db=/dev/shm/rocksdb_stress_db/ --clear_column_family_one_in=0 --destroy_db_initially=0
```

Reviewed By: zhichao-cao

Differential Revision: D33304580

Pulled By: ajkr

fbshipit-source-id: 0df10f87c1fc506e9484b6b42cea2ef96c7ecd65
main
Andrew Kryczka 3 years ago committed by Facebook GitHub Bot
parent 2e5f764294
commit 2ee20a669d
  1. 4
      HISTORY.md
  2. 3
      db_stress_tool/expected_state.cc
  3. 8
      include/rocksdb/options.h
  4. 41
      trace_replay/trace_replay.cc

@ -1,4 +1,8 @@
# Rocksdb Change Log # Rocksdb Change Log
## Unreleased
### Public API change
* Added values to `TraceFilterType`: `kTraceFilterIteratorSeek`, `kTraceFilterIteratorSeekForPrev`, and `kTraceFilterMultiGet`. They can be set in `TraceOptions` to filter out the operation types after which they are named.
## 6.28.0 (2021-12-17) ## 6.28.0 (2021-12-17)
### New Features ### New Features
* Introduced 'CommitWithTimestamp' as a new tag. Currently, there is no API for user to trigger a write with this tag to the WAL. This is part of the efforts to support write-commited transactions with user-defined timestamps. * Introduced 'CommitWithTimestamp' as a new tag. Currently, there is no API for user to trigger a write with this tag to the WAL. This is part of the efforts to support write-commited transactions with user-defined timestamps.

@ -298,6 +298,9 @@ Status FileExpectedStateManager::SaveAtAndAfter(DB* db) {
if (s.ok()) { if (s.ok()) {
TraceOptions trace_opts; TraceOptions trace_opts;
trace_opts.filter |= kTraceFilterGet; trace_opts.filter |= kTraceFilterGet;
trace_opts.filter |= kTraceFilterMultiGet;
trace_opts.filter |= kTraceFilterIteratorSeek;
trace_opts.filter |= kTraceFilterIteratorSeekForPrev;
s = db->StartTrace(trace_opts, std::move(trace_writer)); s = db->StartTrace(trace_opts, std::move(trace_writer));
} }

@ -1851,7 +1851,13 @@ enum TraceFilterType : uint64_t {
// Do not trace the get operations // Do not trace the get operations
kTraceFilterGet = 0x1 << 0, kTraceFilterGet = 0x1 << 0,
// Do not trace the write operations // Do not trace the write operations
kTraceFilterWrite = 0x1 << 1 kTraceFilterWrite = 0x1 << 1,
// Do not trace the `Iterator::Seek()` operations
kTraceFilterIteratorSeek = 0x1 << 2,
// Do not trace the `Iterator::SeekForPrev()` operations
kTraceFilterIteratorSeekForPrev = 0x1 << 3,
// Do not trace the `MultiGet()` operations
kTraceFilterMultiGet = 0x1 << 4,
}; };
// TraceOptions is used for StartTrace // TraceOptions is used for StartTrace

@ -532,11 +532,46 @@ bool Tracer::ShouldSkipTrace(const TraceType& trace_type) {
if (IsTraceFileOverMax()) { if (IsTraceFileOverMax()) {
return true; return true;
} }
if ((trace_options_.filter & kTraceFilterGet && trace_type == kTraceGet) ||
(trace_options_.filter & kTraceFilterWrite && TraceFilterType filter_mask = kTraceFilterNone;
trace_type == kTraceWrite)) { switch (trace_type) {
case kTraceNone:
case kTraceBegin:
case kTraceEnd:
filter_mask = kTraceFilterNone;
break;
case kTraceWrite:
filter_mask = kTraceFilterWrite;
break;
case kTraceGet:
filter_mask = kTraceFilterGet;
break;
case kTraceIteratorSeek:
filter_mask = kTraceFilterIteratorSeek;
break;
case kTraceIteratorSeekForPrev:
filter_mask = kTraceFilterIteratorSeekForPrev;
break;
case kBlockTraceIndexBlock:
case kBlockTraceFilterBlock:
case kBlockTraceDataBlock:
case kBlockTraceUncompressionDictBlock:
case kBlockTraceRangeDeletionBlock:
case kIOTracer:
filter_mask = kTraceFilterNone;
break;
case kTraceMultiGet:
filter_mask = kTraceFilterMultiGet;
break;
case kTraceMax:
assert(false);
filter_mask = kTraceFilterNone;
break;
}
if (filter_mask != kTraceFilterNone && trace_options_.filter & filter_mask) {
return true; return true;
} }
++trace_request_count_; ++trace_request_count_;
if (trace_request_count_ < trace_options_.sampling_frequency) { if (trace_request_count_ < trace_options_.sampling_frequency) {
return true; return true;

Loading…
Cancel
Save