You can not select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
Mark Callaghan
5506954b1f
Enhance to support more tuning options, and universal and integrated… (#9704)
Summary:
… BlobDB for all tests
This does two big things:
* provides more tuning options
* supports universal and integrated BlobDB for all of the benchmarks that are leveled-only
It does several smaller things, and I will list a few
* sets l0_slowdown_writes_trigger which wasn't set before this diff.
* improves readability in report.tsv by using smaller field names in the header
* adds more columns to report.tsv
report.tsv before this diff:
```
ops_sec mb_sec total_size_gb level0_size_gb sum_gb write_amplification write_mbps usec_op percentile_50 percentile_75 percentile_99 percentile_99.9 percentile_99.99 uptime stall_time stall_percent test_name test_date rocksdb_version job_id
823294 329.8 0.0 21.5 21.5 1.0 183.4 1.2 1.0 1.0 3 6 14 120 00:00:0.000 0.0 fillseq.wal_disabled.v400 2022-03-16T15:46:45.000-07:00 7.0
326520 130.8 0.0 0.0 0.0 0.0 0 12.2 139.8 155.1 170 234 250 60 00:00:0.000 0.0 multireadrandom.t4 2022-03-16T15:48:47.000-07:00 7.0
86313 345.7 0.0 0.0 0.0 0.0 0 46.3 44.8 50.6 75 84 108 60 00:00:0.000 0.0 revrangewhilewriting.t4 2022-03-16T15:50:48.000-07:00 7.0
101294 405.7 0.0 0.1 0.1 1.0 1.6 39.5 40.4 45.9 64 75 103 62 00:00:0.000 0.0 fwdrangewhilewriting.t4 2022-03-16T15:52:50.000-07:00 7.0
258141 103.4 0.0 0.1 1.2 18.2 19.8 15.5 14.3 18.1 28 34 48 62 00:00:0.000 0.0 readwhilewriting.t4 2022-03-16T15:54:51.000-07:00 7.0
334690 134.1 0.0 7.6 18.7 4.2 308.8 12.0 11.8 13.7 21 30 62 62 00:00:0.000 0.0 overwrite.t4.s0 2022-03-16T15:56:53.000-07:00 7.0
```
report.tsv with this diff:
```
ops_sec mb_sec lsm_sz blob_sz c_wgb w_amp c_mbps c_wsecs c_csecs b_rgb b_wgb usec_op p50 p99 p99.9 p99.99 pmax uptime stall% Nstall u_cpu s_cpu rss test date version job_id
831144 332.9 22GB 0.0GB, 21.7 1.0 185.1 264 262 0 0 1.2 1.0 3 6 14 9198 120 0.0 0 0.4 0.0 0.7 fillseq.wal_disabled.v400 2022-03-16T16:21:23 7.0
325229 130.3 22GB 0.0GB, 0.0 0.0 0 0 0 0 12.3 139.8 170 237 249 572 60 0.0 0 0.4 0.1 1.2 multireadrandom.t4 2022-03-16T16:23:25 7.0
312920 125.3 26GB 0.0GB, 11.1 2.6 189.3 115 113 0 0 12.8 11.8 21 34 1255 6442 60 0.2 1 0.7 0.1 0.6 overwritesome.t4.s0 2022-03-16T16:25:27 7.0
81698 327.2 25GB 0.0GB, 0.0 0.0 0 0 0 0 48.9 46.2 79 246 369 9445 60 0.0 0 0.4 0.1 1.4 revrangewhilewriting.t4 2022-03-16T16:30:21 7.0
92484 370.4 25GB 0.0GB, 0.1 1.5 1.1 1 0 0 0 43.2 42.3 75 103 110 9512 62 0.0 0 0.4 0.1 1.4 fwdrangewhilewriting.t4 2022-03-16T16:32:24 7.0
241661 96.8 25GB 0.0GB, 0.1 1.5 1.1 1 0 0 0 16.5 17.1 30 34 49 9092 62 0.0 0 0.4 0.1 1.4 readwhilewriting.t4 2022-03-16T16:34:27 7.0
305234 122.3 30GB 0.0GB, 12.1 2.7 201.7 127 124 0 0 13.1 11.8 21 128 1934 6339 62 0.0 0 0.7 0.1 0.7 overwrite.t4.s0 2022-03-16T16:36:30 7.0
```
Pull Request resolved: https://github.com/facebook/rocksdb/pull/9704
Test Plan: run it
Reviewed By: jay-zhuang
Differential Revision: D36864627
Pulled By: mdcallag
fbshipit-source-id: d5af1cfc258a16865210163fa6fd1b803ab1a7d3
|
2 years ago |
| .. |
|
advisor
|
Update branch as "main" in tools/advisor/README.md (#8744)
|
3 years ago |
|
block_cache_analyzer
|
Use std::numeric_limits<> (#9954)
|
3 years ago |
|
dump
|
Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)
|
5 years ago |
|
CMakeLists.txt
|
Mark dependencies as PRIVATE and fix missing dependencies in tools. (#6790)
|
5 years ago |
|
Dockerfile
|
adding docker build script and dockerfile
|
10 years ago |
|
analyze_txn_stress_test.sh
|
Add copyright headers per FB open-source checkup tool. (#5199)
|
6 years ago |
|
auto_sanity_test.sh
|
Add copyright headers per FB open-source checkup tool. (#5199)
|
6 years ago |
|
backup_db.sh
|
Revamp check_format_compatible.sh (#8012)
|
4 years ago |
|
benchmark.sh
|
Enhance to support more tuning options, and universal and integrated… (#9704)
|
2 years ago |
|
benchmark_leveldb.sh
|
Add copyright headers per FB open-source checkup tool. (#5199)
|
6 years ago |
|
blob_dump.cc
|
Remove using namespace (#9369)
|
3 years ago |
|
check_all_python.py
|
Allow missing "unversioned" python, as in CentOS 8 (#6883)
|
5 years ago |
|
check_format_compatible.sh
|
Update version on main to 7.4 and add 7.3 to the format compatibility checks (#10038)
|
3 years ago |
|
db_bench.cc
|
Add (& fix) some simple source code checks (#8821)
|
3 years ago |
|
db_bench_tool.cc
|
Make it possible to enable blob files starting from a certain LSM tree level (#10077)
|
2 years ago |
|
db_bench_tool_test.cc
|
Make it possible to enable blob files starting from a certain LSM tree level (#10077)
|
2 years ago |
|
db_crashtest.py
|
Make it possible to enable blob files starting from a certain LSM tree level (#10077)
|
2 years ago |
|
db_repl_stress.cc
|
Remove using namespace (#9369)
|
3 years ago |
|
db_sanity_test.cc
|
Remove own ToString() (#9955)
|
3 years ago |
|
dbench_monitor
|
Fix /bin/bash shebangs
|
7 years ago |
|
generate_random_db.sh
|
Add copyright headers per FB open-source checkup tool. (#5199)
|
6 years ago |
|
ingest_external_sst.sh
|
Add copyright headers per FB open-source checkup tool. (#5199)
|
6 years ago |
|
io_tracer_parser.cc
|
Add IO Tracer Parser (#7333)
|
4 years ago |
|
io_tracer_parser_test.cc
|
Cleanup includes in dbformat.h (#8930)
|
3 years ago |
|
io_tracer_parser_tool.cc
|
Add request_id in IODebugContext. (#8045)
|
4 years ago |
|
io_tracer_parser_tool.h
|
Add IO Tracer Parser (#7333)
|
4 years ago |
|
ldb.cc
|
Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)
|
5 years ago |
|
ldb_cmd.cc
|
Make it possible to enable blob files starting from a certain LSM tree level (#10077)
|
2 years ago |
|
ldb_cmd_impl.h
|
Support single delete in ldb (#9469)
|
3 years ago |
|
ldb_cmd_test.cc
|
Remove own ToString() (#9955)
|
3 years ago |
|
ldb_test.py
|
Make it possible to enable blob files starting from a certain LSM tree level (#10077)
|
2 years ago |
|
ldb_tool.cc
|
Default `try_load_options` to true when DB is specified (#9937)
|
3 years ago |
|
pflag
|
Fix /bin/bash shebangs
|
7 years ago |
|
reduce_levels_test.cc
|
Remove own ToString() (#9955)
|
3 years ago |
|
regression_test.sh
|
Abort RocksDB performance regression test on failure in test setup (#10053)
|
3 years ago |
|
restore_db.sh
|
Revamp check_format_compatible.sh (#8012)
|
4 years ago |
|
rocksdb_dump_test.sh
|
Add copyright headers per FB open-source checkup tool. (#5199)
|
6 years ago |
|
run_blob_bench.sh
|
Make it possible to enable blob files starting from a certain LSM tree level (#10077)
|
2 years ago |
|
run_flash_bench.sh
|
Add copyright headers per FB open-source checkup tool. (#5199)
|
6 years ago |
|
run_leveldb.sh
|
Add copyright headers per FB open-source checkup tool. (#5199)
|
6 years ago |
|
sample-dump.dmp
|
First version of rocksdb_dump and rocksdb_undump.
|
10 years ago |
|
simulated_hybrid_file_system.cc
|
Improve SimulatedHybridFileSystem (#9301)
|
3 years ago |
|
simulated_hybrid_file_system.h
|
Improve SimulatedHybridFileSystem (#9301)
|
3 years ago |
|
sst_dump.cc
|
Implement a new subcommand "identify" for sst_dump (#6943)
|
4 years ago |
|
sst_dump_test.cc
|
Use the comparator from the sst file table properties in sst_dump_tool (#9491)
|
3 years ago |
|
sst_dump_tool.cc
|
Support using ZDICT_finalizeDictionary to generate zstd dictionary (#9857)
|
3 years ago |
|
trace_analyzer.cc
|
Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)
|
5 years ago |
|
trace_analyzer_test.cc
|
Support read rate-limiting in SequentialFileReader (#9973)
|
3 years ago |
|
trace_analyzer_tool.cc
|
Support read rate-limiting in SequentialFileReader (#9973)
|
3 years ago |
|
trace_analyzer_tool.h
|
Add commit marker with timestamp (#9266)
|
3 years ago |
|
verify_random_db.sh
|
Add copyright headers per FB open-source checkup tool. (#5199)
|
6 years ago |
|
write_external_sst.sh
|
Revamp check_format_compatible.sh (#8012)
|
4 years ago |
|
write_stress.cc
|
Add a SystemClock class to capture the time functions of an Env (#7858)
|
4 years ago |
|
write_stress_runner.py
|
Allow missing "unversioned" python, as in CentOS 8 (#6883)
|
5 years ago |
