You can not select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
gitbw95
f241d082b6
Prevent double caching in the compressed secondary cache (#9747)
Summary:
### **Summary:**
When both LRU Cache and CompressedSecondaryCache are configured together, there possibly are some data blocks double cached.
**Changes include:**
1. Update IS_PROMOTED to IS_IN_SECONDARY_CACHE to prevent confusions.
2. This PR updates SecondaryCacheResultHandle and use IsErasedFromSecondaryCache to determine whether the handle is erased in the secondary cache. Then, the caller can determine whether to SetIsInSecondaryCache().
3. Rename LRUSecondaryCache to CompressedSecondaryCache.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/9747
Test Plan:
**Test Scripts:**
1. Populate a DB. The on disk footprint is 482 MB. The data is set to be 50% compressible, so the total decompressed size is expected to be 964 MB.
./db_bench --benchmarks=fillrandom --num=10000000 -db=/db_bench_1
2. overwrite it to a stable state:
./db_bench --benchmarks=overwrite,stats --num=10000000 -use_existing_db -duration=10 --benchmark_write_rate_limit=2000000 -db=/db_bench_1
4. Run read tests with diffeernt cache setting:
T1:
./db_bench --benchmarks=seekrandom,stats --threads=16 --num=10000000 -use_existing_db -duration=120 --benchmark_write_rate_limit=52000000 -use_direct_reads --cache_size=520000000 --statistics -db=/db_bench_1
T2:
./db_bench --benchmarks=seekrandom,stats --threads=16 --num=10000000 -use_existing_db -duration=120 --benchmark_write_rate_limit=52000000 -use_direct_reads --cache_size=320000000 -compressed_secondary_cache_size=400000000 --statistics -use_compressed_secondary_cache -db=/db_bench_1
T3:
./db_bench --benchmarks=seekrandom,stats --threads=16 --num=10000000 -use_existing_db -duration=120 --benchmark_write_rate_limit=52000000 -use_direct_reads --cache_size=520000000 -compressed_secondary_cache_size=400000000 --statistics -use_compressed_secondary_cache -db=/db_bench_1
T4:
./db_bench --benchmarks=seekrandom,stats --threads=16 --num=10000000 -use_existing_db -duration=120 --benchmark_write_rate_limit=52000000 -use_direct_reads --cache_size=20000000 -compressed_secondary_cache_size=500000000 --statistics -use_compressed_secondary_cache -db=/db_bench_1
**Before this PR**
| Cache Size | Compressed Secondary Cache Size | Cache Hit Rate |
|------------|-------------------------------------|----------------|
|520 MB | 0 MB | 85.5% |
|320 MB | 400 MB | 96.2% |
|520 MB | 400 MB | 98.3% |
|20 MB | 500 MB | 98.8% |
**Before this PR**
| Cache Size | Compressed Secondary Cache Size | Cache Hit Rate |
|------------|-------------------------------------|----------------|
|520 MB | 0 MB | 85.5% |
|320 MB | 400 MB | 99.9% |
|520 MB | 400 MB | 99.9% |
|20 MB | 500 MB | 99.2% |
Reviewed By: anand1976
Differential Revision: D35117499
Pulled By: gitbw95
fbshipit-source-id: ea2657749fc13efebe91a8a1b56bc61d6a224a12
|
3 years ago |
| .. |
|
advisor
|
Update branch as "main" in tools/advisor/README.md (#8744)
|
3 years ago |
|
block_cache_analyzer
|
Cleanup includes in dbformat.h (#8930)
|
3 years ago |
|
dump
|
Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)
|
5 years ago |
|
CMakeLists.txt
|
Mark dependencies as PRIVATE and fix missing dependencies in tools. (#6790)
|
5 years ago |
|
Dockerfile
|
adding docker build script and dockerfile
|
10 years ago |
|
analyze_txn_stress_test.sh
|
Add copyright headers per FB open-source checkup tool. (#5199)
|
6 years ago |
|
auto_sanity_test.sh
|
Add copyright headers per FB open-source checkup tool. (#5199)
|
6 years ago |
|
backup_db.sh
|
Revamp check_format_compatible.sh (#8012)
|
4 years ago |
|
benchmark.sh
|
Remove deprecated API AdvancedColumnFamilyOptions::rate_limit_delay_max_milliseconds (#9455)
|
3 years ago |
|
benchmark_leveldb.sh
|
Add copyright headers per FB open-source checkup tool. (#5199)
|
6 years ago |
|
blob_dump.cc
|
Remove using namespace (#9369)
|
3 years ago |
|
check_all_python.py
|
Allow missing "unversioned" python, as in CentOS 8 (#6883)
|
5 years ago |
|
check_format_compatible.sh
|
Add 7.0.fb/7.1.fb to check_format_compatible.sh (#9772)
|
3 years ago |
|
db_bench.cc
|
Add (& fix) some simple source code checks (#8821)
|
3 years ago |
|
db_bench_tool.cc
|
Prevent double caching in the compressed secondary cache (#9747)
|
3 years ago |
|
db_bench_tool_test.cc
|
Remove BlockBasedTableOptions.hash_index_allow_collision (#9454)
|
3 years ago |
|
db_crashtest.py
|
Add WAL compression to stress tests (#9811)
|
3 years ago |
|
db_repl_stress.cc
|
Remove using namespace (#9369)
|
3 years ago |
|
db_sanity_test.cc
|
Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)
|
5 years ago |
|
dbench_monitor
|
Fix /bin/bash shebangs
|
7 years ago |
|
generate_random_db.sh
|
Add copyright headers per FB open-source checkup tool. (#5199)
|
6 years ago |
|
ingest_external_sst.sh
|
Add copyright headers per FB open-source checkup tool. (#5199)
|
6 years ago |
|
io_tracer_parser.cc
|
Add IO Tracer Parser (#7333)
|
4 years ago |
|
io_tracer_parser_test.cc
|
Cleanup includes in dbformat.h (#8930)
|
3 years ago |
|
io_tracer_parser_tool.cc
|
Add request_id in IODebugContext. (#8045)
|
4 years ago |
|
io_tracer_parser_tool.h
|
Add IO Tracer Parser (#7333)
|
4 years ago |
|
ldb.cc
|
Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)
|
5 years ago |
|
ldb_cmd.cc
|
Fix remaining uses of "backupable" (#9792)
|
3 years ago |
|
ldb_cmd_impl.h
|
Fix remaining uses of "backupable" (#9792)
|
3 years ago |
|
ldb_cmd_test.cc
|
Add manifest fix-up utility for file temperatures (#9683)
|
3 years ago |
|
ldb_test.py
|
Add support for BlobDB to ldb (#9630)
|
3 years ago |
|
ldb_tool.cc
|
Add manifest fix-up utility for file temperatures (#9683)
|
3 years ago |
|
pflag
|
Fix /bin/bash shebangs
|
7 years ago |
|
reduce_levels_test.cc
|
Remove some unneeded code (#8736)
|
3 years ago |
|
regression_test.sh
|
Fix COMMIT_ID in regression_test.sh (#9047)
|
3 years ago |
|
restore_db.sh
|
Revamp check_format_compatible.sh (#8012)
|
4 years ago |
|
rocksdb_dump_test.sh
|
Add copyright headers per FB open-source checkup tool. (#5199)
|
6 years ago |
|
run_blob_bench.sh
|
Add blob compaction readahead size to the BlobDB benchmark script (#9566)
|
3 years ago |
|
run_flash_bench.sh
|
Add copyright headers per FB open-source checkup tool. (#5199)
|
6 years ago |
|
run_leveldb.sh
|
Add copyright headers per FB open-source checkup tool. (#5199)
|
6 years ago |
|
sample-dump.dmp
|
First version of rocksdb_dump and rocksdb_undump.
|
10 years ago |
|
simulated_hybrid_file_system.cc
|
Improve SimulatedHybridFileSystem (#9301)
|
3 years ago |
|
simulated_hybrid_file_system.h
|
Improve SimulatedHybridFileSystem (#9301)
|
3 years ago |
|
sst_dump.cc
|
Implement a new subcommand "identify" for sst_dump (#6943)
|
5 years ago |
|
sst_dump_test.cc
|
Use the comparator from the sst file table properties in sst_dump_tool (#9491)
|
3 years ago |
|
sst_dump_tool.cc
|
New backup meta schema, with file temperatures (#9660)
|
3 years ago |
|
trace_analyzer.cc
|
Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)
|
5 years ago |
|
trace_analyzer_test.cc
|
Disable the QPS verification in test temporally (#9190)
|
3 years ago |
|
trace_analyzer_tool.cc
|
Simplify `TraceAnalyzer` (#8697)
|
3 years ago |
|
trace_analyzer_tool.h
|
Add commit marker with timestamp (#9266)
|
3 years ago |
|
verify_random_db.sh
|
Add copyright headers per FB open-source checkup tool. (#5199)
|
6 years ago |
|
write_external_sst.sh
|
Revamp check_format_compatible.sh (#8012)
|
4 years ago |
|
write_stress.cc
|
Add a SystemClock class to capture the time functions of an Env (#7858)
|
4 years ago |
|
write_stress_runner.py
|
Allow missing "unversioned" python, as in CentOS 8 (#6883)
|
5 years ago |
