You can not select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
gitbw95
f241d082b6
Prevent double caching in the compressed secondary cache (#9747)
Summary:
### **Summary:**
When both LRU Cache and CompressedSecondaryCache are configured together, there possibly are some data blocks double cached.
**Changes include:**
1. Update IS_PROMOTED to IS_IN_SECONDARY_CACHE to prevent confusions.
2. This PR updates SecondaryCacheResultHandle and use IsErasedFromSecondaryCache to determine whether the handle is erased in the secondary cache. Then, the caller can determine whether to SetIsInSecondaryCache().
3. Rename LRUSecondaryCache to CompressedSecondaryCache.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/9747
Test Plan:
**Test Scripts:**
1. Populate a DB. The on disk footprint is 482 MB. The data is set to be 50% compressible, so the total decompressed size is expected to be 964 MB.
./db_bench --benchmarks=fillrandom --num=10000000 -db=/db_bench_1
2. overwrite it to a stable state:
./db_bench --benchmarks=overwrite,stats --num=10000000 -use_existing_db -duration=10 --benchmark_write_rate_limit=2000000 -db=/db_bench_1
4. Run read tests with diffeernt cache setting:
T1:
./db_bench --benchmarks=seekrandom,stats --threads=16 --num=10000000 -use_existing_db -duration=120 --benchmark_write_rate_limit=52000000 -use_direct_reads --cache_size=520000000 --statistics -db=/db_bench_1
T2:
./db_bench --benchmarks=seekrandom,stats --threads=16 --num=10000000 -use_existing_db -duration=120 --benchmark_write_rate_limit=52000000 -use_direct_reads --cache_size=320000000 -compressed_secondary_cache_size=400000000 --statistics -use_compressed_secondary_cache -db=/db_bench_1
T3:
./db_bench --benchmarks=seekrandom,stats --threads=16 --num=10000000 -use_existing_db -duration=120 --benchmark_write_rate_limit=52000000 -use_direct_reads --cache_size=520000000 -compressed_secondary_cache_size=400000000 --statistics -use_compressed_secondary_cache -db=/db_bench_1
T4:
./db_bench --benchmarks=seekrandom,stats --threads=16 --num=10000000 -use_existing_db -duration=120 --benchmark_write_rate_limit=52000000 -use_direct_reads --cache_size=20000000 -compressed_secondary_cache_size=500000000 --statistics -use_compressed_secondary_cache -db=/db_bench_1
**Before this PR**
| Cache Size | Compressed Secondary Cache Size | Cache Hit Rate |
|------------|-------------------------------------|----------------|
|520 MB | 0 MB | 85.5% |
|320 MB | 400 MB | 96.2% |
|520 MB | 400 MB | 98.3% |
|20 MB | 500 MB | 98.8% |
**Before this PR**
| Cache Size | Compressed Secondary Cache Size | Cache Hit Rate |
|------------|-------------------------------------|----------------|
|520 MB | 0 MB | 85.5% |
|320 MB | 400 MB | 99.9% |
|520 MB | 400 MB | 99.9% |
|20 MB | 500 MB | 99.2% |
Reviewed By: anand1976
Differential Revision: D35117499
Pulled By: gitbw95
fbshipit-source-id: ea2657749fc13efebe91a8a1b56bc61d6a224a12
|
3 years ago |
| .. |
|
cf_options.cc
|
`compression_per_level` should be used for flush and changeable (#9658)
|
3 years ago |
|
cf_options.h
|
`compression_per_level` should be used for flush and changeable (#9658)
|
3 years ago |
|
configurable.cc
|
Fix an issue with MemTableRepFactory::CreateFromString (#9273)
|
3 years ago |
|
configurable_helper.h
|
Fix some minor issues in the Customizable infrastructure (#8566)
|
3 years ago |
|
configurable_test.cc
|
Fix GetOptionsPtr for Wrapped Customizable; Allow null options map (#9213)
|
3 years ago |
|
configurable_test.h
|
Move RegisterOptions into the Configurable API (#8223)
|
4 years ago |
|
customizable.cc
|
Add support to the ObjectRegistry for ManagedObjects (#8658)
|
3 years ago |
|
customizable_test.cc
|
Prevent double caching in the compressed secondary cache (#9747)
|
3 years ago |
|
db_options.cc
|
Document SetOptions API (#9778)
|
3 years ago |
|
db_options.h
|
Remove deprecated option new_table_reader_for_compaction_inputs (#9443)
|
3 years ago |
|
options.cc
|
Provide implementation to prefetch data asynchronously in FilePrefetchBuffer (#9674)
|
3 years ago |
|
options_helper.cc
|
New backup meta schema, with file temperatures (#9660)
|
3 years ago |
|
options_helper.h
|
New backup meta schema, with file temperatures (#9660)
|
3 years ago |
|
options_parser.cc
|
Allow unregistered options to be ignored in DBOptions from files (#9045)
|
3 years ago |
|
options_parser.h
|
Bring the Configurable options together (#5753)
|
4 years ago |
|
options_settable_test.cc
|
Account memory of big memory users in BlockBasedTable in global memory limit (#9748)
|
3 years ago |
|
options_test.cc
|
Account memory of big memory users in BlockBasedTable in global memory limit (#9748)
|
3 years ago |
