You can not select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
gitbw95
f241d082b6
Prevent double caching in the compressed secondary cache (#9747)
Summary:
### **Summary:**
When both LRU Cache and CompressedSecondaryCache are configured together, there possibly are some data blocks double cached.
**Changes include:**
1. Update IS_PROMOTED to IS_IN_SECONDARY_CACHE to prevent confusions.
2. This PR updates SecondaryCacheResultHandle and use IsErasedFromSecondaryCache to determine whether the handle is erased in the secondary cache. Then, the caller can determine whether to SetIsInSecondaryCache().
3. Rename LRUSecondaryCache to CompressedSecondaryCache.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/9747
Test Plan:
**Test Scripts:**
1. Populate a DB. The on disk footprint is 482 MB. The data is set to be 50% compressible, so the total decompressed size is expected to be 964 MB.
./db_bench --benchmarks=fillrandom --num=10000000 -db=/db_bench_1
2. overwrite it to a stable state:
./db_bench --benchmarks=overwrite,stats --num=10000000 -use_existing_db -duration=10 --benchmark_write_rate_limit=2000000 -db=/db_bench_1
4. Run read tests with diffeernt cache setting:
T1:
./db_bench --benchmarks=seekrandom,stats --threads=16 --num=10000000 -use_existing_db -duration=120 --benchmark_write_rate_limit=52000000 -use_direct_reads --cache_size=520000000 --statistics -db=/db_bench_1
T2:
./db_bench --benchmarks=seekrandom,stats --threads=16 --num=10000000 -use_existing_db -duration=120 --benchmark_write_rate_limit=52000000 -use_direct_reads --cache_size=320000000 -compressed_secondary_cache_size=400000000 --statistics -use_compressed_secondary_cache -db=/db_bench_1
T3:
./db_bench --benchmarks=seekrandom,stats --threads=16 --num=10000000 -use_existing_db -duration=120 --benchmark_write_rate_limit=52000000 -use_direct_reads --cache_size=520000000 -compressed_secondary_cache_size=400000000 --statistics -use_compressed_secondary_cache -db=/db_bench_1
T4:
./db_bench --benchmarks=seekrandom,stats --threads=16 --num=10000000 -use_existing_db -duration=120 --benchmark_write_rate_limit=52000000 -use_direct_reads --cache_size=20000000 -compressed_secondary_cache_size=500000000 --statistics -use_compressed_secondary_cache -db=/db_bench_1
**Before this PR**
| Cache Size | Compressed Secondary Cache Size | Cache Hit Rate |
|------------|-------------------------------------|----------------|
|520 MB | 0 MB | 85.5% |
|320 MB | 400 MB | 96.2% |
|520 MB | 400 MB | 98.3% |
|20 MB | 500 MB | 98.8% |
**Before this PR**
| Cache Size | Compressed Secondary Cache Size | Cache Hit Rate |
|------------|-------------------------------------|----------------|
|520 MB | 0 MB | 85.5% |
|320 MB | 400 MB | 99.9% |
|520 MB | 400 MB | 99.9% |
|20 MB | 500 MB | 99.2% |
Reviewed By: anand1976
Differential Revision: D35117499
Pulled By: gitbw95
fbshipit-source-id: ea2657749fc13efebe91a8a1b56bc61d6a224a12
|
3 years ago |
| .. |
|
cache.cc
|
Make SecondaryCache Customizable (#8480)
|
3 years ago |
|
cache_bench.cc
|
Add (& fix) some simple source code checks (#8821)
|
3 years ago |
|
cache_bench_tool.cc
|
Change type of cache buffer passed to `Cache::CreateCallback()` to `const void*` (#9595)
|
3 years ago |
|
cache_entry_roles.cc
|
Account memory of big memory users in BlockBasedTable in global memory limit (#9748)
|
3 years ago |
|
cache_entry_roles.h
|
Account memory of big memory users in BlockBasedTable in global memory limit (#9748)
|
3 years ago |
|
cache_entry_stats.h
|
New stable, fixed-length cache keys (#9126)
|
3 years ago |
|
cache_helpers.h
|
Use deleters to label cache entries and collect stats (#8297)
|
4 years ago |
|
cache_key.cc
|
Enhance new cache key testing & comments (#9329)
|
3 years ago |
|
cache_key.h
|
New stable, fixed-length cache keys (#9126)
|
3 years ago |
|
cache_reservation_manager.cc
|
Account memory of big memory users in BlockBasedTable in global memory limit (#9748)
|
3 years ago |
|
cache_reservation_manager.h
|
Account memory of big memory users in BlockBasedTable in global memory limit (#9748)
|
3 years ago |
|
cache_reservation_manager_test.cc
|
Account memory of big memory users in BlockBasedTable in global memory limit (#9748)
|
3 years ago |
|
cache_test.cc
|
Fix/improve 'must free heap allocations' code (#9209)
|
3 years ago |
|
clock_cache.cc
|
Update Cache::Release param from force_erase to erase_if_last_ref (#9728)
|
3 years ago |
|
clock_cache.h
|
Change RocksDB License
|
7 years ago |
|
compressed_secondary_cache.cc
|
Prevent double caching in the compressed secondary cache (#9747)
|
3 years ago |
|
compressed_secondary_cache.h
|
Prevent double caching in the compressed secondary cache (#9747)
|
3 years ago |
|
compressed_secondary_cache_test.cc
|
Prevent double caching in the compressed secondary cache (#9747)
|
3 years ago |
|
lru_cache.cc
|
Prevent double caching in the compressed secondary cache (#9747)
|
3 years ago |
|
lru_cache.h
|
Prevent double caching in the compressed secondary cache (#9747)
|
3 years ago |
|
lru_cache_test.cc
|
Prevent double caching in the compressed secondary cache (#9747)
|
3 years ago |
|
sharded_cache.cc
|
Update Cache::Release param from force_erase to erase_if_last_ref (#9728)
|
3 years ago |
|
sharded_cache.h
|
Update Cache::Release param from force_erase to erase_if_last_ref (#9728)
|
3 years ago |
