You can not select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
gitbw95
f241d082b6
Prevent double caching in the compressed secondary cache (#9747)
Summary:
### **Summary:**
When both LRU Cache and CompressedSecondaryCache are configured together, there possibly are some data blocks double cached.
**Changes include:**
1. Update IS_PROMOTED to IS_IN_SECONDARY_CACHE to prevent confusions.
2. This PR updates SecondaryCacheResultHandle and use IsErasedFromSecondaryCache to determine whether the handle is erased in the secondary cache. Then, the caller can determine whether to SetIsInSecondaryCache().
3. Rename LRUSecondaryCache to CompressedSecondaryCache.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/9747
Test Plan:
**Test Scripts:**
1. Populate a DB. The on disk footprint is 482 MB. The data is set to be 50% compressible, so the total decompressed size is expected to be 964 MB.
./db_bench --benchmarks=fillrandom --num=10000000 -db=/db_bench_1
2. overwrite it to a stable state:
./db_bench --benchmarks=overwrite,stats --num=10000000 -use_existing_db -duration=10 --benchmark_write_rate_limit=2000000 -db=/db_bench_1
4. Run read tests with diffeernt cache setting:
T1:
./db_bench --benchmarks=seekrandom,stats --threads=16 --num=10000000 -use_existing_db -duration=120 --benchmark_write_rate_limit=52000000 -use_direct_reads --cache_size=520000000 --statistics -db=/db_bench_1
T2:
./db_bench --benchmarks=seekrandom,stats --threads=16 --num=10000000 -use_existing_db -duration=120 --benchmark_write_rate_limit=52000000 -use_direct_reads --cache_size=320000000 -compressed_secondary_cache_size=400000000 --statistics -use_compressed_secondary_cache -db=/db_bench_1
T3:
./db_bench --benchmarks=seekrandom,stats --threads=16 --num=10000000 -use_existing_db -duration=120 --benchmark_write_rate_limit=52000000 -use_direct_reads --cache_size=520000000 -compressed_secondary_cache_size=400000000 --statistics -use_compressed_secondary_cache -db=/db_bench_1
T4:
./db_bench --benchmarks=seekrandom,stats --threads=16 --num=10000000 -use_existing_db -duration=120 --benchmark_write_rate_limit=52000000 -use_direct_reads --cache_size=20000000 -compressed_secondary_cache_size=500000000 --statistics -use_compressed_secondary_cache -db=/db_bench_1
**Before this PR**
| Cache Size | Compressed Secondary Cache Size | Cache Hit Rate |
|------------|-------------------------------------|----------------|
|520 MB | 0 MB | 85.5% |
|320 MB | 400 MB | 96.2% |
|520 MB | 400 MB | 98.3% |
|20 MB | 500 MB | 98.8% |
**Before this PR**
| Cache Size | Compressed Secondary Cache Size | Cache Hit Rate |
|------------|-------------------------------------|----------------|
|520 MB | 0 MB | 85.5% |
|320 MB | 400 MB | 99.9% |
|520 MB | 400 MB | 99.9% |
|20 MB | 500 MB | 99.2% |
Reviewed By: anand1976
Differential Revision: D35117499
Pulled By: gitbw95
fbshipit-source-id: ea2657749fc13efebe91a8a1b56bc61d6a224a12
|
3 years ago |
| .. |
|
backup
|
Fix remaining uses of "backupable" (#9792)
|
3 years ago |
|
blob_db
|
Fix FileStorageInfo fields from GetLiveFilesMetaData (#9769)
|
3 years ago |
|
cassandra
|
Restore Regex support for ObjectLibrary::Register, rename new APIs to allow old one to be deprecated in the future (#9362)
|
3 years ago |
|
checkpoint
|
Add Temperature info in `NewSequentialFile()` (#9499)
|
3 years ago |
|
compaction_filters
|
Make MergeOperator+CompactionFilter/Factory into Customizable Classes (#8481)
|
3 years ago |
|
convenience
|
Add a SystemClock class to capture the time functions of an Env (#7858)
|
4 years ago |
|
leveldb_options
|
Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)
|
5 years ago |
|
memory
|
Make types of Immutable/Mutable Options fields match that of the underlying Option (#8176)
|
4 years ago |
|
merge_operators
|
Remove using namespace (#9369)
|
3 years ago |
|
option_change_migration
|
Fix some typos in comments and HISTORY.md (#9798)
|
3 years ago |
|
options
|
Allow WAL dir to change with db dir (#8582)
|
3 years ago |
|
persistent_cache
|
Add rate limiter priority to ReadOptions (#9424)
|
3 years ago |
|
simulator_cache
|
Update Cache::Release param from force_erase to erase_if_last_ref (#9728)
|
3 years ago |
|
table_properties_collectors
|
Restore Regex support for ObjectLibrary::Register, rename new APIs to allow old one to be deprecated in the future (#9362)
|
3 years ago |
|
trace
|
Add rate limiter priority to ReadOptions (#9424)
|
3 years ago |
|
transactions
|
Disallow commit-time-batch for write-prepared/write-unprepared txn conditionally (#9794)
|
3 years ago |
|
ttl
|
Work around some new clang-analyze failures (#9515)
|
3 years ago |
|
write_batch_with_index
|
Support WBWI for keys having timestamps (#9603)
|
3 years ago |
|
cache_dump_load.cc
|
Introduce a mechanism to dump out blocks from block cache and re-insert to secondary cache (#8912)
|
3 years ago |
|
cache_dump_load_impl.cc
|
New stable, fixed-length cache keys (#9126)
|
3 years ago |
|
cache_dump_load_impl.h
|
Add rate limiter priority to ReadOptions (#9424)
|
3 years ago |
|
compaction_filters.cc
|
Restore Regex support for ObjectLibrary::Register, rename new APIs to allow old one to be deprecated in the future (#9362)
|
3 years ago |
|
counted_fs.cc
|
Introduce a CountedFileSystem for counting file operations (#9283)
|
3 years ago |
|
counted_fs.h
|
Introduce a CountedFileSystem for counting file operations (#9283)
|
3 years ago |
|
debug.cc
|
In ParseInternalKey(), include corrupt key info in Status (#7515)
|
4 years ago |
|
env_mirror.cc
|
Fix clang13 build error (#9374)
|
3 years ago |
|
env_mirror_test.cc
|
Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)
|
5 years ago |
|
env_timed.cc
|
Make FileSystem a Customizable Class (#8649)
|
3 years ago |
|
env_timed.h
|
Make FileSystem a Customizable Class (#8649)
|
3 years ago |
|
env_timed_test.cc
|
Make env*_test work with ASSERT_STATUS_CHECKED (#7176)
|
4 years ago |
|
fault_injection_env.cc
|
Protect existing files in `FaultInjectionTest{Env,FS}::ReopenWritableFile()` (#8995)
|
3 years ago |
|
fault_injection_env.h
|
Make the Env class Customizable (#9293)
|
3 years ago |
|
fault_injection_fs.cc
|
Skip directory fsync for filesystem btrfs (#8903)
|
3 years ago |
|
fault_injection_fs.h
|
Fix a bug causing duplicate trailing entries in WritableFile (buffered IO) (#9236)
|
3 years ago |
|
fault_injection_secondary_cache.cc
|
Prevent double caching in the compressed secondary cache (#9747)
|
3 years ago |
|
fault_injection_secondary_cache.h
|
Prevent double caching in the compressed secondary cache (#9747)
|
3 years ago |
|
memory_allocators.h
|
Make MemoryAllocator into a Customizable class (#8980)
|
3 years ago |
|
merge_operators.cc
|
Restore Regex support for ObjectLibrary::Register, rename new APIs to allow old one to be deprecated in the future (#9362)
|
3 years ago |
|
merge_operators.h
|
Make MergeOperator+CompactionFilter/Factory into Customizable Classes (#8481)
|
3 years ago |
|
object_registry.cc
|
Make FilterPolicy Customizable (#9590)
|
3 years ago |
|
object_registry_test.cc
|
Add support for decimals to PatternEntry (#9577)
|
3 years ago |
|
util_merge_operators_test.cc
|
Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)
|
5 years ago |
|
wal_filter.cc
|
Make WalFilter, SstPartitionerFactory, FileChecksumGenFactory, and TableProperties Customizable (#8638)
|
3 years ago |
