fork of https://github.com/oxigraph/rocksdb and https://github.com/facebook/rocksdb for nextgraph and oxigraph

You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

anand76 8ea0a2c1bd Parallelize secondary cache lookup in MultiGet (#8405) ... Summary: Implement the ```WaitAll()``` interface in ```LRUCache``` to allow callers to issue multiple lookups in parallel and wait for all of them to complete. Modify ```MultiGet``` to use this to parallelize the secondary cache lookups in order to reduce the overall latency. A call to ```cache->Lookup()``` returns a handle that has an incomplete value (nullptr), and the caller can call ```cache->IsReady()``` to check whether the lookup is complete, and pass a vector of handles to ```WaitAll``` to wait for completion. If any of the lookups fail, ```MultiGet``` will read the block from the SST file. Another change in this PR is to rename ```SecondaryCacheHandle``` to ```SecondaryCacheResultHandle``` as it more accurately describes the return result of the secondary cache lookup, which is more like a future. Tests: 1. Add unit tests in lru_cache_test 2. Benchmark results with no secondary cache configured Master - ``` readrandom : 41.175 micros/op 388562 ops/sec; 106.7 MB/s (7277999 of 7277999 found) readrandom : 41.217 micros/op 388160 ops/sec; 106.6 MB/s (7274999 of 7274999 found) multireadrandom : 10.309 micros/op 1552082 ops/sec; (28908992 of 28908992 found) multireadrandom : 10.321 micros/op 1550218 ops/sec; (29081984 of 29081984 found) ``` This PR - ``` readrandom : 41.158 micros/op 388723 ops/sec; 106.8 MB/s (7290999 of 7290999 found) readrandom : 41.185 micros/op 388463 ops/sec; 106.7 MB/s (7287999 of 7287999 found) multireadrandom : 10.277 micros/op 1556801 ops/sec; (29346944 of 29346944 found) multireadrandom : 10.253 micros/op 1560539 ops/sec; (29274944 of 29274944 found) ``` Pull Request resolved: https://github.com/facebook/rocksdb/pull/8405 Reviewed By: zhichao-cao Differential Revision: D29190509 Pulled By: anand1976 fbshipit-source-id: 6f8eff6246712af8a297cfe22ea0d1c3b2a01bb0		3 years ago
..
adaptive	Refactor: use TableBuilderOptions to reduce parameter lists (#8240)	4 years ago
block_based	Parallelize secondary cache lookup in MultiGet (#8405)	3 years ago
cuckoo	Make ImmutableOptions struct that inherits from ImmutableCFOptions and ImmutableDBOptions (#8262)	4 years ago
plain	Make it possible to apply only a subrange of table property collectors (#8298)	4 years ago
block_fetcher.cc	Rename variables in ImmutableCFOptions to avoid conflicts with ImmutableDBOptions (#8227)	4 years ago
block_fetcher.h	Make ImmutableOptions struct that inherits from ImmutableCFOptions and ImmutableDBOptions (#8262)	4 years ago
block_fetcher_test.cc	Make it possible to apply only a subrange of table property collectors (#8298)	4 years ago
cleanable_test.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	5 years ago
format.cc	Make ImmutableOptions struct that inherits from ImmutableCFOptions and ImmutableDBOptions (#8262)	4 years ago
format.h	Make ImmutableOptions struct that inherits from ImmutableCFOptions and ImmutableDBOptions (#8262)	4 years ago
get_context.cc	Support for Merge in Integrated BlobDB with base values (#8292)	3 years ago
get_context.h	Support for Merge in Integrated BlobDB with base values (#8292)	3 years ago
internal_iterator.h	Clean up InternalIterator upper bound logic a little bit (#7200)	4 years ago
iter_heap.h	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	5 years ago
iterator.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	5 years ago
iterator_wrapper.h	Redesign block cache pinning API (#7520)	4 years ago
merger_test.cc	More Makefile Cleanup (#7097)	4 years ago
merging_iterator.cc	Add some simulator cache and block tracer tests to ASSERT_STATUS_CHECKED (#7305)	4 years ago
merging_iterator.h	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	5 years ago
meta_blocks.cc	Add table properties for number of entries added to filters (#8323)	4 years ago
meta_blocks.h	Make ImmutableOptions struct that inherits from ImmutableCFOptions and ImmutableDBOptions (#8262)	4 years ago
mock_table.cc	Refactor: use TableBuilderOptions to reduce parameter lists (#8240)	4 years ago
mock_table.h	Refactor: use TableBuilderOptions to reduce parameter lists (#8240)	4 years ago
multiget_context.h	Add initial blob support to batched MultiGet (#7766)	4 years ago
persistent_cache_helper.cc	Add more tests to ASSERT_STATUS_CHECKED (#7367)	4 years ago
persistent_cache_helper.h	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	5 years ago
persistent_cache_options.h	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	5 years ago
scoped_arena_iterator.h	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	5 years ago
sst_file_dumper.cc	Make it possible to apply only a subrange of table property collectors (#8298)	4 years ago
sst_file_dumper.h	Make ImmutableOptions struct that inherits from ImmutableCFOptions and ImmutableDBOptions (#8262)	4 years ago
sst_file_reader.cc	Make ImmutableOptions struct that inherits from ImmutableCFOptions and ImmutableDBOptions (#8262)	4 years ago
sst_file_reader_test.cc	Add CreateFrom methods to Env/FileSystem (#8174)	3 years ago
sst_file_writer.cc	Use DbSessionId as cache key prefix when secondary cache is enabled (#8360)	3 years ago
sst_file_writer_collectors.h	Apply `sample_for_compression` to all block-based tables (#8105)	4 years ago
table_builder.h	Use DbSessionId as cache key prefix when secondary cache is enabled (#8360)	3 years ago
table_factory.cc	Create a Customizable class to load classes and configurations (#6590)	4 years ago
table_properties.cc	Add table properties for number of entries added to filters (#8323)	4 years ago
table_properties_internal.h	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	5 years ago
table_reader.h	dedup ReadOptions in iterator hierarchy (#7210)	4 years ago
table_reader_bench.cc	Make it possible to apply only a subrange of table property collectors (#8298)	4 years ago
table_reader_caller.h	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	5 years ago
table_test.cc	Increase range for GeneralTableTest.ApproximateOffsetOfCompressed (#8387)	3 years ago
two_level_iterator.cc	Redesign block cache pinning API (#7520)	4 years ago
two_level_iterator.h	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	5 years ago