fork of https://github.com/oxigraph/rocksdb and https://github.com/facebook/rocksdb for nextgraph and oxigraph

You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

haoyuhuang 5efa0d6b0d Create a BlockCacheLookupContext to enable fine-grained block cache tracing. (#5421) ... Summary: BlockCacheLookupContext only contains the caller for now. We will trace block accesses at five places: 1. BlockBasedTable::GetFilter. 2. BlockBasedTable::GetUncompressedDict. 3. BlockBasedTable::MaybeReadAndLoadToCache. (To trace access on data, index, and range deletion block.) 4. BlockBasedTable::Get. (To trace the referenced key and whether the referenced key exists in a fetched data block.) 5. BlockBasedTable::MultiGet. (To trace the referenced key and whether the referenced key exists in a fetched data block.) We create the context at: 1. BlockBasedTable::Get. (kUserGet) 2. BlockBasedTable::MultiGet. (kUserMGet) 3. BlockBasedTable::NewIterator. (either kUserIterator, kCompaction, or external SST ingestion calls this function.) 4. BlockBasedTable::Open. (kPrefetch) 5. Index/Filter::CacheDependencies. (kPrefetch) 6. BlockBasedTable::ApproximateOffsetOf. (kCompaction or kUserApproximateSize). I loaded 1 million key-value pairs into the database and ran the readrandom benchmark with a single thread. I gave the block cache 10 GB to make sure all reads hit the block cache after warmup. The throughput is comparable. Throughput of this PR: 231334 ops/s. Throughput of the master branch: 238428 ops/s. Experiment setup: RocksDB: version 6.2 Date: Mon Jun 10 10:42:51 2019 CPU: 24 * Intel Core Processor (Skylake) CPUCache: 16384 KB Keys: 20 bytes each Values: 100 bytes each (100 bytes after compression) Entries: 1000000 Prefix: 20 bytes Keys per prefix: 0 RawSize: 114.4 MB (estimated) FileSize: 114.4 MB (estimated) Write rate: 0 bytes/second Read rate: 0 ops/second Compression: NoCompression Compression sampling rate: 0 Memtablerep: skip_list Perf Level: 1 Load command: ./db_bench --benchmarks="fillseq" --key_size=20 --prefix_size=20 --keys_per_prefix=0 --value_size=100 --statistics --cache_index_and_filter_blocks --cache_size=10737418240 --disable_auto_compactions=1 --disable_wal=1 --compression_type=none --min_level_to_compress=-1 --compression_ratio=1 --num=1000000 Run command: ./db_bench --benchmarks="readrandom,stats" --use_existing_db --threads=1 --duration=120 --key_size=20 --prefix_size=20 --keys_per_prefix=0 --value_size=100 --statistics --cache_index_and_filter_blocks --cache_size=10737418240 --disable_auto_compactions=1 --disable_wal=1 --compression_type=none --min_level_to_compress=-1 --compression_ratio=1 --num=1000000 --duration=120 TODOs: 1. Create a caller for external SST file ingestion and differentiate the callers for iterator. 2. Integrate tracer to trace block cache accesses. Pull Request resolved: https://github.com/facebook/rocksdb/pull/5421 Differential Revision: D15704258 Pulled By: HaoyuHuang fbshipit-source-id: 4aa8a55f8cb1576ffb367bfa3186a91d8f06d93a		5 years ago
..
compaction.cc	simplify include directive involving inttypes (#5402)	5 years ago
compaction.h	Organizing rocksdb/db directory	6 years ago
compaction_iteration_stats.h	Organizing rocksdb/db directory	6 years ago
compaction_iterator.cc	Organizing rocksdb/db directory	6 years ago
compaction_iterator.h	Make format	6 years ago
compaction_iterator_test.cc	Organizing rocksdb/db directory	6 years ago
compaction_job.cc	Create a BlockCacheLookupContext to enable fine-grained block cache tracing. (#5421)	5 years ago
compaction_job.h	Move some logging related files to logging/ (#5387)	6 years ago
compaction_job_stats_test.cc	simplify include directive involving inttypes (#5402)	5 years ago
compaction_job_test.cc	simplify include directive involving inttypes (#5402)	5 years ago
compaction_picker.cc	simplify include directive involving inttypes (#5402)	5 years ago
compaction_picker.h	Organizing rocksdb/db directory	6 years ago
compaction_picker_fifo.cc	simplify include directive involving inttypes (#5402)	5 years ago
compaction_picker_fifo.h	Organizing rocksdb/db directory	6 years ago
compaction_picker_level.cc	simplify include directive involving inttypes (#5402)	5 years ago
compaction_picker_level.h	Organizing rocksdb/db directory	6 years ago
compaction_picker_test.cc	Move some logging related files to logging/ (#5387)	6 years ago
compaction_picker_universal.cc	simplify include directive involving inttypes (#5402)	5 years ago
compaction_picker_universal.h	Organizing rocksdb/db directory	6 years ago