fork of https://github.com/oxigraph/rocksdb and https://github.com/facebook/rocksdb for nextgraph and oxigraph

You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

Andrew Kryczka dd29ad4223 Separate internal and user key comparators in `BlockIter` (#6944) ... Summary: Replace `BlockIter::comparator_` and `IndexBlockIter::user_comparator_wrapper_` with a concrete `UserComparatorWrapper` and `InternalKeyComparator`. The motivation for this change was the inconvenience of not knowing the concrete type of `BlockIter::comparator_`, which prevented calling specialized internal key comparison functions to optimize comparison of keys with global seqno applied. Pull Request resolved: https://github.com/facebook/rocksdb/pull/6944 Test Plan: benchmark setup -- single file DBs, in-memory, no compression. "normal_db" created by regular flush; "ingestion_db" created by ingesting a file. Both DBs have same contents. ``` $ TEST_TMPDIR=/dev/shm/normal_db/ ./db_bench -benchmarks=fillrandom,compact -write_buffer_size=10485760000 -disable_auto_compactions=true -compression_type=none -num=1000000 $ ./ldb write_extern_sst ./tmp.sst --db=/dev/shm/ingestion_db/dbbench/ --compression_type=no --hex --create_if_missing < <(./sst_dump --command=scan --output_hex --file=/dev/shm/normal_db/dbbench/000007.sst | awk 'began {print "0x" substr($1, 2, length($1) - 2), "==>", "0x" $5} ; /^Sst file format: block-based/ {began=1}') $ ./ldb ingest_extern_sst ./tmp.sst --db=/dev/shm/ingestion_db/dbbench/ ``` benchmark run command: ``` $ TEST_TMPDIR=/dev/shm/$DB/ ./db_bench -benchmarks=seekrandom -seek_nexts=$SEEK_NEXT -use_existing_db=true -cache_index_and_filter_blocks=false -num=1000000 -cache_size=0 -threads=1 -reads=200000000 -mmap_read=1 -verify_checksum=false ``` results: perf improved marginally for ingestion_db and did not change significantly for normal_db: SEEK_NEXT | DB | code | ops/sec | % change -- | -- | -- | -- | -- 0 | normal_db | master | 350880 |   0 | normal_db | PR6944 | 351040 | 0.0 0 | ingestion_db | master | 343255 |   0 | ingestion_db | PR6944 | 349424 | 1.8 10 | normal_db | master | 218711 |   10 | normal_db | PR6944 | 217892 | -0.4 10 | ingestion_db | master | 220334 |   10 | ingestion_db | PR6944 | 226437 | 2.8 Reviewed By: pdillinger Differential Revision: D21924676 Pulled By: ajkr fbshipit-source-id: ea4288a2eefa8112eb6c651a671c1de18c12e538		5 years ago
..
binary_search_index_reader.cc	Separate internal and user key comparators in `BlockIter` (#6944)	5 years ago
binary_search_index_reader.h	Extend Get/MultiGet deadline support to table open (#6982)	5 years ago
block.cc	Separate internal and user key comparators in `BlockIter` (#6944)	5 years ago
block.h	Separate internal and user key comparators in `BlockIter` (#6944)	5 years ago
block_based_filter_block.cc	Extend Get/MultiGet deadline support to table open (#6982)	5 years ago
block_based_filter_block.h	Extend Get/MultiGet deadline support to table open (#6982)	5 years ago
block_based_filter_block_test.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	5 years ago
block_based_table_builder.cc	Fix block checksum for >=4GB, refactor (#6978)	5 years ago
block_based_table_builder.h	Store DB identity and DB session ID in SST files (#6983)	5 years ago
block_based_table_factory.cc	Extend Get/MultiGet deadline support to table open (#6982)	5 years ago
block_based_table_factory.h	Extend Get/MultiGet deadline support to table open (#6982)	5 years ago
block_based_table_iterator.cc	Properly report IO errors when IndexType::kBinarySearchWithFirstKey is used (#6621)	5 years ago
block_based_table_iterator.h	Properly report IO errors when IndexType::kBinarySearchWithFirstKey is used (#6621)	5 years ago
block_based_table_reader.cc	Separate internal and user key comparators in `BlockIter` (#6944)	5 years ago
block_based_table_reader.h	Extend Get/MultiGet deadline support to table open (#6982)	5 years ago
block_based_table_reader_impl.h	Divide block_based_table_reader.cc (#6527)	5 years ago
block_based_table_reader_test.cc	Extend Get/MultiGet deadline support to table open (#6982)	5 years ago
block_builder.cc	Add pipelined & parallel compression optimization (#6262)	5 years ago
block_builder.h	Add pipelined & parallel compression optimization (#6262)	5 years ago
block_prefetcher.cc	De-template block based table iterator (#6531)	5 years ago
block_prefetcher.h	De-template block based table iterator (#6531)	5 years ago
block_prefix_index.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	5 years ago
block_prefix_index.h	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	5 years ago
block_test.cc	Separate internal and user key comparators in `BlockIter` (#6944)	5 years ago
block_type.h	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	5 years ago
cachable_entry.h	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	5 years ago
data_block_footer.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	5 years ago
data_block_footer.h	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	5 years ago
data_block_hash_index.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	5 years ago
data_block_hash_index.h	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	5 years ago
data_block_hash_index_test.cc	Separate internal and user key comparators in `BlockIter` (#6944)	5 years ago
filter_block.h	Extend Get/MultiGet deadline support to table open (#6982)	5 years ago
filter_block_reader_common.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	5 years ago
filter_block_reader_common.h	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	5 years ago
filter_policy.cc	Add unity build to CircleCI (#7026)	5 years ago
filter_policy_internal.h	Minimize memory internal fragmentation for Bloom filters (#6427)	5 years ago
flush_block_policy.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	5 years ago
flush_block_policy.h	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	5 years ago
full_filter_block.cc	Extend Get/MultiGet deadline support to table open (#6982)	5 years ago
full_filter_block.h	Extend Get/MultiGet deadline support to table open (#6982)	5 years ago
full_filter_block_test.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	5 years ago
hash_index_reader.cc	Separate internal and user key comparators in `BlockIter` (#6944)	5 years ago
hash_index_reader.h	Extend Get/MultiGet deadline support to table open (#6982)	5 years ago
index_builder.cc	Un-revert #7049, revert #7022 (#7071)	5 years ago
index_builder.h	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	5 years ago
index_reader_common.cc	Divide block_based_table_reader.cc (#6527)	5 years ago
index_reader_common.h	Divide block_based_table_reader.cc (#6527)	5 years ago
mock_block_based_table.h	For ApproximateSizes, pro-rate table metadata size over data blocks (#6784)	5 years ago
parsed_full_filter_block.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	5 years ago
parsed_full_filter_block.h	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	5 years ago
partitioned_filter_block.cc	Separate internal and user key comparators in `BlockIter` (#6944)	5 years ago
partitioned_filter_block.h	Extend Get/MultiGet deadline support to table open (#6982)	5 years ago
partitioned_filter_block_test.cc	Revert "Update googletest from 1.8.1 to 1.10.0 (#6808)" (#6923)	5 years ago
partitioned_index_iterator.cc	Fix regression bug in partitioned index reseek caused by #6531 (#6551)	5 years ago
partitioned_index_iterator.h	Check iterator status BlockBasedTableReader::VerifyChecksumInBlocks() (#6909)	5 years ago
partitioned_index_reader.cc	Separate internal and user key comparators in `BlockIter` (#6944)	5 years ago
partitioned_index_reader.h	Extend Get/MultiGet deadline support to table open (#6982)	5 years ago
reader_common.cc	Fix block checksum for >=4GB, refactor (#6978)	5 years ago
reader_common.h	Fix block checksum for >=4GB, refactor (#6978)	5 years ago
uncompression_dict_reader.cc	Extend Get/MultiGet deadline support to table open (#6982)	5 years ago
uncompression_dict_reader.h	Extend Get/MultiGet deadline support to table open (#6982)	5 years ago