fork of https://github.com/oxigraph/rocksdb and https://github.com/facebook/rocksdb for nextgraph and oxigraph

You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

Andrew Kryczka dd29ad4223 Separate internal and user key comparators in `BlockIter` (#6944) ... Summary: Replace `BlockIter::comparator_` and `IndexBlockIter::user_comparator_wrapper_` with a concrete `UserComparatorWrapper` and `InternalKeyComparator`. The motivation for this change was the inconvenience of not knowing the concrete type of `BlockIter::comparator_`, which prevented calling specialized internal key comparison functions to optimize comparison of keys with global seqno applied. Pull Request resolved: https://github.com/facebook/rocksdb/pull/6944 Test Plan: benchmark setup -- single file DBs, in-memory, no compression. "normal_db" created by regular flush; "ingestion_db" created by ingesting a file. Both DBs have same contents. ``` $ TEST_TMPDIR=/dev/shm/normal_db/ ./db_bench -benchmarks=fillrandom,compact -write_buffer_size=10485760000 -disable_auto_compactions=true -compression_type=none -num=1000000 $ ./ldb write_extern_sst ./tmp.sst --db=/dev/shm/ingestion_db/dbbench/ --compression_type=no --hex --create_if_missing < <(./sst_dump --command=scan --output_hex --file=/dev/shm/normal_db/dbbench/000007.sst | awk 'began {print "0x" substr($1, 2, length($1) - 2), "==>", "0x" $5} ; /^Sst file format: block-based/ {began=1}') $ ./ldb ingest_extern_sst ./tmp.sst --db=/dev/shm/ingestion_db/dbbench/ ``` benchmark run command: ``` $ TEST_TMPDIR=/dev/shm/$DB/ ./db_bench -benchmarks=seekrandom -seek_nexts=$SEEK_NEXT -use_existing_db=true -cache_index_and_filter_blocks=false -num=1000000 -cache_size=0 -threads=1 -reads=200000000 -mmap_read=1 -verify_checksum=false ``` results: perf improved marginally for ingestion_db and did not change significantly for normal_db: SEEK_NEXT | DB | code | ops/sec | % change -- | -- | -- | -- | -- 0 | normal_db | master | 350880 |   0 | normal_db | PR6944 | 351040 | 0.0 0 | ingestion_db | master | 343255 |   0 | ingestion_db | PR6944 | 349424 | 1.8 10 | normal_db | master | 218711 |   10 | normal_db | PR6944 | 217892 | -0.4 10 | ingestion_db | master | 220334 |   10 | ingestion_db | PR6944 | 226437 | 2.8 Reviewed By: pdillinger Differential Revision: D21924676 Pulled By: ajkr fbshipit-source-id: ea4288a2eefa8112eb6c651a671c1de18c12e538		4 years ago
..
adaptive	Extend Get/MultiGet deadline support to table open (#6982)	4 years ago
block_based	Separate internal and user key comparators in `BlockIter` (#6944)	4 years ago
cuckoo	Extend Get/MultiGet deadline support to table open (#6982)	4 years ago
plain	Extend Get/MultiGet deadline support to table open (#6982)	4 years ago
block_fetcher.cc	Skip unnecessary allocation for mmap reads under 5000 bytes (#7043)	4 years ago
block_fetcher.h	Reduce memory copies when fetching and uncompressing blocks from SST files (#6689)	5 years ago
block_fetcher_test.cc	Skip unnecessary allocation for mmap reads under 5000 bytes (#7043)	4 years ago
cleanable_test.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	5 years ago
format.cc	Extend Get/MultiGet deadline support to table open (#6982)	4 years ago
format.h	Extend Get/MultiGet deadline support to table open (#6982)	4 years ago
get_context.cc	Add timestamp to delete (#6253)	5 years ago
get_context.h	Stats for redundant insertions into block cache (#6681)	5 years ago
internal_iterator.h	Properly report IO errors when IndexType::kBinarySearchWithFirstKey is used (#6621)	5 years ago
iter_heap.h	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	5 years ago
iterator.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	5 years ago
iterator_wrapper.h	Properly report IO errors when IndexType::kBinarySearchWithFirstKey is used (#6621)	5 years ago
merger_test.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	5 years ago
merging_iterator.cc	Properly report IO errors when IndexType::kBinarySearchWithFirstKey is used (#6621)	5 years ago
merging_iterator.h	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	5 years ago
meta_blocks.cc	Separate internal and user key comparators in `BlockIter` (#6944)	4 years ago
meta_blocks.h	Extend Get/MultiGet deadline support to table open (#6982)	4 years ago
mock_table.cc	Extend Get/MultiGet deadline support to table open (#6982)	4 years ago
mock_table.h	Extend Get/MultiGet deadline support to table open (#6982)	4 years ago
multiget_context.h	Allow MultiGet users to limit cumulative value size (#6826)	5 years ago
persistent_cache_helper.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	5 years ago
persistent_cache_helper.h	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	5 years ago
persistent_cache_options.h	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	5 years ago
scoped_arena_iterator.h	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	5 years ago
sst_file_dumper.cc	Extend Get/MultiGet deadline support to table open (#6982)	4 years ago
sst_file_dumper.h	Add a new option for BackupEngine to store table files under shared_checksum using DB session id in the backup filenames (#6997)	4 years ago
sst_file_reader.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	5 years ago
sst_file_reader_test.cc	Allow table/sst_file_reader_test.cc to use custom Env (#6536)	5 years ago
sst_file_writer.cc	Store DB identity and DB session ID in SST files (#6983)	4 years ago
sst_file_writer_collectors.h	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	5 years ago
table_builder.h	Store DB identity and DB session ID in SST files (#6983)	4 years ago
table_properties.cc	Store DB identity and DB session ID in SST files (#6983)	4 years ago
table_properties_internal.h	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	5 years ago
table_reader.h	For ApproximateSizes, pro-rate table metadata size over data blocks (#6784)	5 years ago
table_reader_bench.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	5 years ago
table_reader_caller.h	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	5 years ago
table_test.cc	Separate internal and user key comparators in `BlockIter` (#6944)	4 years ago
two_level_iterator.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	5 years ago
two_level_iterator.h	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	5 years ago