fork of https://github.com/oxigraph/rocksdb and https://github.com/facebook/rocksdb for nextgraph and oxigraph

You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

Andrew Kryczka c5abf78bca avoid `IterKey::UpdateInternalKey()` in `BlockIter` (#6843) ... Summary: `IterKey::UpdateInternalKey()` is an error-prone API as it's incompatible with `IterKey::TrimAppend()`, which is used for decoding delta-encoded internal keys. This PR stops using it in `BlockIter`. Instead, it assigns global seqno in a separate `IterKey`'s buffer when needed. The logic for safely getting a Slice with global seqno properly assigned is encapsulated in `GlobalSeqnoAppliedKey`. `BinarySeek()` is also migrated to use this API (previously it ignored global seqno entirely). Pull Request resolved: https://github.com/facebook/rocksdb/pull/6843 Test Plan: benchmark setup -- single file DBs, in-memory, no compression. "normal_db" created by regular flush; "ingestion_db" created by ingesting a file. Both DBs have same contents. ``` $ TEST_TMPDIR=/dev/shm/normal_db/ ./db_bench -benchmarks=fillrandom,compact -write_buffer_size=10485760000 -disable_auto_compactions=true -compression_type=none -num=1000000 $ ./ldb write_extern_sst ./tmp.sst --db=/dev/shm/ingestion_db/dbbench/ --compression_type=no --hex --create_if_missing < <(./sst_dump --command=scan --output_hex --file=/dev/shm/normal_db/dbbench/000007.sst | awk 'began {print "0x" substr($1, 2, length($1) - 2), "==>", "0x" $5} ; /^Sst file format: block-based/ {began=1}') $ ./ldb ingest_extern_sst ./tmp.sst --db=/dev/shm/ingestion_db/dbbench/ ``` benchmark run command: ``` TEST_TMPDIR=/dev/shm/$DB/ ./db_bench -benchmarks=seekrandom -seek_nexts=10 -use_existing_db=true -cache_index_and_filter_blocks=false -num=1000000 -cache_size=1048576000 -threads=1 -reads=40000000 ``` results: | DB | code | throughput | |---|---|---| | normal_db | master | 267.9 | | normal_db | PR6843 | 254.2 (-5.1%) | | ingestion_db | master | 259.6 | | ingestion_db | PR6843 | 250.5 (-3.5%) | Reviewed By: pdillinger Differential Revision: D21562604 Pulled By: ajkr fbshipit-source-id: 937596f836930515da8084d11755e1f247dcb264		5 years ago
..
adaptive	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	5 years ago
block_based	avoid `IterKey::UpdateInternalKey()` in `BlockIter` (#6843)	5 years ago
cuckoo	Pass a timeout to FileSystem for random reads (#6751)	5 years ago
plain	Add Struct Type to OptionsTypeInfo (#6425)	5 years ago
block_fetcher.cc	Pass a timeout to FileSystem for random reads (#6751)	5 years ago
block_fetcher.h	Reduce memory copies when fetching and uncompressing blocks from SST files (#6689)	5 years ago
block_fetcher_test.cc	skip direct I/O tests in rocksdb lite (#6867)	5 years ago
cleanable_test.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	5 years ago
format.cc	Add tests for compression failure in BlockBasedTableBuilder (#6709)	5 years ago
format.h	Basic MultiGet support for partitioned filters (#6757)	5 years ago
get_context.cc	Add timestamp to delete (#6253)	5 years ago
get_context.h	Stats for redundant insertions into block cache (#6681)	5 years ago
internal_iterator.h	Properly report IO errors when IndexType::kBinarySearchWithFirstKey is used (#6621)	5 years ago
iter_heap.h	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	5 years ago
iterator.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	5 years ago
iterator_wrapper.h	Properly report IO errors when IndexType::kBinarySearchWithFirstKey is used (#6621)	5 years ago
merger_test.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	5 years ago
merging_iterator.cc	Properly report IO errors when IndexType::kBinarySearchWithFirstKey is used (#6621)	5 years ago
merging_iterator.h	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	5 years ago
meta_blocks.cc	sst_dump to reduce number of file reads (#6836)	5 years ago
meta_blocks.h	sst_dump to reduce number of file reads (#6836)	5 years ago
mock_table.cc	Pass a timeout to FileSystem for random reads (#6751)	5 years ago
mock_table.h	Properly report IO errors when IndexType::kBinarySearchWithFirstKey is used (#6621)	5 years ago
multiget_context.h	Allow MultiGet users to limit cumulative value size (#6826)	5 years ago
persistent_cache_helper.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	5 years ago
persistent_cache_helper.h	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	5 years ago
persistent_cache_options.h	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	5 years ago
scoped_arena_iterator.h	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	5 years ago
sst_file_reader.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	5 years ago
sst_file_reader_test.cc	Allow table/sst_file_reader_test.cc to use custom Env (#6536)	5 years ago
sst_file_writer.cc	Generate file checksum in SstFileWriter (#6859)	5 years ago
sst_file_writer_collectors.h	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	5 years ago
table_builder.h	sst_dump to reduce number of file reads (#6836)	5 years ago
table_properties.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	5 years ago
table_properties_internal.h	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	5 years ago
table_reader.h	Properly report IO errors when IndexType::kBinarySearchWithFirstKey is used (#6621)	5 years ago
table_reader_bench.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	5 years ago
table_reader_caller.h	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	5 years ago
table_test.cc	Clean up some code related to file checksums (#6861)	5 years ago
two_level_iterator.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	5 years ago
two_level_iterator.h	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	5 years ago