fork of https://github.com/oxigraph/rocksdb and https://github.com/facebook/rocksdb for nextgraph and oxigraph

You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

Andrew Kryczka fea2b1dfb2 Copy Get() result when file reads use mmap ... Summary: For iterator reads, a `SuperVersion` is pinned to preserve a snapshot of SST files, and `Block`s are pinned to allow `key()` and `value()` to return pointers directly into a RocksDB memory region. This works for both non-mmap reads, where the block owns the memory region, and mmap reads, where the file owns the memory region. For point reads with `PinnableSlice`, only the `Block` object is pinned. This works for non-mmap reads because the block owns the memory region, so even if the file is deleted after compaction, the memory region survives. However, for mmap reads, file deletion causes the memory region to which the `PinnableSlice` refers to be unmapped. The result is usually a segfault upon accessing the `PinnableSlice`, although sometimes it returned wrong results (I repro'd this a bunch of times with `db_stress`). This PR copies the value into the `PinnableSlice` when it comes from mmap'd memory. We can tell whether the `Block` owns its memory using `Block::cachable()`, which is unset when reads do not use the provided buffer as is the case with mmap file reads. When that is false we ensure the result of `Get()` is copied. This feels like a short-term solution as ideally we'd have the `PinnableSlice` pin the mmap'd memory so we can do zero-copy reads. It seemed hard so I chose this approach to fix correctness in the meantime. Closes https://github.com/facebook/rocksdb/pull/3881 Differential Revision: D8076288 Pulled By: ajkr fbshipit-source-id: 31d78ec010198723522323dbc6ea325122a46b08		7 years ago
..
adaptive_table_factory.cc	Comment out unused variables	7 years ago
adaptive_table_factory.h	Comment out unused variables	7 years ago
block.cc	Copy Get() result when file reads use mmap	7 years ago
block.h	Copy Get() result when file reads use mmap	7 years ago
block_based_filter_block.cc	Move prefix_extractor to MutableCFOptions	7 years ago
block_based_filter_block.h	Move prefix_extractor to MutableCFOptions	7 years ago
block_based_filter_block_test.cc	Move prefix_extractor to MutableCFOptions	7 years ago
block_based_table_builder.cc	Exclude seq from index keys	7 years ago
block_based_table_builder.h	Move prefix_extractor to MutableCFOptions	7 years ago
block_based_table_factory.cc	Move prefix_extractor to MutableCFOptions	7 years ago
block_based_table_factory.h	Align SST file data blocks to avoid spanning multiple pages	7 years ago
block_based_table_reader.cc	Copy Get() result when file reads use mmap	7 years ago
block_based_table_reader.h	Exclude seq from index keys	7 years ago
block_builder.cc	Change RocksDB License	8 years ago
block_builder.h	Change RocksDB License	8 years ago
block_fetcher.cc	Imporve perf of random read and insert compare by suggesting inlining to the compiler	7 years ago
block_fetcher.h	Fix BlockFetcher ASAN error	7 years ago
block_prefix_index.cc	Change RocksDB License	8 years ago
block_prefix_index.h	Change RocksDB License	8 years ago
block_test.cc	Exclude seq from index keys	7 years ago
bloom_block.cc	Change RocksDB License	8 years ago
bloom_block.h	Change RocksDB License	8 years ago
cleanable_test.cc	Change RocksDB License	8 years ago
cuckoo_table_builder.cc	Enable MSVC W4 with a few exceptions. Fix warnings and bugs	7 years ago
cuckoo_table_builder.h	Change RocksDB License	8 years ago
cuckoo_table_builder_test.cc	fix memory leak in two_level_iterator	7 years ago
cuckoo_table_factory.cc	Comment out unused variables	7 years ago
cuckoo_table_factory.h	comment unused parameters to turn on -Wunused-parameter flag	7 years ago
cuckoo_table_reader.cc	Move prefix_extractor to MutableCFOptions	7 years ago
cuckoo_table_reader.h	Move prefix_extractor to MutableCFOptions	7 years ago
cuckoo_table_reader_test.cc	Move prefix_extractor to MutableCFOptions	7 years ago
filter_block.h	Move prefix_extractor to MutableCFOptions	7 years ago
flush_block_policy.cc	Align SST file data blocks to avoid spanning multiple pages	7 years ago
format.cc	Several small "fixes"	7 years ago
format.h	comment unused parameters to turn on -Wunused-parameter flag	7 years ago
full_filter_bits_builder.h	Skip duplicate bloom keys when whole_key and prefix are mixed	7 years ago
full_filter_block.cc	Move prefix_extractor to MutableCFOptions	7 years ago
full_filter_block.h	Move prefix_extractor to MutableCFOptions	7 years ago
full_filter_block_test.cc	Move prefix_extractor to MutableCFOptions	7 years ago
get_context.cc	comment unused parameters to turn on -Wunused-parameter flag	7 years ago
get_context.h	Stats for false positive rate of full filtesr	7 years ago
index_builder.cc	Exclude seq from index keys	7 years ago
index_builder.h	Exclude seq from index keys	7 years ago
internal_iterator.h	Change and clarify the relationship between Valid(), status() and Seek*() for all iterators. Also fix some bugs	7 years ago
iter_heap.h	Make InternalKeyComparator final and directly use it in merging iterator	7 years ago
iterator.cc	Comment out unused variables	7 years ago
iterator_wrapper.h	Change and clarify the relationship between Valid(), status() and Seek*() for all iterators. Also fix some bugs	7 years ago
merger_test.cc	Make InternalKeyComparator final and directly use it in merging iterator	7 years ago
merging_iterator.cc	Change and clarify the relationship between Valid(), status() and Seek*() for all iterators. Also fix some bugs	7 years ago
merging_iterator.h	fix DBImpl::NewInternalIterator super-version leak on failure	7 years ago
meta_blocks.cc	Exclude seq from index keys	7 years ago
meta_blocks.h	Support prefetch last 512KB with direct I/O in block based file reader	7 years ago
mock_table.cc	Move prefix_extractor to MutableCFOptions	7 years ago
mock_table.h	Move prefix_extractor to MutableCFOptions	7 years ago
partitioned_filter_block.cc	Exclude seq from index keys	7 years ago
partitioned_filter_block.h	Move prefix_extractor to MutableCFOptions	7 years ago
partitioned_filter_block_test.cc	Move prefix_extractor to MutableCFOptions	7 years ago
persistent_cache_helper.cc	comment unused parameters to turn on -Wunused-parameter flag	7 years ago
persistent_cache_helper.h	Change RocksDB License	8 years ago
persistent_cache_options.h	Change RocksDB License	8 years ago
plain_table_builder.cc	Move prefix_extractor to MutableCFOptions	7 years ago
plain_table_builder.h	Move prefix_extractor to MutableCFOptions	7 years ago
plain_table_factory.cc	Move prefix_extractor to MutableCFOptions	7 years ago
plain_table_factory.h	Comment out unused variables	7 years ago
plain_table_index.cc	Change RocksDB License	8 years ago
plain_table_index.h	Move prefix_extractor to MutableCFOptions	7 years ago
plain_table_key_coding.cc	Comment out unused variables	7 years ago
plain_table_key_coding.h	Change RocksDB License	8 years ago
plain_table_reader.cc	Move prefix_extractor to MutableCFOptions	7 years ago
plain_table_reader.h	Move prefix_extractor to MutableCFOptions	7 years ago
scoped_arena_iterator.h	Change RocksDB License	8 years ago
sst_file_writer.cc	Move prefix_extractor to MutableCFOptions	7 years ago
sst_file_writer_collectors.h	Comment out unused variables	7 years ago
table_builder.h	Move prefix_extractor to MutableCFOptions	7 years ago
table_properties.cc	Exclude seq from index keys	7 years ago
table_properties_internal.h	Eliminate some redundant block reads.	7 years ago
table_reader.h	Move prefix_extractor to MutableCFOptions	7 years ago
table_reader_bench.cc	Move prefix_extractor to MutableCFOptions	7 years ago
table_test.cc	Exclude seq from index keys	7 years ago
two_level_iterator.cc	Change and clarify the relationship between Valid(), status() and Seek*() for all iterators. Also fix some bugs	7 years ago
two_level_iterator.h	fix memory leak in two_level_iterator	7 years ago