fork of https://github.com/oxigraph/rocksdb and https://github.com/facebook/rocksdb for nextgraph and oxigraph

You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

Peter Dillinger 5bf9a7d5ee Clarify caching behavior for index and filter partitions (#9068) ... Summary: Somewhat confusingly, index and filter partition blocks are never owned by table readers, even with cache_index_and_filter_blocks=false. They still go into block cache (possibly pinned by table reader) if there is a block cache. If no block cache, they are only loaded transiently on demand. This PR primarily clarifies the options APIs and some internal code comments. Also, this closes a hypothetical data corruption vulnerability where some but not all index partitions are pinned. I haven't been able to reproduce a case where it can happen (the failure seems to propagate to abort table open) but it's worth patching nonetheless. Fixes https://github.com/facebook/rocksdb/issues/8979 Pull Request resolved: https://github.com/facebook/rocksdb/pull/9068 Test Plan: existing tests :-/ I could cover the new code using sync points, but then I'd have to very carefully relax my `assert(false)` Reviewed By: ajkr Differential Revision: D31898284 Pulled By: pdillinger fbshipit-source-id: f2511a7d3a36bc04b627935d8e6cfea6422f98be		3 years ago
..
binary_search_index_reader.cc	Separate internal and user key comparators in `BlockIter` (#6944)	4 years ago
binary_search_index_reader.h	Extend Get/MultiGet deadline support to table open (#6982)	4 years ago
block.cc	Support timestamps in SstFileWriter (#8899)	3 years ago
block.h	Cleanup includes in dbformat.h (#8930)	3 years ago
block_based_filter_block.cc	Allow intentionally swallowed errors in BlockBasedFilterBlockReader (#8695)	3 years ago
block_based_filter_block.h	Add table properties for number of entries added to filters (#8323)	4 years ago
block_based_filter_block_test.cc	Add table properties for number of entries added to filters (#8323)	4 years ago
block_based_table_builder.cc	Improve data block construction performance (#9040)	3 years ago
block_based_table_builder.h	Cache warming blocks during flush (#8561)	3 years ago
block_based_table_factory.cc	Cleanup includes in dbformat.h (#8930)	3 years ago
block_based_table_factory.h	Cleanup includes in dbformat.h (#8930)	3 years ago
block_based_table_iterator.cc	Clean up InternalIterator upper bound logic a little bit (#7200)	4 years ago
block_based_table_iterator.h	Exclude timestamp from prefix extractor (#7668)	4 years ago
block_based_table_reader.cc	Clarify caching behavior for index and filter partitions (#9068)	3 years ago
block_based_table_reader.h	Add lowest_used_cache_tier to ImmutableDBOptions to enable or disable Secondary Cache (#9050)	3 years ago
block_based_table_reader_impl.h	Parallelize secondary cache lookup in MultiGet (#8405)	3 years ago
block_based_table_reader_test.cc	Cleanup includes in dbformat.h (#8930)	3 years ago
block_builder.cc	Improve data block construction performance (#9040)	3 years ago
block_builder.h	Improve data block construction performance (#9040)	3 years ago
block_like_traits.h	fix lru caching test and fix reference binding to null pointer (#8326)	4 years ago
block_prefetcher.cc	Improve BlockPrefetcher to prefetch only for sequential scans (#7394)	4 years ago
block_prefetcher.h	Improve BlockPrefetcher to prefetch only for sequential scans (#7394)	4 years ago
block_prefix_index.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	5 years ago
block_prefix_index.h	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	5 years ago
block_test.cc	More Makefile Cleanup (#7097)	4 years ago
block_type.h	Fix and detect headers with missing dependencies (#8893)	3 years ago
cachable_entry.h	Parallelize secondary cache lookup in MultiGet (#8405)	3 years ago
data_block_footer.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	5 years ago
data_block_footer.h	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	5 years ago
data_block_hash_index.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	5 years ago
data_block_hash_index.h	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	5 years ago
data_block_hash_index_test.cc	Make it possible to apply only a subrange of table property collectors (#8298)	4 years ago
filter_block.h	Cleanup includes in dbformat.h (#8930)	3 years ago
filter_block_reader_common.cc	Parallelize secondary cache lookup in MultiGet (#8405)	3 years ago
filter_block_reader_common.h	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	5 years ago
filter_policy.cc	Cleanup includes in dbformat.h (#8930)	3 years ago
filter_policy_internal.h	Add Bloom/Ribbon hybrid API support (#8679)	3 years ago
flush_block_policy.cc	Make FlushBlockPolicyFactory into a Customizable class (#8432)	3 years ago
flush_block_policy.h	Make FlushBlockPolicyFactory into a Customizable class (#8432)	3 years ago
full_filter_block.cc	Add table properties for number of entries added to filters (#8323)	4 years ago
full_filter_block.h	Cleanup includes in dbformat.h (#8930)	3 years ago
full_filter_block_test.cc	Add table properties for number of entries added to filters (#8323)	4 years ago
hash_index_reader.cc	Make ImmutableOptions struct that inherits from ImmutableCFOptions and ImmutableDBOptions (#8262)	4 years ago
hash_index_reader.h	Extend Get/MultiGet deadline support to table open (#6982)	4 years ago
index_builder.cc	Add (& fix) some simple source code checks (#8821)	3 years ago
index_builder.h	Make db_basic_test pass assert status checked (#7452)	4 years ago
index_reader_common.cc	Parallelize secondary cache lookup in MultiGet (#8405)	3 years ago
index_reader_common.h	Divide block_based_table_reader.cc (#6527)	5 years ago
mock_block_based_table.h	Make ImmutableOptions struct that inherits from ImmutableCFOptions and ImmutableDBOptions (#8262)	4 years ago
parsed_full_filter_block.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	5 years ago
parsed_full_filter_block.h	Use new Insert and Lookup APIs in table reader to support secondary cache (#8315)	4 years ago
partitioned_filter_block.cc	Cleanup includes in dbformat.h (#8930)	3 years ago
partitioned_filter_block.h	Clarify caching behavior for index and filter partitions (#9068)	3 years ago
partitioned_filter_block_test.cc	Make ImmutableOptions struct that inherits from ImmutableCFOptions and ImmutableDBOptions (#8262)	4 years ago
partitioned_index_iterator.cc	Fix misspelling of PartitionedIndexIterator (#7450)	4 years ago
partitioned_index_iterator.h	Fix misspelling of PartitionedIndexIterator (#7450)	4 years ago
partitioned_index_reader.cc	Clarify caching behavior for index and filter partitions (#9068)	3 years ago
partitioned_index_reader.h	Clarify caching behavior for index and filter partitions (#9068)	3 years ago
reader_common.cc	Fix block checksum for >=4GB, refactor (#6978)	4 years ago
reader_common.h	Bring the Configurable options together (#5753)	4 years ago
uncompression_dict_reader.cc	Cleanup includes in dbformat.h (#8930)	3 years ago
uncompression_dict_reader.h	Extend Get/MultiGet deadline support to table open (#6982)	4 years ago