fork of https://github.com/oxigraph/rocksdb and https://github.com/facebook/rocksdb for nextgraph and oxigraph

You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

Yi Wu eb287c72d7 Fix wrong key being read on ingested file with global seqno and delta encoding (#6669) ... Summary: On reading an ingested SST file, `DataBlockIter` will replace seqno encoded in a key with global seqno. However, if the original seqno was part of the prefix used for the next key, the global seqno is by mistake used as part of the prefix to construct the next key, causing wrong result being returned. Although at this point it is only software error while data in the file is not corrupted, the issue can further cause compaction output out of order and corrupted result when the ingested SST participated in compaction. Fixing the issue by save the actual seqno and restore it before the key being used as prefix to construct next key. The unit test is by Little-Wallace from https://github.com/facebook/rocksdb/issues/6666. Fixing https://github.com/facebook/rocksdb/issues/6666. Pull Request resolved: https://github.com/facebook/rocksdb/pull/6669 Test Plan: New unit test Signed-off-by: Yi Wu <yiwu@pingcap.com> Reviewed By: cheng-chang Differential Revision: D20931808 Pulled By: ajkr fbshipit-source-id: f01959c35d6a493954dca981663766c7a5a9e8ab		4 years ago
..
blob	Add blob files to VersionStorageInfo/VersionBuilder (#6597)	4 years ago
compaction	Add pipelined & parallel compression optimization (#6262)	4 years ago
db_impl	Hex encode keys in compaction flush logs (#6616)	4 years ago
arena_wrapped_db_iter.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	4 years ago
arena_wrapped_db_iter.h	Iterator with timestamp (#6255)	4 years ago
builder.cc	In block based table builder, make variables for estimating file size atomic (#6636)	4 years ago
builder.h	Pass IOStatus to write path and set retryable IO Error as hard error in BG jobs (#6487)	4 years ago
c.cc	added new functions to c-api (#5630)	4 years ago
c_test.c	Return NotSupported from WriteBatchWithIndex::DeleteRange (#5393)	4 years ago
column_family.cc	Attempt to recover from db with missing table files (#6334)	4 years ago
column_family.h	Attempt to recover from db with missing table files (#6334)	4 years ago
column_family_test.cc	fix some spelling typos (#6464)	4 years ago
compact_files_test.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	4 years ago
compacted_db_impl.cc	return timestamp from get (#6409)	4 years ago
compacted_db_impl.h	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	4 years ago
comparator_db_test.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	4 years ago
convenience.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	4 years ago
corruption_test.cc	Simplify migration to FileSystem API (#6552)	4 years ago
cuckoo_table_db_test.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	4 years ago
db_basic_test.cc	Add pipelined & parallel compression optimization (#6262)	4 years ago
db_block_cache_test.cc	Revert the recent cache deleter change (#6620)	4 years ago
db_bloom_filter_test.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	4 years ago
db_compaction_filter_test.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	4 years ago
db_compaction_test.cc	Use flush time for the props.creation_time for FIFO compaction (#6612)	4 years ago
db_dynamic_level_test.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	4 years ago
db_encryption_test.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	4 years ago
db_filesnapshot.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	4 years ago
db_flush_test.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	4 years ago
db_info_dumper.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	4 years ago
db_info_dumper.h	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	4 years ago
db_inplace_update_test.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	4 years ago
db_io_failure_test.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	4 years ago
db_iter.cc	Iterator with timestamp (#6255)	4 years ago
db_iter.h	Iterator with timestamp (#6255)	4 years ago
db_iter_stress_test.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	4 years ago
db_iter_test.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	4 years ago
db_iterator_test.cc	fix some spelling typos (#6464)	4 years ago
db_log_iter_test.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	4 years ago
db_logical_block_size_cache_test.cc	Get block size only in direct IO mode (#6522)	4 years ago
db_memtable_test.cc	return timestamp from get (#6409)	4 years ago
db_merge_operand_test.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	4 years ago
db_merge_operator_test.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	4 years ago
db_options_test.cc	Move the OptionTypeMap code closer to home (#6198)	4 years ago
db_properties_test.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	4 years ago
db_range_del_test.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	4 years ago
db_sst_test.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	4 years ago
db_statistics_test.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	4 years ago
db_table_properties_test.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	4 years ago
db_tailing_iter_test.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	4 years ago
db_test.cc	Migrate AppVeyor to CircleCI (#6518)	4 years ago
db_test2.cc	Add pipelined & parallel compression optimization (#6262)	4 years ago
db_test_util.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	4 years ago
db_test_util.h	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	4 years ago
db_universal_compaction_test.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	4 years ago
db_wal_test.cc	Fix db_wal_test::TruncateLastLogAfterRecoverWithoutFlush failure (#6437)	4 years ago
db_with_timestamp_basic_test.cc	Add pipelined & parallel compression optimization (#6262)	4 years ago
db_write_test.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	4 years ago
dbformat.cc	Iterator with timestamp (#6255)	4 years ago
dbformat.h	Iterator with timestamp (#6255)	4 years ago
dbformat_test.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	4 years ago
deletefile_test.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	4 years ago
error_handler.cc	Be able to decrease background thread's CPU priority when creating database backup (#6602)	4 years ago
error_handler.h	Pass IOStatus to write path and set retryable IO Error as hard error in BG jobs (#6487)	4 years ago
error_handler_fs_test.cc	Revamp cache_bench to resemble a real workload (#6629)	4 years ago
event_helpers.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	4 years ago
event_helpers.h	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	4 years ago
experimental.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	4 years ago
external_sst_file_basic_test.cc	Fix range deletion tombstone ingestion with global seqno (#6429)	4 years ago
external_sst_file_ingestion_job.cc	Replace Directory with FSDirectory in DB (#6468)	4 years ago
external_sst_file_ingestion_job.h	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	4 years ago
external_sst_file_test.cc	Fix wrong key being read on ingested file with global seqno and delta encoding (#6669)	4 years ago
fault_injection_test.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	4 years ago
file_indexer.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	4 years ago
file_indexer.h	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	4 years ago
file_indexer_test.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	4 years ago
filename_test.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	4 years ago
flush_job.cc	Make options.bottommost_compression, compression_opts and bottommost_compression_opts dynamically changeable. (#6615)	4 years ago
flush_job.h	Pass IOStatus to write path and set retryable IO Error as hard error in BG jobs (#6487)	4 years ago
flush_job_test.cc	Fix initializer syntax for old Xcode compiler (#6662)	4 years ago
flush_scheduler.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	4 years ago
flush_scheduler.h	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	4 years ago
forward_iterator.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	4 years ago
forward_iterator.h	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	4 years ago
forward_iterator_bench.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	4 years ago
import_column_family_job.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	4 years ago
import_column_family_job.h	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	4 years ago
import_column_family_test.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	4 years ago
internal_stats.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	4 years ago
internal_stats.h	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	4 years ago
job_context.h	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	4 years ago
listener_test.cc	Move BlobDB related files under db/ to db/blob/ (#6519)	4 years ago
log_format.h	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	4 years ago
log_reader.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	4 years ago
log_reader.h	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	4 years ago
log_test.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	4 years ago
log_writer.cc	Pass IOStatus to write path and set retryable IO Error as hard error in BG jobs (#6487)	4 years ago
log_writer.h	Pass IOStatus to write path and set retryable IO Error as hard error in BG jobs (#6487)	4 years ago
logs_with_prep_tracker.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	4 years ago
logs_with_prep_tracker.h	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	4 years ago
lookup_key.h	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	4 years ago
malloc_stats.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	4 years ago
malloc_stats.h	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	4 years ago
manual_compaction_test.cc	Skip high levels with no key falling in the range in CompactRange (#6482)	4 years ago
memtable.cc	multiget support for timestamps (#6483)	4 years ago
memtable.h	return timestamp from get (#6409)	4 years ago
memtable_list.cc	Pass IOStatus to write path and set retryable IO Error as hard error in BG jobs (#6487)	4 years ago
memtable_list.h	Pass IOStatus to write path and set retryable IO Error as hard error in BG jobs (#6487)	4 years ago
memtable_list_test.cc	Pass IOStatus to write path and set retryable IO Error as hard error in BG jobs (#6487)	4 years ago
merge_context.h	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	4 years ago
merge_helper.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	4 years ago
merge_helper.h	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	4 years ago
merge_helper_test.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	4 years ago
merge_operator.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	4 years ago
merge_test.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	4 years ago
obsolete_files_test.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	4 years ago
options_file_test.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	4 years ago
perf_context_test.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	4 years ago
pinned_iterators_manager.h	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	4 years ago
plain_table_db_test.cc	Add ppc64le builds to Travis (#6144)	4 years ago
pre_release_callback.h	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	4 years ago
prefix_test.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	4 years ago
range_del_aggregator.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	4 years ago
range_del_aggregator.h	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	4 years ago
range_del_aggregator_bench.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	4 years ago
range_del_aggregator_test.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	4 years ago
range_tombstone_fragmenter.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	4 years ago
range_tombstone_fragmenter.h	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	4 years ago
range_tombstone_fragmenter_test.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	4 years ago
read_callback.h	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	4 years ago
repair.cc	Pass IOStatus to write path and set retryable IO Error as hard error in BG jobs (#6487)	4 years ago
repair_test.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	4 years ago
snapshot_checker.h	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	4 years ago
snapshot_impl.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	4 years ago
snapshot_impl.h	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	4 years ago
table_cache.cc	Revert the recent cache deleter change (#6620)	4 years ago
table_cache.h	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	4 years ago
table_properties_collector.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	4 years ago
table_properties_collector.h	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	4 years ago
table_properties_collector_test.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	4 years ago
transaction_log_impl.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	4 years ago
transaction_log_impl.h	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	4 years ago
trim_history_scheduler.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	4 years ago
trim_history_scheduler.h	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	4 years ago
version_builder.cc	Add blob files to VersionStorageInfo/VersionBuilder (#6597)	4 years ago
version_builder.h	Attempt to recover from db with missing table files (#6334)	4 years ago
version_builder_test.cc	Add blob files to VersionStorageInfo/VersionBuilder (#6597)	4 years ago
version_edit.cc	Revert "Added the safe-to-ignore tag to version_edit (#6530)" (#6569)	4 years ago
version_edit.h	Revert "Added the safe-to-ignore tag to version_edit (#6530)" (#6569)	4 years ago
version_edit_handler.cc	Attempt to recover from db with missing table files (#6334)	4 years ago
version_edit_handler.h	Attempt to recover from db with missing table files (#6334)	4 years ago
version_edit_test.cc	Revert "Added the safe-to-ignore tag to version_edit (#6530)" (#6569)	4 years ago
version_set.cc	Re-persist blob file metadata when a new manifest file is created (#6630)	4 years ago
version_set.h	Pass IOStatus to write path and set retryable IO Error as hard error in BG jobs (#6487)	4 years ago
version_set_test.cc	Re-persist blob file metadata when a new manifest file is created (#6630)	4 years ago
wal_manager.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	4 years ago
wal_manager.h	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	4 years ago
wal_manager_test.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	4 years ago
write_batch.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	4 years ago
write_batch_base.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	4 years ago
write_batch_internal.h	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	4 years ago
write_batch_test.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	4 years ago
write_callback.h	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	4 years ago
write_callback_test.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	4 years ago
write_controller.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	4 years ago
write_controller.h	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	4 years ago
write_controller_test.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	4 years ago
write_thread.cc	fix some spelling typos (#6464)	4 years ago
write_thread.h	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	4 years ago