rocksdb

fork of https://github.com/oxigraph/rocksdb and https://github.com/facebook/rocksdb for nextgraph and oxigraph

History

Yanqin Jin 9b53f14a35 Fixed a bug in CompactionIterator when write-preared transaction is used (#9060 ) Summary: Pull Request resolved: https://github.com/facebook/rocksdb/pull/9060 RocksDB bottommost level compaction may zero out an internal key's sequence if the key's sequence is in the earliest_snapshot. In write-prepared transaction, checking the visibility of a certain sequence in a specific released snapshot may return a "snapshot released" result. Therefore, it is possible, after a certain sequence of events, a PUT has its sequence zeroed out, but a subsequent SingleDelete of the same key will still be output with its original sequence. This violates the ascending order of keys and leads to incorrect result. The solution is to use an extra variable `last_key_seq_zeroed_` to track the information about visibility in earliest snapshot. With this variable, we can know for sure that a SingleDelete is in the earliest snapshot even if the said snapshot is released during compaction before processing the SD. Reviewed By: ltamasi Differential Revision: D31813016 fbshipit-source-id: d8cff59d6f34e0bdf282614034aaea99be9174e1		3 years ago
..
blob	Cleanup multiple implementations of VectorIterator (#8901 )	3 years ago
compaction	Fixed a bug in CompactionIterator when write-preared transaction is used (#9060 )	3 years ago
db_impl	Skip directory fsync for filesystem btrfs (#8903 )	3 years ago
arena_wrapped_db_iter.cc	Rename ImmutableOptions variables (#8409 )	3 years ago
arena_wrapped_db_iter.h	Cleanup includes in dbformat.h (#8930 )	3 years ago
builder.cc	Don't call OnTableFileCreated with OK for empty+deleted file (#9118 )	3 years ago
builder.h	Expose blob file information through the EventListener interface (#8675 )	3 years ago
c.cc	Make it possible to force the garbage collection of the oldest blob files (#8994 )	3 years ago
c_test.c	Make it possible to force the garbage collection of the oldest blob files (#8994 )	3 years ago
column_family.cc	Make it possible to force the garbage collection of the oldest blob files (#8994 )	3 years ago
column_family.h	Add DB properties for BlobDB (#8734 )	3 years ago
column_family_test.cc	Make it possible to force the garbage collection of the oldest blob files (#8994 )	3 years ago
compact_files_test.cc	Compaction should not move data to up level (#8116 )	4 years ago
comparator_db_test.cc	Remove unused includes (#7604 )	4 years ago
convenience.cc	Make ImmutableOptions struct that inherits from ImmutableCFOptions and ImmutableDBOptions (#8262 )	4 years ago
corruption_test.cc	Add CreateFrom methods to Env/FileSystem (#8174 )	3 years ago
cuckoo_table_db_test.cc	Experimental support for SST unique IDs (#8990 )	3 years ago
db_basic_test.cc	Make FileSystem a Customizable Class (#8649 )	3 years ago
db_block_cache_test.cc	Fix PrepopulateBlockCache::kFlushOnly (#8750 )	3 years ago
db_bloom_filter_test.cc	Make SliceTransform into a Customizable class (#8641 )	3 years ago
db_compaction_filter_test.cc	Fix a minor issue with initializing the test path (#8555 )	3 years ago
db_compaction_test.cc	Try to start TTL earlier with kMinOverlappingRatio is used (#8749 )	3 years ago
db_dynamic_level_test.cc	Make SystemClock into a Customizable Class (#8636 )	3 years ago
db_encryption_test.cc	Fix a minor issue with initializing the test path (#8555 )	3 years ago
db_filesnapshot.cc	Add (Live)FileStorageInfo API (#8968 )	3 years ago
db_flush_test.cc	Fix TSAN report on MemPurge test (#9115 )	3 years ago
db_info_dumper.cc	Allow WAL dir to change with db dir (#8582 )	3 years ago
db_info_dumper.h	Add a DB Session ID (#6959 )	4 years ago
db_inplace_update_test.cc	Fix a minor issue with initializing the test path (#8555 )	3 years ago
db_io_failure_test.cc	Make MemTableRepFactory into a Customizable class (#8419 )	3 years ago
db_iter.cc	Rename ImmutableOptions variables (#8409 )	3 years ago
db_iter.h	Cleanup includes in dbformat.h (#8930 )	3 years ago
db_iter_stress_test.cc	Make ImmutableOptions struct that inherits from ImmutableCFOptions and ImmutableDBOptions (#8262 )	4 years ago
db_iter_test.cc	Remove some unneeded code (#8736 )	3 years ago
db_iterator_test.cc	Fix a minor issue with initializing the test path (#8555 )	3 years ago
db_kv_checksum_test.cc	Allow WriteBatch to have keys with different timestamp sizes (#8725 )	3 years ago
db_log_iter_test.cc	Attempt to deflake DBTestXactLogIterator.TransactionLogIteratorCorruptedLog (#8627 )	3 years ago
db_logical_block_size_cache_test.cc	Add further tests to ASSERT_STATUS_CHECKED (2) (#7698 )	4 years ago
db_memtable_test.cc	Fix a minor issue with initializing the test path (#8555 )	3 years ago
db_merge_operand_test.cc	Fix a minor issue with initializing the test path (#8555 )	3 years ago
db_merge_operator_test.cc	Fix a minor issue with initializing the test path (#8555 )	3 years ago
db_options_test.cc	Fix a minor issue with initializing the test path (#8555 )	3 years ago
db_properties_test.cc	Add support for building on s390x platform (#8962 )	3 years ago
db_range_del_test.cc	Incremental Space Amp Compactions in Universal Style (#8655 )	3 years ago
db_secondary_test.cc	Skip directory fsync for filesystem btrfs (#8903 )	3 years ago
db_sst_test.cc	Don't ignore deletion rate limit if WAL dir is different (#8967 )	3 years ago
db_statistics_test.cc	Bytes read stat for `VerifyChecksum()` and `VerifyFileChecksums()` APIs (#8741 )	3 years ago
db_table_properties_test.cc	Experimental support for SST unique IDs (#8990 )	3 years ago
db_tailing_iter_test.cc	Fix a minor issue with initializing the test path (#8555 )	3 years ago
db_test.cc	Experimental support for SST unique IDs (#8990 )	3 years ago
db_test2.cc	Fix out-of-bounds access in MultiDBParallelOpenTest (#9046 )	3 years ago
db_test_util.cc	Implement XXH3 block checksum type (#9069 )	3 years ago
db_test_util.h	Implement XXH3 block checksum type (#9069 )	3 years ago
db_universal_compaction_test.cc	Make SystemClock into a Customizable Class (#8636 )	3 years ago
db_wal_test.cc	Fix a minor issue with initializing the test path (#8555 )	3 years ago
db_with_timestamp_basic_test.cc	Support SingleDelete for user-defined timestamps (#8921 )	3 years ago
db_with_timestamp_compaction_test.cc	Make MemTableRepFactory into a Customizable class (#8419 )	3 years ago
db_write_buffer_manager_test.cc	Fix a minor issue with initializing the test path (#8555 )	3 years ago
db_write_test.cc	Fix a minor issue with initializing the test path (#8555 )	3 years ago
dbformat.cc	Cleanup includes in dbformat.h (#8930 )	3 years ago
dbformat.h	Cleanup includes in dbformat.h (#8930 )	3 years ago
dbformat_test.cc	Remove unused includes (#7604 )	4 years ago
deletefile_test.cc	Skip directory fsync for filesystem btrfs (#8903 )	3 years ago
error_handler.cc	DB::GetSortedWalFiles() to ensure file deletion is disabled (#8591 )	3 years ago
error_handler.h	Fix some typos in comments (#8066 )	4 years ago
error_handler_fs_test.cc	Remove some unneeded code (#8736 )	3 years ago
event_helpers.cc	Expose blob file information through the EventListener interface (#8675 )	3 years ago
event_helpers.h	Expose blob file information through the EventListener interface (#8675 )	3 years ago
experimental.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433 )	5 years ago
external_sst_file_basic_test.cc	Protect existing files in `FaultInjectionTest{Env,FS}::ReopenWritableFile()` (#8995 )	3 years ago
external_sst_file_ingestion_job.cc	Skip directory fsync for filesystem btrfs (#8903 )	3 years ago
external_sst_file_ingestion_job.h	Fix sequence number bump logic in multi-CF SST ingestion (#9005 )	3 years ago
external_sst_file_test.cc	Implement XXH3 block checksum type (#9069 )	3 years ago
fault_injection_test.cc	Make SystemClock into a Customizable Class (#8636 )	3 years ago
file_indexer.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433 )	5 years ago
file_indexer.h	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433 )	5 years ago
file_indexer_test.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433 )	5 years ago
filename_test.cc	Remove unused includes (#7604 )	4 years ago
flush_job.cc	Skip directory fsync for filesystem btrfs (#8903 )	3 years ago
flush_job.h	Cleanup includes in dbformat.h (#8930 )	3 years ago
flush_job_test.cc	Fix NotifyOnFlushCompleted() for atomic flush (#8585 )	3 years ago
flush_scheduler.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433 )	5 years ago
flush_scheduler.h	Include C++ standard library headers instead of C compatibility headers (#8068 )	4 years ago
forward_iterator.cc	Fix some typos in comments (#8066 )	4 years ago
forward_iterator.h	Cleanup includes in dbformat.h (#8930 )	3 years ago
forward_iterator_bench.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433 )	5 years ago
import_column_family_job.cc	Cleanup includes in dbformat.h (#8930 )	3 years ago
import_column_family_job.h	Cleanup includes in dbformat.h (#8930 )	3 years ago
import_column_family_test.cc	Fix a minor issue with initializing the test path (#8555 )	3 years ago
internal_stats.cc	InternalStats::DumpCFMapStat: fix sum.w_amp (#9065 )	3 years ago
internal_stats.h	Support `GetMapProperty()` with "rocksdb.dbstats" (#9057 )	3 years ago
job_context.h	Fix and detect headers with missing dependencies (#8893 )	3 years ago
kv_checksum.h	Fix some lint warnings reported on 6.25 (#8945 )	3 years ago
listener_test.cc	Don't call OnTableFileCreated with OK for empty+deleted file (#9118 )	3 years ago
log_format.h	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433 )	5 years ago
log_reader.cc	Fix kPointInTimeRecovery handling of truncated WAL (#7701 )	4 years ago
log_reader.h	Real fix for race in backup custom checksum checking (#7309 )	4 years ago
log_test.cc	Make StringEnv, StringSink, StringSource use FS classes (#7786 )	4 years ago
log_writer.cc	Using existing crc32c checksum in checksum handoff for Manifest and WAL (#8412 )	3 years ago
log_writer.h	Include C++ standard library headers instead of C compatibility headers (#8068 )	4 years ago
logs_with_prep_tracker.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433 )	5 years ago
logs_with_prep_tracker.h	Include C++ standard library headers instead of C compatibility headers (#8068 )	4 years ago
lookup_key.h	Cleanup includes in dbformat.h (#8930 )	3 years ago
malloc_stats.cc	Replace most typedef with using= (#8751 )	3 years ago
malloc_stats.h	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433 )	5 years ago
manual_compaction_test.cc	Add more tests for assert status checked (#7524 )	4 years ago
memtable.cc	Cleanup includes in dbformat.h (#8930 )	3 years ago
memtable.h	Adapt key-value checksum for timestamp-suffixed keys (#8914 )	3 years ago
memtable_list.cc	Cleanup includes in dbformat.h (#8930 )	3 years ago
memtable_list.h	Cleanup includes in dbformat.h (#8930 )	3 years ago
memtable_list_test.cc	Fix NotifyOnFlushCompleted() for atomic flush (#8585 )	3 years ago
merge_context.h	Add Merge Operator support to WriteBatchWithIndex (#8135 )	4 years ago
merge_helper.cc	Add support for Merge with base value during Compaction in IntegratedBlobDB (#8445 )	3 years ago
merge_helper.h	Cleanup includes in dbformat.h (#8930 )	3 years ago
merge_helper_test.cc	Cleanup multiple implementations of VectorIterator (#8901 )	3 years ago
merge_operator.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433 )	5 years ago
merge_test.cc	Built-in support for generating unique IDs, bug fix (#8708 )	3 years ago
obsolete_files_test.cc	Deflaky ObsoleteFilesTest (#9049 )	3 years ago
options_file_test.cc	No elide constructors (#7798 )	4 years ago
output_validator.cc	Cleanup includes in dbformat.h (#8930 )	3 years ago
output_validator.h	Cleanup includes in dbformat.h (#8930 )	3 years ago
perf_context_test.cc	Use SystemClock* instead of std::shared_ptr<SystemClock> in lower level routines (#8033 )	4 years ago
periodic_work_scheduler.cc	Use SystemClock* instead of std::shared_ptr<SystemClock> in lower level routines (#8033 )	4 years ago
periodic_work_scheduler.h	Add a SystemClock class to capture the time functions of an Env (#7858 )	4 years ago
periodic_work_scheduler_test.cc	Fix a minor issue with initializing the test path (#8555 )	3 years ago
pinned_iterators_manager.h	Replace most typedef with using= (#8751 )	3 years ago
plain_table_db_test.cc	Remove some unneeded code (#8736 )	3 years ago
pre_release_callback.h	Fix and detect headers with missing dependencies (#8893 )	3 years ago
prefix_test.cc	Use SystemClock* instead of std::shared_ptr<SystemClock> in lower level routines (#8033 )	4 years ago
range_del_aggregator.cc	In ParseInternalKey(), include corrupt key info in Status (#7515 )	4 years ago
range_del_aggregator.h	Fix some typos in comments (#8066 )	4 years ago
range_del_aggregator_bench.cc	Cleanup multiple implementations of VectorIterator (#8901 )	3 years ago
range_del_aggregator_test.cc	Cleanup multiple implementations of VectorIterator (#8901 )	3 years ago
range_tombstone_fragmenter.cc	Added memtable garbage statistics (#8411 )	3 years ago
range_tombstone_fragmenter.h	Added memtable garbage statistics (#8411 )	3 years ago
range_tombstone_fragmenter_test.cc	Cleanup multiple implementations of VectorIterator (#8901 )	3 years ago
read_callback.h	Fix and detect headers with missing dependencies (#8893 )	3 years ago
repair.cc	Cleanup includes in dbformat.h (#8930 )	3 years ago
repair_test.cc	Some fixes and enhancements to `ldb repair` (#8544 )	3 years ago
snapshot_checker.h	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433 )	5 years ago
snapshot_impl.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433 )	5 years ago
snapshot_impl.h	Fix and detect headers with missing dependencies (#8893 )	3 years ago
table_cache.cc	Add file temperature related counter and bytes stats to and io_stats (#8710 )	3 years ago
table_cache.h	Add file temperature related counter and bytes stats to and io_stats (#8710 )	3 years ago
table_properties_collector.cc	Apply `sample_for_compression` to all block-based tables (#8105 )	4 years ago
table_properties_collector.h	Support "level_at_creation" in TablePropertiesCollectorFactory::Context (#8919 )	3 years ago
table_properties_collector_test.cc	Support "level_at_creation" in TablePropertiesCollectorFactory::Context (#8919 )	3 years ago
transaction_log_impl.cc	Add file operation callbacks to SequentialFileReader (#8982 )	3 years ago
transaction_log_impl.h	Cleanup includes in dbformat.h (#8930 )	3 years ago
trim_history_scheduler.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433 )	5 years ago
trim_history_scheduler.h	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433 )	5 years ago
version_builder.cc	Refactor the detailed consistency checks and the SST saving logic in VersionBuilder (#9099 )	3 years ago
version_builder.h	Refactor the detailed consistency checks and the SST saving logic in VersionBuilder (#9099 )	3 years ago
version_builder_test.cc	Try to start TTL earlier with kMinOverlappingRatio is used (#8749 )	3 years ago
version_edit.cc	Write file temperature information to manifest (#8284 )	4 years ago
version_edit.h	Write file temperature information to manifest (#8284 )	4 years ago
version_edit_handler.cc	Cleanup includes in dbformat.h (#8930 )	3 years ago
version_edit_handler.h	Fixed manifest_dump issues when printing keys and values containing null characters (#8378 )	3 years ago
version_edit_test.cc	Make it able to ignore WAL related VersionEdits in older versions (#7873 )	4 years ago
version_set.cc	Refactor the detailed consistency checks and the SST saving logic in VersionBuilder (#9099 )	3 years ago
version_set.h	Refactor the detailed consistency checks and the SST saving logic in VersionBuilder (#9099 )	3 years ago
version_set_test.cc	Try to start TTL earlier with kMinOverlappingRatio is used (#8749 )	3 years ago
wal_edit.cc	Always track WAL obsoletion (#7759 )	4 years ago
wal_edit.h	Always track WAL obsoletion (#7759 )	4 years ago
wal_edit_test.cc	Always track WAL obsoletion (#7759 )	4 years ago
wal_manager.cc	Allow WAL dir to change with db dir (#8582 )	3 years ago
wal_manager.h	Allow WAL dir to change with db dir (#8582 )	3 years ago
wal_manager_test.cc	Make SystemClock into a Customizable Class (#8636 )	3 years ago
write_batch.cc	Adapt key-value checksum for timestamp-suffixed keys (#8914 )	3 years ago
write_batch_base.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433 )	5 years ago
write_batch_internal.h	Adapt key-value checksum for timestamp-suffixed keys (#8914 )	3 years ago
write_batch_test.cc	Remove some unneeded code (#8736 )	3 years ago
write_callback.h	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433 )	5 years ago
write_callback_test.cc	Move slow valgrind tests behind -DROCKSDB_FULL_VALGRIND_RUN (#8475 )	3 years ago
write_controller.cc	Revamp WriteController (#8064 )	4 years ago
write_controller.h	Revamp WriteController (#8064 )	4 years ago
write_controller_test.cc	Revamp WriteController (#8064 )	4 years ago
write_thread.cc	Stall writes in WriteBufferManager when memory_usage exceeds buffer_size (#7898 )	4 years ago
write_thread.h	typo: fix typo in db/write_thread's state (#8423 )	3 years ago