fork of https://github.com/oxigraph/rocksdb and https://github.com/facebook/rocksdb for nextgraph and oxigraph

You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

Akanksha Mahajan 56463d443d Provide support for subcompactions with user-defined timestamps (#10344) ... Summary: The subcompaction logic currently picks file boundaries as subcompaction boundaries. This is not compatible with user-defined timestamps because of two issues. Issue1: ReadOptions.iterate_lower_bound and ReadOptions.iterate_upper_bound contains timestamps which results in assertion failure as BlockBasedTableIterator expects bounds to be without timestamps. As result, because of wrong comparison end key is returned as user_key resulting in assertion failure. Issue2: Since it might result in two keys that only differ by user timestamp getting processed by two different subcompactions (and thus two different CompactionIterator state machines), which in turn can cause data correction issues. This PR provide support to reenable subcompactions with user-defined timestamps. Pull Request resolved: https://github.com/facebook/rocksdb/pull/10344 Test Plan: Added new unit test - Without fix for Issue1 unit test MultipleSubCompactions fails with error: ``` db_with_timestamp_compaction_test: ./db/compaction/clipping_iterator.h:247: void rocksdb::ClippingIterat│ or::AssertBounds(): Assertion `!valid_ || !end_ || cmp_->Compare(key(), *end_) < 0' failed. Received signal 6 (Aborted) │ #0 /usr/local/fbcode/platform009/lib/libc.so.6(gsignal+0x100) [0x7f8fbbbfe530] db_with_timestamp_compaction_test: ./db/compaction/clipping_iterator.h:247: void rocksdb::ClippingIterator::AssertBounds(): Assertion `!valid_ || !end_ || cmp_->Compare(key(), *end_) < 0' failed. Aborted (core dumped) ``` Ran stress test `make crash_test_with_ts -j32` Reviewed By: riversand963 Differential Revision: D38220841 Pulled By: akankshamahajan15 fbshipit-source-id: 5d5cae2bd37fcaeba1e77fce0a69070ad4158ccb		2 years ago
..
clipping_iterator.h	Make InternalKeyComparator not configurable (#10342)	2 years ago
clipping_iterator_test.cc	Cleanup multiple implementations of VectorIterator (#8901)	3 years ago
compaction.cc	Provide support for subcompactions with user-defined timestamps (#10344)	2 years ago
compaction.h	Tiered Compaction: per key placement support (#9964)	2 years ago
compaction_iteration_stats.h	Support readahead during compaction for blob files (#9187)	3 years ago
compaction_iterator.cc	Tiered compaction: integrate Seqno time mapping with per key placement (#10370)	2 years ago
compaction_iterator.h	Tiered compaction: integrate Seqno time mapping with per key placement (#10370)	2 years ago
compaction_iterator_test.cc	Tiered Compaction: per key placement support (#9964)	2 years ago
compaction_job.cc	Provide support for subcompactions with user-defined timestamps (#10344)	2 years ago
compaction_job.h	Support subcmpct using reserved resources for round-robin priority (#10341)	2 years ago
compaction_job_stats_test.cc	Remove own ToString() (#9955)	3 years ago
compaction_job_test.cc	Tiered Compaction: per key placement support (#9964)	2 years ago
compaction_outputs.cc	Add seqno to time mapping (#10338)	2 years ago
compaction_outputs.h	Add seqno to time mapping (#10338)	2 years ago
compaction_picker.cc	Support subcmpct using reserved resources for round-robin priority (#10341)	2 years ago
compaction_picker.h	Support subcmpct using reserved resources for round-robin priority (#10341)	2 years ago
compaction_picker_fifo.cc	Fix underflow in FIFOCompactionPicker (#10386)	2 years ago
compaction_picker_fifo.h	Add OpenAndTrimHistory API to support trimming data with specified timestamp (#9410)	3 years ago
compaction_picker_level.cc	Support subcmpct using reserved resources for round-robin priority (#10341)	2 years ago
compaction_picker_level.h	Make ImmutableOptions struct that inherits from ImmutableCFOptions and ImmutableDBOptions (#8262)	4 years ago
compaction_picker_test.cc	Support subcmpct using reserved resources for round-robin priority (#10341)	2 years ago
compaction_picker_universal.cc	Multi-File Trivial Move in L0->L1 (#10188)	3 years ago
compaction_picker_universal.h	Make ImmutableOptions struct that inherits from ImmutableCFOptions and ImmutableDBOptions (#8262)	4 years ago
compaction_service_job.cc	Improve SubCompaction Partitioning (#10393)	2 years ago
compaction_service_test.cc	Fix failed VerifySstUniqueIds unittests (#10043)	3 years ago
compaction_state.cc	Tiered Compaction: per key placement support (#9964)	2 years ago
compaction_state.h	Tiered Compaction: per key placement support (#9964)	2 years ago
file_pri.h	Try to start TTL earlier with kMinOverlappingRatio is used (#8749)	3 years ago
sst_partitioner.cc	Restore Regex support for ObjectLibrary::Register, rename new APIs to allow old one to be deprecated in the future (#9362)	3 years ago
subcompaction_state.cc	Tiered Compaction: per key placement support (#9964)	2 years ago
subcompaction_state.h	Support subcmpct using reserved resources for round-robin priority (#10341)	2 years ago
tiered_compaction_test.cc	Tiered compaction: integrate Seqno time mapping with per key placement (#10370)	2 years ago