fork of https://github.com/oxigraph/rocksdb and https://github.com/facebook/rocksdb for nextgraph and oxigraph

You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

sdong 246d469750 Reduce overhead of SortFileByOverlappingRatio() (#10161) ... Summary: Currently SortFileByOverlappingRatio() is O(nlogn). It is usually OK but When there are a lot of files in an LSM-tree, SortFileByOverlappingRatio() can take non-trivial amount of time. The problem is severe when the user is loading keys in sorted order, where compaction is only trivial move and this operation becomes the bottleneck and limit the total throughput. This commit makes SortFileByOverlappingRatio() only find the top 50 files based on score. 50 files are usually enough for the parallel compactions needed for the level, and in case it is not enough, we would fall back to random, which should be acceptable. Pull Request resolved: https://github.com/facebook/rocksdb/pull/10161 Test Plan: Run a fillseq that generates a lot of files, and observe throughput improved (although stall is not yet eliminated). The command ran: TEST_TMPDIR=/dev/shm/ ./db_bench_sort --benchmarks=fillseq --compression_type=lz4 --write_buffer_size=5000000 --num=100000000 --value_size=1000 The throughput improved by 11%. Reviewed By: jay-zhuang Differential Revision: D37129469 fbshipit-source-id: 492da2ef5bfc7cdd6daa3986b50d2ff91f88542d		2 years ago
..
clipping_iterator.h	Add a clipping internal iterator (#8327)	3 years ago
clipping_iterator_test.cc	Cleanup multiple implementations of VectorIterator (#8901)	3 years ago
compaction.cc	Cut output files at compaction cursors (#10227)	2 years ago
compaction.h	Cut output files at compaction cursors (#10227)	2 years ago
compaction_iteration_stats.h	Support readahead during compaction for blob files (#9187)	3 years ago
compaction_iterator.cc	Abort in dbg mode after logging (#10183)	2 years ago
compaction_iterator.h	Consolidate manual_compaction_paused_ check (#10070)	2 years ago
compaction_iterator_test.cc	Consolidate manual_compaction_paused_ check (#10070)	2 years ago
compaction_job.cc	Cut output files at compaction cursors (#10227)	2 years ago
compaction_job.h	Consolidate manual_compaction_paused_ check (#10070)	2 years ago
compaction_job_stats_test.cc	Remove own ToString() (#9955)	3 years ago
compaction_job_test.cc	Add blob source to retrieve blobs in RocksDB (#10198)	2 years ago
compaction_picker.cc	Support specifying blob garbage collection parameters when CompactRange() (#10073)	2 years ago
compaction_picker.h	Add OpenAndTrimHistory API to support trimming data with specified timestamp (#9410)	3 years ago
compaction_picker_fifo.cc	Add OpenAndTrimHistory API to support trimming data with specified timestamp (#9410)	3 years ago
compaction_picker_fifo.h	Add OpenAndTrimHistory API to support trimming data with specified timestamp (#9410)	3 years ago
compaction_picker_level.cc	Add basic kRoundRobin compaction policy (#10107)	2 years ago
compaction_picker_level.h	Make ImmutableOptions struct that inherits from ImmutableCFOptions and ImmutableDBOptions (#8262)	4 years ago
compaction_picker_test.cc	Reduce overhead of SortFileByOverlappingRatio() (#10161)	2 years ago
compaction_picker_universal.cc	Use std::numeric_limits<> (#9954)	3 years ago
compaction_picker_universal.h	Make ImmutableOptions struct that inherits from ImmutableCFOptions and ImmutableDBOptions (#8262)	4 years ago
compaction_service_test.cc	Fix failed VerifySstUniqueIds unittests (#10043)	2 years ago
file_pri.h	Try to start TTL earlier with kMinOverlappingRatio is used (#8749)	3 years ago
sst_partitioner.cc	Restore Regex support for ObjectLibrary::Register, rename new APIs to allow old one to be deprecated in the future (#9362)	3 years ago