fork of https://github.com/oxigraph/rocksdb and https://github.com/facebook/rocksdb for nextgraph and oxigraph

You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

Baptiste Lemaire e3a96c4823 Memtable sampling for mempurge heuristic. (#8628) ... Summary: Changes the API of the MemPurge process: the `bool experimental_allow_mempurge` and `experimental_mempurge_policy` flags have been replaced by a `double experimental_mempurge_threshold` option. This change of API reflects another major change introduced in this PR: the MemPurgeDecider() function now works by sampling the memtables being flushed to estimate the overall amount of useful payload (payload minus the garbage), and then compare this useful payload estimate with the `double experimental_mempurge_threshold` value. Therefore, when the value of this flag is `0.0` (default value), mempurge is simply deactivated. On the other hand, a value of `DBL_MAX` would be equivalent to always going through a mempurge regardless of the garbage ratio estimate. At the moment, a `double experimental_mempurge_threshold` value else than 0.0 or `DBL_MAX` is opnly supported`with the `SkipList` memtable representation. Regarding the sampling, this PR includes the introduction of a `MemTable::UniqueRandomSample` function that collects (approximately) random entries from the memtable by using the new `SkipList::Iterator::RandomSeek()` under the hood, or by iterating through each memtable entry, depending on the target sample size and the total number of entries. The unit tests have been readapted to support this new API. Pull Request resolved: https://github.com/facebook/rocksdb/pull/8628 Reviewed By: pdillinger Differential Revision: D30149315 Pulled By: bjlemaire fbshipit-source-id: 1feef5390c95db6f4480ab4434716533d3947f27		3 years ago
..
alloc_tracker.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	5 years ago
hash_linklist_rep.cc	Genericize and clean up FastRange (#7436)	4 years ago
hash_linklist_rep.h	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	5 years ago
hash_skiplist_rep.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	5 years ago
hash_skiplist_rep.h	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	5 years ago
inlineskiplist.h	Memtable sampling for mempurge heuristic. (#8628)	3 years ago
inlineskiplist_test.cc	Move slow valgrind tests behind -DROCKSDB_FULL_VALGRIND_RUN (#8475)	3 years ago
memtablerep_bench.cc	Use SystemClock* instead of std::shared_ptr<SystemClock> in lower level routines (#8033)	4 years ago
skiplist.h	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	5 years ago
skiplist_test.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	5 years ago
skiplistrep.cc	Memtable sampling for mempurge heuristic. (#8628)	3 years ago
stl_wrappers.h	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	5 years ago
vectorrep.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	5 years ago
write_buffer_manager.cc	Use deleters to label cache entries and collect stats (#8297)	4 years ago
write_buffer_manager_test.cc	Feature: add SetBufferSize() so that managed size can be dynamic (#7961)	4 years ago