fork of https://github.com/oxigraph/rocksdb and https://github.com/facebook/rocksdb for nextgraph and oxigraph

You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

Andrew Kryczka 91166012c8 Prevent a case of WriteBufferManager flush thrashing (#6364) ... Summary: Previously, the flushes triggered by `WriteBufferManager` could affect the same CF repeatedly if it happens to get consecutive writes. Such flushes are not particularly useful for reducing memory usage since they switch nearly-empty memtables to immutable while they've just begun filling their first arena block. In fact they may not even reduce the mutable memory count if they involve replacing one mutable memtable containing one arena block with a new mutable memtable containing one arena block. Further, if such switches happen even a few times before a flush finishes, the immutable memtable limit will be reached and writes will stall. This PR adds a heuristic to not switch memtables to immutable for CFs that already have one or more immutable memtables awaiting flush. There is a memory usage regression if the user continues writing to the same CF, that DB does not have any CFs eligible for switching, flushes are not finishing, and the `WriteBufferManager` was constructed with `allow_stall=false`. Before, it would grow by switching nearly empty memtables until writes stall. Now, it would grow by filling memtables until writes stall. This feels like an acceptable behavior change because users who prefer to stall over violate the memory limit should be using `allow_stall=true`, which is unaffected by this PR. Pull Request resolved: https://github.com/facebook/rocksdb/pull/6364 Test Plan: - Command: `rm -rf /dev/shm/dbbench/ && TEST_TMPDIR=/dev/shm ./db_bench -benchmarks=fillrandom -num_multi_db=8 -num_column_families=2 -write_buffer_size=4194304 -db_write_buffer_size=16777216 -compression_type=none -statistics=true -target_file_size_base=4194304 -max_bytes_for_level_base=16777216` - `rocksdb.db.write.stall` count before this PR: 175 - `rocksdb.db.write.stall` count after this PR: 0 Reviewed By: jay-zhuang Differential Revision: D20167197 Pulled By: ajkr fbshipit-source-id: 4a64064e9bc33d57c0a35f15547542d0191d0cb7		2 years ago
..
compacted_db_impl.cc	Return "invalid argument" when read timestamp is too old (#10109)	2 years ago
compacted_db_impl.h	Add API for writing wide-column entities (#10242)	2 years ago
db_impl.cc	Fragment memtable range tombstone in the write path (#10380)	2 years ago
db_impl.h	Avoid allocations/copies for large `GetMergeOperands()` results (#10458)	2 years ago
db_impl_compaction_flush.cc	WritableFileWriter tries to skip operations after failure (#10489)	2 years ago
db_impl_debug.cc	Do not hold mutex when write keys if not necessary (#7516)	2 years ago
db_impl_experimental.cc	Remove unused fields from FileMetaData (temporarily) (#10443)	2 years ago
db_impl_files.cc	Do not hold mutex when write keys if not necessary (#7516)	2 years ago
db_impl_open.cc	Fragment memtable range tombstone in the write path (#10380)	2 years ago
db_impl_readonly.cc	Fragment memtable range tombstone in the write path (#10380)	2 years ago
db_impl_readonly.h	Add API for writing wide-column entities (#10242)	2 years ago
db_impl_secondary.cc	Fragment memtable range tombstone in the write path (#10380)	2 years ago
db_impl_secondary.h	Update code comment and logging for secondary instance (#10260)	2 years ago
db_impl_write.cc	Prevent a case of WriteBufferManager flush thrashing (#6364)	2 years ago