fork of https://github.com/oxigraph/rocksdb and https://github.com/facebook/rocksdb for nextgraph and oxigraph

You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

Peter Dillinger 17bc27741f Improve memory efficiency of many OptimisticTransactionDBs (#11439) ... Summary: Currently it's easy to use a ton of memory with many small OptimisticTransactionDB instances, because each one by default allocates a million mutexes (40 bytes each on my compiler) for validating transactions. It even puts a lot of pressure on the allocator by allocating each one individually! In this change: * Create a new object and option that enables sharing these buckets of mutexes between instances. This is generally good for load balancing potential contention as various DBs become hotter or colder with txn writes. About the only cases where this sharing wouldn't make sense (e.g. each DB usually written by one thread) are cases that would be better off with OccValidationPolicy::kValidateSerial which doesn't use the buckets anyway. * Allocate the mutexes in a contiguous array, for efficiency * Add an option to ensure the mutexes are cache-aligned. In several other places we use cache-aligned mutexes but OptimisticTransactionDB historically does not. It should be a space-time trade-off the user can choose. * Provide some visibility into the memory used by the mutex buckets with an ApproximateMemoryUsage() function (also used in unit testing) * Share code with other users of "striped" mutexes, appropriate refactoring for customization & efficiency (e.g. using FastRange instead of modulus) Pull Request resolved: https://github.com/facebook/rocksdb/pull/11439 Test Plan: unit tests added. Ran sized-up versions of stress test in unit test, including a before-and-after performance test showing no consistent difference. (NOTE: OptimisticTransactionDB not currently covered by db_stress!) Reviewed By: ltamasi Differential Revision: D45796393 Pulled By: pdillinger fbshipit-source-id: ae2b3a26ad91ceeec15debcdc63ff48df6736a54		2 years ago
..
blob_constants.h	Move BlobDB related files under db/ to db/blob/ (#6519)	5 years ago
blob_contents.cc	Major Cache refactoring, CPU efficiency improvement (#10975)	2 years ago
blob_contents.h	Change internal headers with duplicate names (#11408)	2 years ago
blob_counting_iterator.h	Refactor AddRangeDels() + consider range tombstone during compaction file cutting (#11113)	2 years ago
blob_counting_iterator_test.cc	Print stack traces on frozen tests in CI (#10828)	2 years ago
blob_fetcher.cc	Support readahead during compaction for blob files (#9187)	3 years ago
blob_fetcher.h	Support readahead during compaction for blob files (#9187)	3 years ago
blob_file_addition.cc	Print blob file checksums as hex (#8437)	4 years ago
blob_file_addition.h	Move BlobDB related files under db/ to db/blob/ (#6519)	5 years ago
blob_file_addition_test.cc	Print stack traces on frozen tests in CI (#10828)	2 years ago
blob_file_builder.cc	Changes and enhancements to compression stats, thresholds (#11388)	2 years ago
blob_file_builder.h	Support prepopulating/warming the blob cache (#10298)	3 years ago
blob_file_builder_test.cc	Print stack traces on frozen tests in CI (#10828)	2 years ago
blob_file_cache.cc	Improve memory efficiency of many OptimisticTransactionDBs (#11439)	2 years ago
blob_file_cache.h	Improve memory efficiency of many OptimisticTransactionDBs (#11439)	2 years ago
blob_file_cache_test.cc	Group rocksdb.sst.read.micros stat by IOActivity flush and compaction (#11288)	2 years ago
blob_file_completion_callback.h	Remove RocksDB LITE (#11147)	2 years ago
blob_file_garbage.cc	Move BlobDB related files under db/ to db/blob/ (#6519)	5 years ago
blob_file_garbage.h	Move BlobDB related files under db/ to db/blob/ (#6519)	5 years ago
blob_file_garbage_test.cc	Print stack traces on frozen tests in CI (#10828)	2 years ago
blob_file_meta.cc	Add BlobMetaData retrieval methods (#8273)	4 years ago
blob_file_meta.h	Add BlobMetaData retrieval methods (#8273)	4 years ago
blob_file_reader.cc	Change internal headers with duplicate names (#11408)	2 years ago
blob_file_reader.h	Group rocksdb.sst.read.micros stat by IOActivity flush and compaction (#11288)	2 years ago
blob_file_reader_test.cc	Group rocksdb.sst.read.micros stat by IOActivity flush and compaction (#11288)	2 years ago
blob_garbage_meter.cc	Add a class for measuring the amount of garbage generated during compaction (#8426)	4 years ago
blob_garbage_meter.h	Add a class for measuring the amount of garbage generated during compaction (#8426)	4 years ago
blob_garbage_meter_test.cc	Print stack traces on frozen tests in CI (#10828)	2 years ago
blob_index.h	Remove local static string (#8103)	3 years ago
blob_log_format.cc	Remove local static string (#8103)	3 years ago
blob_log_format.h	Run clang-format on db/blob/ (#10856)	2 years ago
blob_log_sequential_reader.cc	Change internal headers with duplicate names (#11408)	2 years ago
blob_log_sequential_reader.h	Fix a issue with initializing blob header buffer (#8537)	4 years ago
blob_log_writer.cc	Change internal headers with duplicate names (#11408)	2 years ago
blob_log_writer.h	Use SystemClock* instead of std::shared_ptr<SystemClock> in lower level routines (#8033)	4 years ago
blob_read_request.h	Enable blob caching for MultiGetBlob in RocksDB (#10272)	3 years ago
blob_source.cc	Change internal headers with duplicate names (#11408)	2 years ago
blob_source.h	Group rocksdb.sst.read.micros stat by IOActivity flush and compaction (#11288)	2 years ago
blob_source_test.cc	Group rocksdb.sst.read.micros stat by IOActivity flush and compaction (#11288)	2 years ago
db_blob_basic_test.cc	Fix backward iteration issue when user defined timestamp is enabled in BlobDB (#11258)	2 years ago
db_blob_compaction_test.cc	Remove RocksDB LITE (#11147)	2 years ago
db_blob_corruption_test.cc	Remove RocksDB LITE (#11147)	2 years ago
db_blob_index_test.cc	Remove RocksDB LITE (#11147)	2 years ago
prefetch_buffer_collection.cc	Support readahead during compaction for blob files (#9187)	3 years ago
prefetch_buffer_collection.h	Support readahead during compaction for blob files (#9187)	3 years ago