fork of https://github.com/oxigraph/rocksdb and https://github.com/facebook/rocksdb for nextgraph and oxigraph

You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

Andrew Kryczka cc01985db0 Introduce bottom-pri thread pool for large universal compactions ... Summary: When we had a single thread pool for compactions, a thread could be busy for a long time (minutes) executing a compaction involving the bottom level. In multi-instance setups, the entire thread pool could be consumed by such bottom-level compactions. Then, top-level compactions (e.g., a few L0 files) would be blocked for a long time ("head-of-line blocking"). Such top-level compactions are critical to prevent compaction stalls as they can quickly reduce number of L0 files / sorted runs. This diff introduces a bottom-priority queue for universal compactions including the bottom level. This alleviates the head-of-line blocking situation for fast, top-level compactions. - Added `Env::Priority::BOTTOM` thread pool. This feature is only enabled if user explicitly configures it to have a positive number of threads. - Changed `ThreadPoolImpl`'s default thread limit from one to zero. This change is invisible to users as we call `IncBackgroundThreadsIfNeeded` on the low-pri/high-pri pools during `DB::Open` with values of at least one. It is necessary, though, for bottom-pri to start with zero threads so the feature is disabled by default. - Separated `ManualCompaction` into two parts in `PrepickedCompaction`. `PrepickedCompaction` is used for any compaction that's picked outside of its execution thread, either manual or automatic. - Forward universal compactions involving last level to the bottom pool (worker thread's entry point is `BGWorkBottomCompaction`). - Track `bg_bottom_compaction_scheduled_` so we can wait for bottom-level compactions to finish. We don't count them against the background jobs limits. So users of this feature will get an extra compaction for free. Closes https://github.com/facebook/rocksdb/pull/2580 Differential Revision: D5422916 Pulled By: ajkr fbshipit-source-id: a74bd11f1ea4933df3739b16808bb21fcd512333		7 years ago
..
env.cc	Change RocksDB License	7 years ago
env_basic_test.cc	Move some files under util/ to separate dirs	7 years ago
env_chroot.cc	Change RocksDB License	7 years ago
env_chroot.h	Change RocksDB License	7 years ago
env_encryption.cc	Revert "comment out unused parameters"	7 years ago
env_hdfs.cc	Revert "comment out unused parameters"	7 years ago
env_posix.cc	Introduce bottom-pri thread pool for large universal compactions	7 years ago
env_test.cc	Introduce bottom-pri thread pool for large universal compactions	7 years ago
io_posix.cc	Revert "comment out unused parameters"	7 years ago
io_posix.h	Revert "comment out unused parameters"	7 years ago
mock_env.cc	Replace dynamic_cast<>	7 years ago
mock_env.h	Change RocksDB License	7 years ago
mock_env_test.cc	Move some files under util/ to separate dirs	7 years ago
posix_logger.h	Change RocksDB License	7 years ago