fork of https://github.com/oxigraph/rocksdb and https://github.com/facebook/rocksdb for nextgraph and oxigraph

You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

Peter Dillinger 3182beeffc Observe and warn about misconfigured HyperClockCache (#10965) ... Summary: Background. One of the core risks of chosing HyperClockCache is ending up with degraded performance if estimated_entry_charge is very significantly wrong. Too low leads to under-utilized hash table, which wastes a bit of (tracked) memory and likely increases access times due to larger working set size (more TLB misses). Too high leads to fully populated hash table (at some limit with reasonable lookup performance) and not being able to cache as many objects as the memory limit would allow. In either case, performance degradation is graceful/continuous but can be quite significant. For example, cutting block size in half without updating estimated_entry_charge could lead to a large portion of configured block cache memory (up to roughly 1/3) going unused. Fix. This change adds a mechanism through which the DB periodically probes the block cache(s) for "problems" to report, and adds diagnostics to the HyperClockCache for bad estimated_entry_charge. The periodic probing is currently done with DumpStats / stats_dump_period_sec, and diagnostics reported to info_log (normally LOG file). Pull Request resolved: https://github.com/facebook/rocksdb/pull/10965 Test Plan: unit test included. Doesn't cover all the implemented subtleties of reporting, but ensures basics of when to report or not. Also manual testing with db_bench. Create db with ``` ./db_bench --benchmarks=fillrandom,flush --num=3000000 --disable_wal=1 ``` Use and check LOG file for HyperClockCache for various block sizes (used as estimated_entry_charge) ``` ./db_bench --use_existing_db --benchmarks=readrandom --num=3000000 --duration=20 --stats_dump_period_sec=8 --cache_type=hyper_clock_cache -block_size=XXXX ``` Seeing warnings / errors or not as expected. Reviewed By: anand1976 Differential Revision: D41406932 Pulled By: pdillinger fbshipit-source-id: 4ca56162b73017e4b9cec2cad74466f49c27a0a7		1 year ago
..
auto_roll_logger.cc	Run clang format on logging folder (#10861)	2 years ago
auto_roll_logger.h	Run clang format on logging folder (#10861)	2 years ago
auto_roll_logger_test.cc	Run clang format on logging folder (#10861)	2 years ago
env_logger.h	Fix build error with NIOSTATS_CONTEXT (#10506)	2 years ago
env_logger_test.cc	Run clang format on logging folder (#10861)	2 years ago
event_logger.cc	Run clang format on logging folder (#10861)	2 years ago
event_logger.h	Run clang format on logging folder (#10861)	2 years ago
event_logger_test.cc	Run clang format on logging folder (#10861)	2 years ago
log_buffer.cc	Run clang format on logging folder (#10861)	2 years ago
log_buffer.h	Run clang format on logging folder (#10861)	2 years ago
logging.h	Observe and warn about misconfigured HyperClockCache (#10965)	1 year ago