fork of https://github.com/oxigraph/rocksdb and https://github.com/facebook/rocksdb for nextgraph and oxigraph

You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

Hui Xiao 3573558ec5 Rewrite memory-charging feature's option API (#9926) ... Summary: **Context:** Previous PR https://github.com/facebook/rocksdb/pull/9748, https://github.com/facebook/rocksdb/pull/9073, https://github.com/facebook/rocksdb/pull/8428 added separate flag for each charged memory area. Such API design is not scalable as we charge more and more memory areas. Also, we foresee an opportunity to consolidate this feature with other cache usage related features such as `cache_index_and_filter_blocks` using `CacheEntryRole`. Therefore we decided to consolidate all these flags with `CacheUsageOptions cache_usage_options` and this PR serves as the first step by consolidating memory-charging related flags. **Summary:** - Replaced old API reference with new ones, including making `kCompressionDictionaryBuildingBuffer` opt-out and added a unit test for that - Added missing db bench/stress test for some memory charging features - Renamed related test suite to indicate they are under the same theme of memory charging - Refactored a commonly used mocked cache component in memory charging related tests to reduce code duplication - Replaced the phrases "memory tracking" / "cache reservation" (other than CacheReservationManager-related ones) with "memory charging" for standard description of this feature. Pull Request resolved: https://github.com/facebook/rocksdb/pull/9926 Test Plan: - New unit test for opt-out `kCompressionDictionaryBuildingBuffer` `TEST_F(ChargeCompressionDictionaryBuildingBufferTest, Basic)` - New unit test for option validation/sanitization `TEST_F(CacheUsageOptionsOverridesTest, SanitizeAndValidateOptions)` - CI - db bench (in case querying new options introduces regression) **+0.5% micros/op**: `TEST_TMPDIR=/dev/shm/testdb ./db_bench -benchmarks=fillseq -db=$TEST_TMPDIR -charge_compression_dictionary_building_buffer=1(remove this for comparison) -compression_max_dict_bytes=10000 -disable_auto_compactions=1 -write_buffer_size=100000 -num=4000000 | egrep 'fillseq'` #-run | (pre-PR) avg micros/op | std micros/op | (post-PR) micros/op | std micros/op | change (%) -- | -- | -- | -- | -- | -- 10 | 3.9711 | 0.264408 | 3.9914 | 0.254563 | 0.5111933721 20 | 3.83905 | 0.0664488 | 3.8251 | 0.0695456 | **-0.3633711465** 40 | 3.86625 | 0.136669 | 3.8867 | 0.143765 | **0.5289363078** - db_stress: `python3 tools/db_crashtest.py blackbox -charge_compression_dictionary_building_buffer=1 -charge_filter_construction=1 -charge_table_reader=1 -cache_size=1` killed as normal Reviewed By: ajkr Differential Revision: D36054712 Pulled By: hx235 fbshipit-source-id: d406e90f5e0c5ea4dbcb585a484ad9302d4302af		3 years ago
..
advisor	Update branch as "main" in tools/advisor/README.md (#8744)	3 years ago
block_cache_analyzer	Use std::numeric_limits<> (#9954)	3 years ago
dump	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	5 years ago
CMakeLists.txt	Mark dependencies as PRIVATE and fix missing dependencies in tools. (#6790)	5 years ago
Dockerfile	adding docker build script and dockerfile	10 years ago
analyze_txn_stress_test.sh	Add copyright headers per FB open-source checkup tool. (#5199)	6 years ago
auto_sanity_test.sh	Add copyright headers per FB open-source checkup tool. (#5199)	6 years ago
backup_db.sh	Revamp check_format_compatible.sh (#8012)	4 years ago
benchmark.sh	Remove deprecated API AdvancedColumnFamilyOptions::rate_limit_delay_max_milliseconds (#9455)	3 years ago
benchmark_leveldb.sh	Add copyright headers per FB open-source checkup tool. (#5199)	6 years ago
blob_dump.cc	Remove using namespace (#9369)	3 years ago
check_all_python.py	Allow missing "unversioned" python, as in CentOS 8 (#6883)	5 years ago
check_format_compatible.sh	Fix format_compatible blowing away its TEST_TMPDIR (#9970)	3 years ago
db_bench.cc	Add (& fix) some simple source code checks (#8821)	3 years ago
db_bench_tool.cc	Rewrite memory-charging feature's option API (#9926)	3 years ago
db_bench_tool_test.cc	Remove BlockBasedTableOptions.hash_index_allow_collision (#9454)	3 years ago
db_crashtest.py	Rewrite memory-charging feature's option API (#9926)	3 years ago
db_repl_stress.cc	Remove using namespace (#9369)	3 years ago
db_sanity_test.cc	Remove own ToString() (#9955)	3 years ago
dbench_monitor	Fix /bin/bash shebangs	7 years ago
generate_random_db.sh	Add copyright headers per FB open-source checkup tool. (#5199)	6 years ago
ingest_external_sst.sh	Add copyright headers per FB open-source checkup tool. (#5199)	6 years ago
io_tracer_parser.cc	Add IO Tracer Parser (#7333)	4 years ago
io_tracer_parser_test.cc	Cleanup includes in dbformat.h (#8930)	3 years ago
io_tracer_parser_tool.cc	Add request_id in IODebugContext. (#8045)	4 years ago
io_tracer_parser_tool.h	Add IO Tracer Parser (#7333)	4 years ago
ldb.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	5 years ago
ldb_cmd.cc	Support single delete in ldb (#9469)	3 years ago
ldb_cmd_impl.h	Support single delete in ldb (#9469)	3 years ago
ldb_cmd_test.cc	Remove own ToString() (#9955)	3 years ago
ldb_test.py	Add blob dump support to the dump_live_files command (#9896)	3 years ago
ldb_tool.cc	Default `try_load_options` to true when DB is specified (#9937)	3 years ago
pflag	Fix /bin/bash shebangs	7 years ago
reduce_levels_test.cc	Remove own ToString() (#9955)	3 years ago
regression_test.sh	Fix COMMIT_ID in regression_test.sh (#9047)	3 years ago
restore_db.sh	Revamp check_format_compatible.sh (#8012)	4 years ago
rocksdb_dump_test.sh	Add copyright headers per FB open-source checkup tool. (#5199)	6 years ago
run_blob_bench.sh	Add blob compaction readahead size to the BlobDB benchmark script (#9566)	3 years ago
run_flash_bench.sh	Add copyright headers per FB open-source checkup tool. (#5199)	6 years ago
run_leveldb.sh	Add copyright headers per FB open-source checkup tool. (#5199)	6 years ago
sample-dump.dmp	First version of rocksdb_dump and rocksdb_undump.	10 years ago
simulated_hybrid_file_system.cc	Improve SimulatedHybridFileSystem (#9301)	3 years ago
simulated_hybrid_file_system.h	Improve SimulatedHybridFileSystem (#9301)	3 years ago
sst_dump.cc	Implement a new subcommand "identify" for sst_dump (#6943)	4 years ago
sst_dump_test.cc	Use the comparator from the sst file table properties in sst_dump_tool (#9491)	3 years ago
sst_dump_tool.cc	Use std::numeric_limits<> (#9954)	3 years ago
trace_analyzer.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433)	5 years ago
trace_analyzer_test.cc	Disable the QPS verification in test temporally (#9190)	3 years ago
trace_analyzer_tool.cc	Use std::numeric_limits<> (#9954)	3 years ago
trace_analyzer_tool.h	Add commit marker with timestamp (#9266)	3 years ago
verify_random_db.sh	Add copyright headers per FB open-source checkup tool. (#5199)	6 years ago
write_external_sst.sh	Revamp check_format_compatible.sh (#8012)	4 years ago
write_stress.cc	Add a SystemClock class to capture the time functions of an Env (#7858)	4 years ago
write_stress_runner.py	Allow missing "unversioned" python, as in CentOS 8 (#6883)	5 years ago