rocksdb

fork of https://github.com/oxigraph/rocksdb and https://github.com/facebook/rocksdb for nextgraph and oxigraph

History

Bo Wang f706a9c199 Add a secondary cache implementation based on LRUCache 1 (#9518 ) Summary: Summary: RocksDB uses a block cache to reduce IO and make queries more efficient. The block cache is based on the LRU algorithm (LRUCache) and keeps objects containing uncompressed data, such as Block, ParsedFullFilterBlock etc. It allows the user to configure a second level cache (rocksdb::SecondaryCache) to extend the primary block cache by holding items evicted from it. Some of the major RocksDB users, like MyRocks, use direct IO and would like to use a primary block cache for uncompressed data and a secondary cache for compressed data. The latter allows us to mitigate the loss of the Linux page cache due to direct IO. This PR includes a concrete implementation of rocksdb::SecondaryCache that integrates with compression libraries such as LZ4 and implements an LRU cache to hold compressed blocks. Pull Request resolved: https://github.com/facebook/rocksdb/pull/9518 Test Plan: In this PR, the lru_secondary_cache_test.cc includes the following tests: 1. The unit tests for the secondary cache with either compression or no compression, such as basic tests, fails tests. 2. The integration tests with both primary cache and this secondary cache . Follow Up: 1. Statistics (e.g. compression ratio) will be added in another PR. 2. Once this implementation is ready, I will do some shadow testing and benchmarking with UDB to measure the impact. Reviewed By: anand1976 Differential Revision: D34430930 Pulled By: gitbw95 fbshipit-source-id: 218d78b672a2f914856d8a90ff32f2f5b5043ded		3 years ago
..
advisor	Update branch as "main" in tools/advisor/README.md (#8744 )	3 years ago
block_cache_analyzer	Cleanup includes in dbformat.h (#8930 )	3 years ago
dump	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433 )	5 years ago
CMakeLists.txt	Mark dependencies as PRIVATE and fix missing dependencies in tools. (#6790 )	5 years ago
Dockerfile	adding docker build script and dockerfile	10 years ago
analyze_txn_stress_test.sh	Add copyright headers per FB open-source checkup tool. (#5199 )	6 years ago
auto_sanity_test.sh	Add copyright headers per FB open-source checkup tool. (#5199 )	6 years ago
backup_db.sh	Revamp check_format_compatible.sh (#8012 )	4 years ago
benchmark.sh	Remove deprecated API AdvancedColumnFamilyOptions::rate_limit_delay_max_milliseconds (#9455 )	3 years ago
benchmark_leveldb.sh	Add copyright headers per FB open-source checkup tool. (#5199 )	6 years ago
blob_dump.cc	Remove using namespace (#9369 )	3 years ago
check_all_python.py	Allow missing "unversioned" python, as in CentOS 8 (#6883 )	5 years ago
check_format_compatible.sh	Add releases till 6.29.fb to compatibility check (#9529 )	3 years ago
db_bench.cc	Add (& fix) some simple source code checks (#8821 )	3 years ago
db_bench_tool.cc	Add a secondary cache implementation based on LRUCache 1 (#9518 )	3 years ago
db_bench_tool_test.cc	Remove deprecated option new_table_reader_for_compaction_inputs (#9443 )	3 years ago
db_crashtest.py	minor tweaks to db_crashtest.py settings (#9483 )	3 years ago
db_repl_stress.cc	Remove using namespace (#9369 )	3 years ago
db_sanity_test.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433 )	5 years ago
dbench_monitor	Fix /bin/bash shebangs	7 years ago
generate_random_db.sh	Add copyright headers per FB open-source checkup tool. (#5199 )	6 years ago
ingest_external_sst.sh	Add copyright headers per FB open-source checkup tool. (#5199 )	6 years ago
io_tracer_parser.cc	Add IO Tracer Parser (#7333 )	4 years ago
io_tracer_parser_test.cc	Cleanup includes in dbformat.h (#8930 )	3 years ago
io_tracer_parser_tool.cc	Add request_id in IODebugContext. (#8045 )	4 years ago
io_tracer_parser_tool.h	Add IO Tracer Parser (#7333 )	4 years ago
ldb.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433 )	5 years ago
ldb_cmd.cc	Remove obsolete backupable_db.h, utility_db.h (#9438 )	3 years ago
ldb_cmd_impl.h	Some fixes and enhancements to `ldb repair` (#8544 )	3 years ago
ldb_cmd_test.cc	Clean up VersionStorageInfo a bit (#9494 )	3 years ago
ldb_test.py	Add list live files metadata (#8446 )	3 years ago
ldb_tool.cc	LDB to add --secondary_path to help (#9582 )	3 years ago
pflag	Fix /bin/bash shebangs	7 years ago
reduce_levels_test.cc	Remove some unneeded code (#8736 )	3 years ago
regression_test.sh	Fix COMMIT_ID in regression_test.sh (#9047 )	3 years ago
restore_db.sh	Revamp check_format_compatible.sh (#8012 )	4 years ago
rocksdb_dump_test.sh	Add copyright headers per FB open-source checkup tool. (#5199 )	6 years ago
run_blob_bench.sh	Add blob compaction readahead size to the BlobDB benchmark script (#9566 )	3 years ago
run_flash_bench.sh	Add copyright headers per FB open-source checkup tool. (#5199 )	6 years ago
run_leveldb.sh	Add copyright headers per FB open-source checkup tool. (#5199 )	6 years ago
sample-dump.dmp	First version of rocksdb_dump and rocksdb_undump.	10 years ago
simulated_hybrid_file_system.cc	Improve SimulatedHybridFileSystem (#9301 )	3 years ago
simulated_hybrid_file_system.h	Improve SimulatedHybridFileSystem (#9301 )	3 years ago
sst_dump.cc	Implement a new subcommand "identify" for sst_dump (#6943 )	5 years ago
sst_dump_test.cc	Use the comparator from the sst file table properties in sst_dump_tool (#9491 )	3 years ago
sst_dump_tool.cc	Add CreateFrom methods to Env/FileSystem (#8174 )	3 years ago
trace_analyzer.cc	Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433 )	5 years ago
trace_analyzer_test.cc	Disable the QPS verification in test temporally (#9190 )	3 years ago
trace_analyzer_tool.cc	Simplify `TraceAnalyzer` (#8697 )	3 years ago
trace_analyzer_tool.h	Add commit marker with timestamp (#9266 )	3 years ago
verify_random_db.sh	Add copyright headers per FB open-source checkup tool. (#5199 )	6 years ago
write_external_sst.sh	Revamp check_format_compatible.sh (#8012 )	4 years ago
write_stress.cc	Add a SystemClock class to capture the time functions of an Env (#7858 )	4 years ago
write_stress_runner.py	Allow missing "unversioned" python, as in CentOS 8 (#6883 )	5 years ago