fork of https://github.com/oxigraph/rocksdb and https://github.com/facebook/rocksdb for nextgraph and oxigraph

You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

Gang Liao e6432dfd4c Make it possible to enable blob files starting from a certain LSM tree level (#10077) ... Summary: Currently, if blob files are enabled (i.e. `enable_blob_files` is true), large values are extracted both during flush/recovery (when SST files are written into level 0 of the LSM tree) and during compaction into any LSM tree level. For certain use cases that have a mix of short-lived and long-lived values, it might make sense to support extracting large values only during compactions whose output level is greater than or equal to a specified LSM tree level (e.g. compactions into L1/L2/... or above). This could reduce the space amplification caused by large values that are turned into garbage shortly after being written at the price of some write amplification incurred by long-lived values whose extraction to blob files is delayed. In order to achieve this, we would like to do the following: - Add a new configuration option `blob_file_starting_level` (default: 0) to `AdvancedColumnFamilyOptions` (and `MutableCFOptions` and extend the related logic) - Instantiate `BlobFileBuilder` in `BuildTable` (used during flush and recovery, where the LSM tree level is L0) and `CompactionJob` iff `enable_blob_files` is set and the LSM tree level is `>= blob_file_starting_level` - Add unit tests for the new functionality, and add the new option to our stress tests (`db_stress` and `db_crashtest.py` ) - Add the new option to our benchmarking tool `db_bench` and the BlobDB benchmark script `run_blob_bench.sh` - Add the new option to the `ldb` tool (see https://github.com/facebook/rocksdb/wiki/Administration-and-Data-Access-Tool) - Ideally extend the C and Java bindings with the new option - Update the BlobDB wiki to document the new option. Pull Request resolved: https://github.com/facebook/rocksdb/pull/10077 Reviewed By: ltamasi Differential Revision: D36884156 Pulled By: gangliao fbshipit-source-id: 942bab025f04633edca8564ed64791cb5e31627d		2 years ago
..
cf_options.cc	Make it possible to enable blob files starting from a certain LSM tree level (#10077)	2 years ago
cf_options.h	Make it possible to enable blob files starting from a certain LSM tree level (#10077)	2 years ago
configurable.cc	Option type info functions (#9411)	2 years ago
configurable_helper.h	Fix some minor issues in the Customizable infrastructure (#8566)	3 years ago
configurable_test.cc	Remove own ToString() (#9955)	2 years ago
configurable_test.h	Move RegisterOptions into the Configurable API (#8223)	3 years ago
customizable.cc	Option type info functions (#9411)	2 years ago
customizable_test.cc	Added GetFactoryCount/Names/Types to ObjectRegistry (#9358)	2 years ago
db_options.cc	Track SST unique id in MANIFEST and verify (#9990)	2 years ago
db_options.h	Track SST unique id in MANIFEST and verify (#9990)	2 years ago
options.cc	Make it possible to enable blob files starting from a certain LSM tree level (#10077)	2 years ago
options_helper.cc	Make it possible to enable blob files starting from a certain LSM tree level (#10077)	2 years ago
options_helper.h	New backup meta schema, with file temperatures (#9660)	2 years ago
options_parser.cc	Support read rate-limiting in SequentialFileReader (#9973)	2 years ago
options_parser.h	Bring the Configurable options together (#5753)	4 years ago
options_settable_test.cc	Make it possible to enable blob files starting from a certain LSM tree level (#10077)	2 years ago
options_test.cc	Make it possible to enable blob files starting from a certain LSM tree level (#10077)	2 years ago