fork of https://github.com/oxigraph/rocksdb and https://github.com/facebook/rocksdb for nextgraph and oxigraph

You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

Andrew Kryczka 6c40806e51 Digest ZSTD compression dictionary once per SST file (#4251) ... Summary: In RocksDB, for a given SST file, all data blocks are compressed with the same dictionary. When we compress a block using the dictionary's raw bytes, the compression library first has to digest the dictionary to get it into a usable form. This digestion work is redundant and ideally should be done once per file. ZSTD offers APIs for the caller to create and reuse a digested dictionary object (`ZSTD_CDict`). In this PR, we call `ZSTD_createCDict` once per file to digest the raw bytes. Then we use `ZSTD_compress_usingCDict` to compress each data block using the pre-digested dictionary. Once the file's created `ZSTD_freeCDict` releases the resources held by the digested dictionary. There are a couple other changes included in this PR: - Changed the parameter object for (un)compression functions from `CompressionContext`/`UncompressionContext` to `CompressionInfo`/`UncompressionInfo`. This avoids the previous pattern, where `CompressionContext`/`UncompressionContext` had to be mutated before calling a (un)compression function depending on whether dictionary should be used. I felt that mutation was error-prone so eliminated it. - Added support for digested uncompression dictionaries (`ZSTD_DDict`) as well. However, this PR does not support reusing them across uncompression calls for the same file. That work is deferred to a later PR when we will store the `ZSTD_DDict` objects in block cache. Pull Request resolved: https://github.com/facebook/rocksdb/pull/4251 Differential Revision: D9257078 Pulled By: ajkr fbshipit-source-id: 21b8cb6bbdd48e459f1c62343780ab66c0a64438		6 years ago
..
backupable	Add path to WritableFileWriter. (#4039)	6 years ago
blob_db	Digest ZSTD compression dictionary once per SST file (#4251)	6 years ago
cassandra	Per-thread unique test db names (#4135)	7 years ago
checkpoint	Per-thread unique test db names (#4135)	7 years ago
compaction_filters	Comment out unused variables	7 years ago
convenience	Change RocksDB License	8 years ago
date_tiered	Per-thread unique test db names (#4135)	7 years ago
document	Suppress clang analyzer error (#4299)	6 years ago
geodb	Per-thread unique test db names (#4135)	7 years ago
leveldb_options	Change RocksDB License	8 years ago
lua	Per-thread unique test db names (#4135)	7 years ago
memory	Per-thread unique test db names (#4135)	7 years ago
merge_operators	Per-thread unique test db names (#4135)	7 years ago
option_change_migration	comment unused parameters to turn on -Wunused-parameter flag	7 years ago
options	Per-thread unique test db names (#4135)	7 years ago
persistent_cache	fix compilation with g++ option `-Wsuggest-override` (#4272)	6 years ago
redis	Per-thread unique test db names (#4135)	7 years ago
simulator_cache	Add path to WritableFileWriter. (#4039)	6 years ago
spatialdb	Per-thread unique test db names (#4135)	7 years ago
table_properties_collectors	Reduce runtime of compact_on_deletion_collector_test (#4117)	7 years ago
trace	Add path to WritableFileWriter. (#4039)	6 years ago
transactions	Suppress clang analyzer error (#4299)	6 years ago
ttl	Per-thread unique test db names (#4135)	7 years ago
write_batch_with_index	WriteUnPrepared: Implement unprepared batches for transactions (#4104)	7 years ago
col_buf_decoder.cc	Change RocksDB License	8 years ago
col_buf_decoder.h	Comment out unused variables	7 years ago
col_buf_encoder.cc	fix shift UBSAN error in col_buf_encoder.cc	7 years ago
col_buf_encoder.h	Change RocksDB License	8 years ago
column_aware_encoding_exp.cc	fix memory leak in two_level_iterator	7 years ago
column_aware_encoding_test.cc	Change RocksDB License	8 years ago
column_aware_encoding_util.cc	Digest ZSTD compression dictionary once per SST file (#4251)	6 years ago
column_aware_encoding_util.h	Move prefix_extractor to MutableCFOptions	7 years ago
debug.cc	GetAllKeyVersions() to take an extra argument of `max_num_ikeys`. (#4271)	6 years ago
env_librados.cc	Suppress lint in old files	7 years ago
env_librados.md	Add EnvLibrados - RocksDB Env of RADOS (#1222)	9 years ago
env_librados_test.cc	Change RocksDB License	8 years ago
env_mirror.cc	Add path to WritableFileWriter. (#4039)	6 years ago
env_mirror_test.cc	Change RocksDB License	8 years ago
env_timed.cc	comment unused parameters to turn on -Wunused-parameter flag	7 years ago
env_timed_test.cc	fix memory leak in two_level_iterator	7 years ago
merge_operators.h	Support StringAppendOperator(delimiter_char) constructor in java-api	7 years ago
object_registry_test.cc	fix memory leak in two_level_iterator	7 years ago
util_merge_operators_test.cc	Change RocksDB License	8 years ago