fork of https://github.com/oxigraph/rocksdb and https://github.com/facebook/rocksdb for nextgraph and oxigraph

You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

Zhongyi Xie 26015f3b48 add compression options to table properties (#5081) ... Summary: Since we are planning to use dictionary compression and to use different compression level, it is quite useful to add compression options to TableProperties. For example, in MyRocks, if the feature is available, we can query from information_schema.rocksdb_sst_props to see if all sst files are converted to ZSTD dictionary compressions. Resolves https://github.com/facebook/rocksdb/issues/4992 With this PR, user can query table properties through `GetPropertiesOfAllTables` API and get compression options as std::string: `window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0;` or table_properties->ToString() will also contain it `# data blocks=1; # entries=13; # deletions=0; # merge operands=0; # range deletions=0; raw key size=143; raw average key size=11.000000; raw value size=39; raw average value size=3.000000; data block size=120; index block size (user-key? 0, delta-value? 0)=27; filter block size=0; (estimated) table size=147; filter policy name=N/A; prefix extractor name=nullptr; column family ID=0; column family name=default; comparator name=leveldb.BytewiseComparator; merge operator name=nullptr; property collectors names=[]; SST file compression algo=Snappy; SST file compression options=window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ; creation time=1552946632; time stamp of earliest key=1552946632;` Pull Request resolved: https://github.com/facebook/rocksdb/pull/5081 Differential Revision: D14716692 Pulled By: miasantreble fbshipit-source-id: 7d2f2cf84e052bff876e71b4212cfdebf5be32dd		6 years ago
..
adaptive_table_factory.cc	Update all unique/shared_ptr instances to be qualified with namespace std (#4638)	6 years ago
adaptive_table_factory.h	Remove some "using std::..." from header files. (#5113)	6 years ago
block.cc	Apply automatic formatting to some files (#5114)	6 years ago
block.h	Apply automatic formatting to some files (#5114)	6 years ago
block_based_filter_block.cc	Apply automatic formatting to some files (#5114)	6 years ago
block_based_filter_block.h	Apply automatic formatting to some files (#5114)	6 years ago
block_based_filter_block_test.cc	Apply automatic formatting to some files (#5114)	6 years ago
block_based_table_builder.cc	add compression options to table properties (#5081)	6 years ago
block_based_table_builder.h	Feature for sampling and reporting compressibility (#4842)	6 years ago
block_based_table_factory.cc	Feature for sampling and reporting compressibility (#4842)	6 years ago
block_based_table_factory.h	Remove some "using std::..." from header files. (#5113)	6 years ago
block_based_table_reader.cc	Revert "Avoid per-key upper bound check in BlockBasedTableIterator (#5101)" (#5132)	6 years ago
block_based_table_reader.h	Revert "Avoid per-key upper bound check in BlockBasedTableIterator (#5101)" (#5132)	6 years ago
block_builder.cc	Apply automatic formatting to some files (#5114)	6 years ago
block_builder.h	Apply automatic formatting to some files (#5114)	6 years ago
block_fetcher.cc	Apply automatic formatting to some files (#5114)	6 years ago
block_fetcher.h	Cache dictionary used for decompressing data blocks (#4881)	6 years ago
block_prefix_index.cc	Apply automatic formatting to some files (#5114)	6 years ago
block_prefix_index.h	Apply automatic formatting to some files (#5114)	6 years ago
block_test.cc	Apply automatic formatting to some files (#5114)	6 years ago
bloom_block.cc	Change RocksDB License	7 years ago
bloom_block.h	Disallow customized hash function in DynamicBloom (#4915)	6 years ago
cleanable_test.cc	Change RocksDB License	7 years ago
cuckoo_table_builder.cc	Promote rocksdb.{deleted.keys,merge.operands} to main table properties (#4594)	6 years ago
cuckoo_table_builder.h	Change RocksDB License	7 years ago
cuckoo_table_builder_test.cc	Update all unique/shared_ptr instances to be qualified with namespace std (#4638)	6 years ago
cuckoo_table_factory.cc	Update all unique/shared_ptr instances to be qualified with namespace std (#4638)	6 years ago
cuckoo_table_factory.h	Update all unique/shared_ptr instances to be qualified with namespace std (#4638)	6 years ago
cuckoo_table_reader.cc	Apply modernize-use-override (2nd iteration)	6 years ago
cuckoo_table_reader.h	Index value delta encoding (#3983)	6 years ago
cuckoo_table_reader_test.cc	Update all unique/shared_ptr instances to be qualified with namespace std (#4638)	6 years ago
data_block_footer.cc	Add db_bench options of data block hash index (#4281)	6 years ago
data_block_footer.h	Add db_bench options of data block hash index (#4281)	6 years ago
data_block_hash_index.cc	DataBlockHashIndex: Remove the division from EstimateSize() (#4293)	6 years ago
data_block_hash_index.h	DataBlockHashIndex: Remove the division from EstimateSize() (#4293)	6 years ago
data_block_hash_index_test.cc	Feature for sampling and reporting compressibility (#4842)	6 years ago
filter_block.h	use user_key and iterate_upper_bound to determine compatibility of bloom filters (#3899)	6 years ago
flush_block_policy.cc	Apply modernize-use-override (2nd iteration)	6 years ago
format.cc	Apply automatic formatting to some files (#5114)	6 years ago
format.h	Apply automatic formatting to some files (#5114)	6 years ago
full_filter_bits_builder.h	Skip duplicate bloom keys when whole_key and prefix are mixed	7 years ago
full_filter_block.cc	Charging block cache more accurately (#4073)	6 years ago
full_filter_block.h	Charging block cache more accurately (#4073)	6 years ago
full_filter_block_test.cc	Apply modernize-use-override (2nd iteration)	6 years ago
get_context.cc	PlainTable should avoid copying Get() results from immortal source. (#4924)	6 years ago
get_context.h	Cache dictionary used for decompressing data blocks (#4881)	6 years ago
index_builder.cc	Add path to WritableFileWriter. (#4039)	6 years ago
index_builder.h	Add path to WritableFileWriter. (#4039)	6 years ago
internal_iterator.h	Index value delta encoding (#3983)	6 years ago
iter_heap.h	Make InternalKeyComparator final and directly use it in merging iterator	7 years ago
iterator.cc	Fix arena allocation size in NewEmptyInternalIterator (#4905)	6 years ago
iterator_wrapper.h	Index value delta encoding (#3983)	6 years ago
merger_test.cc	Apply modernize-use-override (2nd iteration)	6 years ago
merging_iterator.cc	Apply modernize-use-override (2nd iteration)	6 years ago
merging_iterator.h	Index value delta encoding (#3983)	6 years ago
meta_blocks.cc	add compression options to table properties (#5081)	6 years ago
meta_blocks.h	Feature for sampling and reporting compressibility (#4842)	6 years ago
mock_table.cc	Update all unique/shared_ptr instances to be qualified with namespace std (#4638)	6 years ago
mock_table.h	Update all unique/shared_ptr instances to be qualified with namespace std (#4638)	6 years ago
partitioned_filter_block.cc	Fix bug in partition filters with format_version=4 (#4381)	6 years ago
partitioned_filter_block.h	Fix bug in partition filters with format_version=4 (#4381)	6 years ago
partitioned_filter_block_test.cc	Apply modernize-use-override (2nd iteration)	6 years ago
persistent_cache_helper.cc	Remove two variables from BlockContents class and don't use class Block for compressed block (#4650)	6 years ago
persistent_cache_helper.h	Change RocksDB License	7 years ago
persistent_cache_options.h	Change RocksDB License	7 years ago
plain_table_builder.cc	Revert "Remove PlainTable's feature store_index_in_file (#4914)" (#5034)	6 years ago
plain_table_builder.h	Revert "Remove PlainTable's feature store_index_in_file (#4914)" (#5034)	6 years ago
plain_table_factory.cc	Revert "Remove PlainTable's feature store_index_in_file (#4914)" (#5034)	6 years ago
plain_table_factory.h	Remove some "using std::..." from header files. (#5113)	6 years ago
plain_table_index.cc	Change RocksDB License	7 years ago
plain_table_index.h	Move prefix_extractor to MutableCFOptions	7 years ago
plain_table_key_coding.cc	Comment out unused variables	7 years ago
plain_table_key_coding.h	Update all unique/shared_ptr instances to be qualified with namespace std (#4638)	6 years ago
plain_table_reader.cc	Remove some "using std::..." from header files. (#5113)	6 years ago
plain_table_reader.h	Remove some "using std::..." from header files. (#5113)	6 years ago
scoped_arena_iterator.h	Change RocksDB License	7 years ago
sst_file_reader.cc	Fix SstFileReader not able to open ingested file (#5097)	6 years ago
sst_file_reader_test.cc	Fix SstFileReader not able to open ingested file (#5097)	6 years ago
sst_file_writer.cc	Feature for sampling and reporting compressibility (#4842)	6 years ago
sst_file_writer_collectors.h	Fix SstFileReader not able to open ingested file (#5097)	6 years ago
table_builder.h	Feature for sampling and reporting compressibility (#4842)	6 years ago
table_properties.cc	add compression options to table properties (#5081)	6 years ago
table_properties_internal.h	Index value delta encoding (#3983)	6 years ago
table_reader.h	Clean up FragmentedRangeTombstoneList (#4692)	6 years ago
table_reader_bench.cc	Feature for sampling and reporting compressibility (#4842)	6 years ago
table_test.cc	Revert "Avoid per-key upper bound check in BlockBasedTableIterator (#5101)" (#5132)	6 years ago
two_level_iterator.cc	Apply modernize-use-override (2nd iteration)	6 years ago
two_level_iterator.h	Index value delta encoding (#3983)	6 years ago