rocksdb

fork of https://github.com/oxigraph/rocksdb and https://github.com/facebook/rocksdb for nextgraph and oxigraph

Levi Tamasi cb25bc1128 Update compaction statistics to include the amount of data read from blob files (#8022 ) Summary: The patch does the following: 1) Exposes the amount of data (number of bytes) read from blob files from `BlobFileReader::GetBlob` / `Version::GetBlob`. 2) Tracks the total number and size of blobs read from blob files during a compaction (due to garbage collection or compaction filter usage) in `CompactionIterationStats` and propagates this data to `InternalStats::CompactionStats` / `CompactionJobStats`. 3) Updates the formulae for write amplification calculations to include the amount of data read from blob files. 4) Extends the compaction stats dump with a new column `Rblob(GB)` and a new line containing the total number and size of blob files in the current `Version` to complement the information about the shape and size of the LSM tree that's already there. 5) Updates `CompactionJobStats` so that the number of files and amount of data written by a compaction are broken down per file type (i.e. table/blob file). Pull Request resolved: https://github.com/facebook/rocksdb/pull/8022 Test Plan: Ran `make check` and `db_bench`. Reviewed By: riversand963 Differential Revision: D26801199 Pulled By: ltamasi fbshipit-source-id: 28a5f072048a702643b28cb5971b4099acabbfb2		4 years ago
.circleci	Add circleci format_compatible nightly build (#7926 )	4 years ago
.github/workflows	Update clang-format-diff.py path (#7944 )	4 years ago
buckifier	Update zstd in buck build (#7923 )	4 years ago
build_tools	Extract test cases correctly in run_ci_db_test.ps1 script (#7989 )	4 years ago
cache	Add a SystemClock class to capture the time functions of an Env (#7858 )	4 years ago
cmake	Add `find_dependency()` in cmake config file. (#6791 )	5 years ago
coverage	Find the correct gcov (#6904 )	5 years ago
db	Update compaction statistics to include the amount of data read from blob files (#8022 )	4 years ago
db_stress_tool	Enable compact filter for blob in dbstress and dbbench (#8011 )	4 years ago
docs	Update github-pages and dependencies (#7850 )	4 years ago
env	Wal recovery failure with encryption due to zero bytes WAL size. (#7924 )	4 years ago
examples	Bring the Configurable options together (#5753 )	4 years ago
file	Make BlockBasedTable::kMaxAutoReadAheadSize configurable (#7951 )	4 years ago
fuzz	Remove Legacy and Custom FileWrapper classes from header files (#7851 )	4 years ago
hdfs	fix build with 'USE_HDFS' on windows (#6950 )	5 years ago
include/rocksdb	Update compaction statistics to include the amount of data read from blob files (#8022 )	4 years ago
java	Limit buffering for collecting samples for compression dictionary (#7970 )	4 years ago
logging	Add a SystemClock class to capture the time functions of an Env (#7858 )	4 years ago
memory	slightly improve jemalloc allocator API header (#7592 )	4 years ago
memtable	Feature: add SetBufferSize() so that managed size can be dynamic (#7961 )	4 years ago
monitoring	Still use SystemClock* instead of shared_ptr in StepPerfTimer (#8006 )	4 years ago
options	Make BlockBasedTable::kMaxAutoReadAheadSize configurable (#7951 )	4 years ago
plugin	Makefile support to statically link external plugin code (#7918 )	4 years ago
port	Update win_logger.cc : assert failed when return value not checked. (-DROCKSDB_ASSERT_STATUS_CHECKED) (#7955 )	4 years ago
table	Refine Ribbon configuration, improve testing, add Homogeneous (#7879 )	4 years ago
test_util	Add a SystemClock class to capture the time functions of an Env (#7858 )	4 years ago
third-party	Fix Compilation on ppc64le using Clang 11 (#7713 )	4 years ago
tools	Revamp check_format_compatible.sh (#8012 )	4 years ago
trace_replay	Introduce a new trace file format (v 0.2) for better extension (#7977 )	4 years ago
util	Update compaction statistics to include the amount of data read from blob files (#8022 )	4 years ago
utilities	Support retrieving checksums for blob files from the MANIFEST when checkpointing (#8003 )	4 years ago
.clang-format	A script that automatically reformat affected lines	11 years ago
.gitignore	gitignore cmake-build-* for CLion integration (#7933 )	4 years ago
.lgtm.yml	Create lgtm.yml for LGTM.com C/C++ analysis (#4058 )	7 years ago
.travis.yml	Cleanup Travis CI config (#7848 )	4 years ago
.watchmanconfig	Added .watchmanconfig file to rocksdb repo (#5593 )	6 years ago
AUTHORS	Update RocksDB Authors File	7 years ago
CMakeLists.txt	Refine Ribbon configuration, improve testing, add Homogeneous (#7879 )	4 years ago
CODE_OF_CONDUCT.md	Adopt Contributor Covenant	5 years ago
CONTRIBUTING.md	Add Code of Conduct	7 years ago
COPYING	Add GPLv2 as an alternative license.	8 years ago
DEFAULT_OPTIONS_HISTORY.md	options.delayed_write_rate use the rate of rate_limiter by default.	8 years ago
DUMP_FORMAT.md	First version of rocksdb_dump and rocksdb_undump.	10 years ago
HISTORY.md	Update compaction statistics to include the amount of data read from blob files (#8022 )	4 years ago
INSTALL.md	Update the version of the dependencies used by the RocksJava static build (#4761 )	6 years ago
LANGUAGE-BINDINGS.md	Add RestoreDBFromLatestBackup to C API, add new C# package (#7092 )	5 years ago
LICENSE.Apache	Change RocksDB License	8 years ago
LICENSE.leveldb	Add back the LevelDB license file	8 years ago
Makefile	Compaction filter support for (new) BlobDB (#7974 )	4 years ago
PLUGINS.md	Makefile support to statically link external plugin code (#7918 )	4 years ago
README.md	Fix the CI badge for ppc64le Jenkins (#7561 )	4 years ago
ROCKSDB_LITE.md	Fix some typos in comments and docs.	7 years ago
TARGETS	Refine Ribbon configuration, improve testing, add Homogeneous (#7879 )	4 years ago
USERS.md	Add Apache Doris to USERS (#7865 )	4 years ago
Vagrantfile	Adding CentOS 7 Vagrantfile & build script	7 years ago
WINDOWS_PORT.md	#5145 , rename port/dirent.h to port/port_dirent.h to avoid compile err when use port dir as header dir output (#5152 )	6 years ago
appveyor.yml	Remove 2019 from appveyor (#7038 )	5 years ago
defs.bzl	Make testpilot recognize that these tests have coverage instrumentation	5 years ago
issue_template.md	Add Google Group to Issue Template	5 years ago
src.mk	Refine Ribbon configuration, improve testing, add Homogeneous (#7879 )	4 years ago
thirdparty.inc	Fix build jemalloc api (#5470 )	6 years ago

README.md

RocksDB: A Persistent Key-Value Store for Flash and RAM Storage

RocksDB is developed and maintained by Facebook Database Engineering Team. It is built on earlier work on LevelDB by Sanjay Ghemawat (sanjay@google.com) and Jeff Dean (jeff@google.com)

This code is a library that forms the core building block for a fast key-value server, especially suited for storing data on flash drives. It has a Log-Structured-Merge-Database (LSM) design with flexible tradeoffs between Write-Amplification-Factor (WAF), Read-Amplification-Factor (RAF) and Space-Amplification-Factor (SAF). It has multi-threaded compactions, making it especially suitable for storing multiple terabytes of data in a single database.

Start with example usage here: https://github.com/facebook/rocksdb/tree/master/examples

See the github wiki for more explanation.

The public interface is in include/. Callers should not include or rely on the details of any other header files in this package. Those internal APIs may be changed without warning.

Design discussions are conducted in https://www.facebook.com/groups/rocksdb.dev/ and https://rocksdb.slack.com/

License

RocksDB is dual-licensed under both the GPLv2 (found in the COPYING file in the root directory) and Apache 2.0 License (found in the LICENSE.Apache file in the root directory). You may select, at your option, one of the above-listed licenses.