rocksdb

fork of https://github.com/oxigraph/rocksdb and https://github.com/facebook/rocksdb for nextgraph and oxigraph

Peter Dillinger ef443cead4 Refactor to avoid confusing "raw block" (#10408 ) Summary: We have a lot of confusing code because of mixed, sometimes completely opposite uses of of the term "raw block" or "raw contents", sometimes within the same source file. For example, in `BlockBasedTableBuilder`, `raw_block_contents` and `raw_size` generally referred to uncompressed block contents and size, while `WriteRawBlock` referred to writing a block that is already compressed if it is going to be. Meanwhile, in `BlockBasedTable`, `raw_block_contents` either referred to a (maybe compressed) block with trailer, or a maybe compressed block maybe without trailer. (Note: left as follow-up work to use C++ typing to better sort out the various kinds of BlockContents.) This change primarily tries to apply some consistent terminology around the kinds of block representations, avoiding the unclear "raw". (Any meaning of "raw" assumes some bias toward the storage layer or toward the logical data layer.) Preferred terminology: * Serialized block - bytes that go into storage. For block-based table (usually the case) this includes the block trailer. WART: block `size` may or may not include the trailer; need to be clear about whether it does or not. * Maybe compressed block - like a serialized block, but without the trailer (or no promise of including a trailer). Must be accompanied by a CompressionType. * Uncompressed block - "payload" bytes that are either stored with no compression, used as input to compression function, or result of decompression function. * Parsed block - an in-memory form of a block in block cache, as it is used by the table reader. Different C++ types are used depending on the block type (see block_like_traits.h). Other refactorings: * Misc corrections/improvements of internal API comments * Remove a few misleading / unhelpful / redundant comments. * Use move semantics in some places to simplify contracts * Use better parameter names to indicate which parameters are used for outputs * Remove some extraneous `extern` * Various clean-ups to `CacheDumperImpl` (mostly unnecessary code) Pull Request resolved: https://github.com/facebook/rocksdb/pull/10408 Test Plan: existing tests Reviewed By: akankshamahajan15 Differential Revision: D38172617 Pulled By: pdillinger fbshipit-source-id: ccb99299f324ac5ca46996d34c5089621a4f260c		2 years ago
.circleci	Build and link libfolly with RocksDB (#10103 )	2 years ago
.github/workflows	Use released clang-format instead of the one from dev branch (#9646 )	3 years ago
buckifier	Enable BLACK for internal_repo_rocksdb (#10710 )	2 years ago
build_tools	Fix lint issues after enable BLACK (#10717 )	2 years ago
cache	Clarify comments for cache priorities and pool options (#10718 )	2 years ago
cmake	gcc-11 and cmake related cleanup (#9286 )	3 years ago
coverage	Enable BLACK for internal_repo_rocksdb (#10710 )	2 years ago
db	Refactor to avoid confusing "raw block" (#10408 )	2 years ago
db_stress_tool	Disable tiered storage + BlobDB stress test (#10699 )	2 years ago
docs	Make header more natural. (#10580 )	2 years ago
env	Fix sqe->addr passed in cancel request in io_uring (#10644 )	2 years ago
examples	Add a simple example of backup and restore (#10054 )	3 years ago
file	Async optimization in scan path (#10602 )	2 years ago
fuzz	Add some missing headers (#10519 )	2 years ago
include/rocksdb	Refactor to avoid confusing "raw block" (#10408 )	2 years ago
java	Update statistics for async scan readaheads (#10585 )	2 years ago
logging	Fix build error with NIOSTATS_CONTEXT (#10506 )	2 years ago
memory	Add a blob-specific cache priority (#10461 )	2 years ago
memtable	Migrate to docker for CI run (#10496 )	2 years ago
microbench	Avoid allocations/copies for large `GetMergeOperands()` results (#10458 )	2 years ago
monitoring	Inject spurious wakeup and sleep before acquiring db mutex to expose race condition (#10291 )	2 years ago
options	Change MultiGet multi-level optimization to default on (#10671 )	2 years ago
plugin	Add initial CMake support to plugin (#9214 )	3 years ago
port	Fix FreeBSD building (#10575 )	2 years ago
table	Refactor to avoid confusing "raw block" (#10408 )	2 years ago
test_util	Add seqno to time mapping (#10338 )	2 years ago
third-party	Meta-internal folly integration with F14FastMap (#9546 )	3 years ago
tools	Fix lint issues after enable BLACK (#10717 )	2 years ago
trace_replay	Use std::numeric_limits<> (#9954 )	3 years ago
util	Disable RateLimiterTest.Rate with valgrind (#10637 )	2 years ago
utilities	Refactor to avoid confusing "raw block" (#10408 )	2 years ago
.clang-format	A script that automatically reformat affected lines	11 years ago
.gitignore	Snapshots with user-specified timestamps (#9879 )	3 years ago
.lgtm.yml	Create lgtm.yml for LGTM.com C/C++ analysis (#4058 )	7 years ago
.watchmanconfig	Added .watchmanconfig file to rocksdb repo (#5593 )	6 years ago
AUTHORS	Update RocksDB Authors File	7 years ago
CMakeLists.txt	Build and link libfolly with RocksDB (#10103 )	2 years ago
CODE_OF_CONDUCT.md	Adopt Contributor Covenant	5 years ago
CONTRIBUTING.md	Add Code of Conduct	7 years ago
COPYING	Add GPLv2 as an alternative license.	8 years ago
DEFAULT_OPTIONS_HISTORY.md	Add Options::DisableExtraChecks, clarify force_consistency_checks (#9363 )	3 years ago
DUMP_FORMAT.md	First version of rocksdb_dump and rocksdb_undump.	10 years ago
HISTORY.md	Mention in HISTORY.md the fix in #10705 (#10720 )	2 years ago
INSTALL.md	Update supported VS versions in INSTALL.md (#9823 )	3 years ago
LANGUAGE-BINDINGS.md	Add grocksdb in Go language bindings (#10498 )	2 years ago
LICENSE.Apache	Change RocksDB License	8 years ago
LICENSE.leveldb	Add back the LevelDB license file	8 years ago
Makefile	Fix platform 10 build with folly (#10708 )	2 years ago
PLUGINS.md	Add pmem-rocksdb-plugin link in PLUGINs.md (#9934 )	3 years ago
README.md	Remove Travis CI (#10407 )	2 years ago
ROCKSDB_LITE.md	Fix remaining uses of "backupable" (#9792 )	3 years ago
TARGETS	Fix an import issue in fbcode. (#10604 )	2 years ago
USERS.md	Add Solana's RocksDB use case in USERS.md (#9558 )	3 years ago
Vagrantfile	Adding CentOS 7 Vagrantfile & build script	7 years ago
WINDOWS_PORT.md	Update branch name in WINDOWS_PORT.md (#8745 )	3 years ago
common.mk	Clean up variables for temporary directory (#9961 )	3 years ago
crash_test.mk	minor cleanups to db_crashtest.py (#10654 )	2 years ago
issue_template.md	Add Google Group to Issue Template	5 years ago
rocksdb.pc.in	build: fix pkg-config file generation (#9953 )	3 years ago
src.mk	Fix an import issue in fbcode. (#10604 )	2 years ago
thirdparty.inc	Fix build jemalloc api (#5470 )	6 years ago

README.md

RocksDB: A Persistent Key-Value Store for Flash and RAM Storage

RocksDB is developed and maintained by Facebook Database Engineering Team. It is built on earlier work on LevelDB by Sanjay Ghemawat (sanjay@google.com) and Jeff Dean (jeff@google.com)

This code is a library that forms the core building block for a fast key-value server, especially suited for storing data on flash drives. It has a Log-Structured-Merge-Database (LSM) design with flexible tradeoffs between Write-Amplification-Factor (WAF), Read-Amplification-Factor (RAF) and Space-Amplification-Factor (SAF). It has multi-threaded compactions, making it especially suitable for storing multiple terabytes of data in a single database.

Start with example usage here: https://github.com/facebook/rocksdb/tree/main/examples

See the github wiki for more explanation.

The public interface is in include/. Callers should not include or rely on the details of any other header files in this package. Those internal APIs may be changed without warning.

Questions and discussions are welcome on the RocksDB Developers Public Facebook group and email list on Google Groups.

License

RocksDB is dual-licensed under both the GPLv2 (found in the COPYING file in the root directory) and Apache 2.0 License (found in the LICENSE.Apache file in the root directory). You may select, at your option, one of the above-listed licenses.