rocksdb

fork of https://github.com/oxigraph/rocksdb and https://github.com/facebook/rocksdb for nextgraph and oxigraph

Ewout Prangsma 51778612c9 Encryption at rest support Summary: This PR adds support for encrypting data stored by RocksDB when written to disk. It adds an `EncryptedEnv` override of the `Env` class with matching overrides for sequential&random access files. The encryption itself is done through a configurable `EncryptionProvider`. This class creates is asked to create `BlockAccessCipherStream` for a file. This is where the actual encryption/decryption is being done. Currently there is a Counter mode implementation of `BlockAccessCipherStream` with a `ROT13` block cipher (NOTE the `ROT13` is for demo purposes only!!). The Counter operation mode uses an initial counter & random initialization vector (IV). Both are created randomly for each file and stored in a 4K (default size) block that is prefixed to that file. The `EncryptedEnv` implementation is such that clients of the `Env` class do not see this prefix (nor data, nor in filesize). The largest part of the prefix block is also encrypted, and there is room left for implementation specific settings/values/keys in there. To test the encryption, the `DBTestBase` class has been extended to consider a new environment variable called `ENCRYPTED_ENV`. If set, the test will setup a encrypted instance of the `Env` class to use for all tests. Typically you would run it like this: ``` ENCRYPTED_ENV=1 make check_some ``` There is also an added test that checks that some data inserted into the database is or is not "visible" on disk. With `ENCRYPTED_ENV` active it must not find plain text strings, with `ENCRYPTED_ENV` unset, it must find the plain text strings. Closes https://github.com/facebook/rocksdb/pull/2424 Differential Revision: D5322178 Pulled By: sdwilsh fbshipit-source-id: 253b0a9c2c498cc98f580df7f2623cbf7678a27f		7 years ago
arcanist_util	Fix arc setting for Facebook internal tools	8 years ago
buckifier	update buckifer/TARGETS	8 years ago
build_tools	fixed typo	8 years ago
cache	Add GPLv2 as an alternative license.	8 years ago
cmake/modules	CMake: more MinGW fixes	8 years ago
coverage	Fix coverage script	10 years ago
db	Encryption at rest support	7 years ago
docs	Intra-L0 blog post	7 years ago
env	Encryption at rest support	7 years ago
examples	CMake: more MinGW fixes	8 years ago
hdfs	New API for background work in single thread pool	8 years ago
include/rocksdb	Encryption at rest support	7 years ago
java	Fix jni WriteBatchThreadedTest	7 years ago
memtable	WriteBufferManager will not trigger flush if much data is already being flushed	7 years ago
monitoring	revert perf_context and io_stats to __thread	7 years ago
options	Optimize for serial commits in 2PC	7 years ago
port	Implement ReopenWritibaleFile on Windows and other fixes	8 years ago
table	Unit Tests for sync, range sync and file close failures	7 years ago
third-party	fixed typo	8 years ago
tools	Remove pin_slice option by making it the default	8 years ago
util	revert perf_context and io_stats to __thread	7 years ago
utilities	Optimize for serial commits in 2PC	7 years ago
.clang-format	A script that automatically reformat affected lines	11 years ago
.deprecated_arcconfig	Update ShipIt to honor TARGETS updates	8 years ago
.gitignore	Simple blob file dumper	8 years ago
.travis.yml	Force travis to build with clang on MacOS	8 years ago
AUTHORS	Add AUTHORS file. Fix #203	10 years ago
CMakeLists.txt	Encryption at rest support	7 years ago
CONTRIBUTING.md	facebook accounts are not required for CLA signers	10 years ago
COPYING	Add GPLv2 as an alternative license.	8 years ago
DEFAULT_OPTIONS_HISTORY.md	options.delayed_write_rate use the rate of rate_limiter by default.	8 years ago
DUMP_FORMAT.md	First version of rocksdb_dump and rocksdb_undump.	10 years ago
HISTORY.md	Trivial typo in HISTORY.md	7 years ago
INSTALL.md	cross-platform compatibility improvements	8 years ago
LANGUAGE-BINDINGS.md	Adding Dlang to the list	8 years ago
LICENSE	Updated all copyright headers to the new format.	9 years ago
Makefile	Encryption at rest support	7 years ago
PATENTS	Update Patent Grant.	10 years ago
README.md	Appveyor badge to show master branch	8 years ago
ROCKSDB_LITE.md	Optimistic Transactions	10 years ago
TARGETS	Encryption at rest support	7 years ago
USERS.md	fixed typo	8 years ago
Vagrantfile	Update Vagrant file (test internal phabricator workflow)	8 years ago
WINDOWS_PORT.md	Commit both PR and internal code review changes	9 years ago
appveyor.yml	Rework test running script.	8 years ago
src.mk	Encryption at rest support	7 years ago
thirdparty.inc	Introduce XPRESS compresssion on Windows. (#1081 )	9 years ago

README.md

RocksDB: A Persistent Key-Value Store for Flash and RAM Storage

RocksDB is developed and maintained by Facebook Database Engineering Team. It is built on earlier work on LevelDB by Sanjay Ghemawat (sanjay@google.com) and Jeff Dean (jeff@google.com)

This code is a library that forms the core building block for a fast key value server, especially suited for storing data on flash drives. It has a Log-Structured-Merge-Database (LSM) design with flexible tradeoffs between Write-Amplification-Factor (WAF), Read-Amplification-Factor (RAF) and Space-Amplification-Factor (SAF). It has multi-threaded compactions, making it specially suitable for storing multiple terabytes of data in a single database.

Start with example usage here: https://github.com/facebook/rocksdb/tree/master/examples

See the github wiki for more explanation.

The public interface is in include/. Callers should not include or rely on the details of any other header files in this package. Those internal APIs may be changed without warning.

Design discussions are conducted in https://www.facebook.com/groups/rocksdb.dev/