rocksdb

fork of https://github.com/oxigraph/rocksdb and https://github.com/facebook/rocksdb for nextgraph and oxigraph

Jay Edgar efd013d6d8 Miscellaneous performance improvements Summary: I was investigating performance issues in the SstFileWriter and found all of the following: - The SstFileWriter::Add() function created a local InternalKey every time it was called generating a allocation and free each time. Changed to have an InternalKey member variable that can be reset with the new InternalKey::Set() function. - In SstFileWriter::Add() the smallest_key and largest_key values were assigned the result of a ToString() call, but it is simpler to just assign them directly from the user's key. - The Slice class had no move constructor so each time one was returned from a function a new one had to be allocated, the old data copied to the new, and the old one was freed. I added the move constructor which also required a copy constructor and assignment operator. - The BlockBuilder::CurrentSizeEstimate() function calculates the current estimate size, but was being called 2 or 3 times for each key added. I changed the class to maintain a running estimate (equal to the original calculation) so that the function can return an already calculated value. - The code in BlockBuilder::Add() that calculated the shared bytes between the last key and the new key duplicated what Slice::difference_offset does, so I replaced it with the standard function. - BlockBuilder::Add() had code to copy just the changed portion into the last key value (and asserted that it now matched the new key). It is more efficient just to copy the whole new key over. - Moved this same code up into the 'if (use_delta_encoding_)' since the last key value is only needed when delta encoding is on. - FlushBlockBySizePolicy::BlockAlmostFull calculated a standard deviation value each time it was called, but this information would only change if block_size of block_size_deviation changed, so I created a member variable to hold the value to avoid the calculation each time. - Each PutVarint??() function has a buffer and calls std::string::append(). Two or three calls in a row could share a buffer and a single call to std::string::append(). Some of these will be helpful outside of the SstFileWriter. I'm not 100% the addition of the move constructor is appropriate as I wonder why this wasn't done before - maybe because of compiler compatibility? I tried it on gcc 4.8 and 4.9. Test Plan: The changes should not affect the results so the existing tests should all still work and no new tests were added. The value of the changes was seen by manually testing the SstFileWriter class through MyRocks and adding timing code to identify problem areas. Reviewers: sdong, IslamAbdelRahman Reviewed By: IslamAbdelRahman Subscribers: andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D59607		9 years ago
arcanist_util	Add release build to RocksDB per-diff/post-commit tests	9 years ago
build_tools	Add release build to RocksDB per-diff/post-commit tests	9 years ago
coverage	Fix coverage script	10 years ago
db	Miscellaneous performance improvements	9 years ago
doc	Lint everything	9 years ago
examples	Fix examples/Makefile jemalloc error	9 years ago
hdfs	using java7 in runtime for hdfs env (#1072 )	9 years ago
include/rocksdb	Fix Log() doc for default level	9 years ago
java	Add options.write_buffer_manager: control total memtable size across DB instances	9 years ago
memtable	Updated all copyright headers to the new format.	9 years ago
port	stack_trace,cc: The current Stacktrace code does not compile for FreeBSD (#1153 )	9 years ago
table	Miscellaneous performance improvements	9 years ago
third-party	Fix clang analyzer errors	9 years ago
tools	Fix clang analyzer errors	9 years ago
util	Miscellaneous performance improvements	9 years ago
utilities	Fix clang analyzer errors	9 years ago
.arcconfig	Integrate Jenkins with Phabricator	10 years ago
.clang-format	A script that automatically reformat affected lines	11 years ago
.gitignore	Ignore db_test2	9 years ago
.travis.yml	Re-enable linux on travis	9 years ago
AUTHORS	Add AUTHORS file. Fix #203	10 years ago
CMakeLists.txt	Fixed Minor Bug on Windows Build and db_bench_tool.cc (#1189 )	9 years ago
CONTRIBUTING.md	facebook accounts are not required for CLA signers	11 years ago
DEFAULT_OPTIONS_HISTORY.md	Release RocksDB 4.8.0	9 years ago
DUMP_FORMAT.md	First version of rocksdb_dump and rocksdb_undump.	10 years ago
HISTORY.md	Add options.write_buffer_manager: control total memtable size across DB instances	9 years ago
INSTALL.md	Simple changes to support builds for ppc64[le] consistent with X86	9 years ago
LANGUAGE-BINDINGS.md	Merge pull request #1056 from facebook/igorcanadi-patch-1	9 years ago
LICENSE	Updated all copyright headers to the new format.	9 years ago
Makefile	Update Makefile to fix dependency	9 years ago
PATENTS	Update Patent Grant.	10 years ago
README.md	Replaced "built on on earlier work" by "built on earlier work" in README.md	10 years ago
ROCKSDB_LITE.md	Optimistic Transactions	10 years ago
USERS.md	Update USER.md to include more services at Facebook.	9 years ago
Vagrantfile	RocksDB on FreeBSD support	10 years ago
WINDOWS_PORT.md	Commit both PR and internal code review changes	10 years ago
appveyor.yml	Disable long running GroupCommitTest (#1125 )	9 years ago
src.mk	Update Makefile to fix dependency	9 years ago
thirdparty.inc	Introduce XPRESS compresssion on Windows. (#1081 )	9 years ago

README.md

RocksDB: A Persistent Key-Value Store for Flash and RAM Storage

RocksDB is developed and maintained by Facebook Database Engineering Team. It is built on earlier work on LevelDB by Sanjay Ghemawat (sanjay@google.com) and Jeff Dean (jeff@google.com)

This code is a library that forms the core building block for a fast key value server, especially suited for storing data on flash drives. It has a Log-Structured-Merge-Database (LSM) design with flexible tradeoffs between Write-Amplification-Factor (WAF), Read-Amplification-Factor (RAF) and Space-Amplification-Factor (SAF). It has multi-threaded compactions, making it specially suitable for storing multiple terabytes of data in a single database.

Start with example usage here: https://github.com/facebook/rocksdb/tree/master/examples

See the github wiki for more explanation.

The public interface is in include/. Callers should not include or rely on the details of any other header files in this package. Those internal APIs may be changed without warning.

Design discussions are conducted in https://www.facebook.com/groups/rocksdb.dev/