rocksdb

fork of https://github.com/oxigraph/rocksdb and https://github.com/facebook/rocksdb for nextgraph and oxigraph

Aaron Gao cb2476a0ca fix rate limiter to avoid starvation Summary: The current implementation of rate limiter has the possibility to introduce resource starvation when change its limit. This diff aims to fix this problem by consuming request bytes partially. Test Plan: ``` ./rate_limiter_test [==========] Running 4 tests from 1 test case. [----------] Global test environment set-up. [----------] 4 tests from RateLimiterTest [ RUN ] RateLimiterTest.OverflowRate [ OK ] RateLimiterTest.OverflowRate (0 ms) [ RUN ] RateLimiterTest.StartStop [ OK ] RateLimiterTest.StartStop (0 ms) [ RUN ] RateLimiterTest.Rate request size [1 - 1023], limit 10 KB/sec, actual rate: 10.355712 KB/sec, elapsed 2.00 seconds request size [1 - 1023], limit 20 KB/sec, actual rate: 19.136564 KB/sec, elapsed 2.00 seconds request size [1 - 2047], limit 20 KB/sec, actual rate: 20.783976 KB/sec, elapsed 2.10 seconds request size [1 - 2047], limit 40 KB/sec, actual rate: 39.308144 KB/sec, elapsed 2.10 seconds request size [1 - 4095], limit 40 KB/sec, actual rate: 40.318349 KB/sec, elapsed 2.20 seconds request size [1 - 4095], limit 80 KB/sec, actual rate: 79.667396 KB/sec, elapsed 2.20 seconds request size [1 - 8191], limit 80 KB/sec, actual rate: 81.807158 KB/sec, elapsed 2.30 seconds request size [1 - 8191], limit 160 KB/sec, actual rate: 160.659761 KB/sec, elapsed 2.20 seconds request size [1 - 16383], limit 160 KB/sec, actual rate: 160.700990 KB/sec, elapsed 3.00 seconds request size [1 - 16383], limit 320 KB/sec, actual rate: 317.639481 KB/sec, elapsed 2.50 seconds [ OK ] RateLimiterTest.Rate (22618 ms) [ RUN ] RateLimiterTest.LimitChangeTest [COMPLETE] request size 10 KB, new limit 20KB/sec, refill period 1000 ms [COMPLETE] request size 10 KB, new limit 5KB/sec, refill period 1000 ms [COMPLETE] request size 20 KB, new limit 40KB/sec, refill period 1000 ms [COMPLETE] request size 20 KB, new limit 10KB/sec, refill period 1000 ms [COMPLETE] request size 40 KB, new limit 80KB/sec, refill period 1000 ms [COMPLETE] request size 40 KB, new limit 20KB/sec, refill period 1000 ms [COMPLETE] request size 80 KB, new limit 160KB/sec, refill period 1000 ms [COMPLETE] request size 80 KB, new limit 40KB/sec, refill period 1000 ms [COMPLETE] request size 160 KB, new limit 320KB/sec, refill period 1000 ms [COMPLETE] request size 160 KB, new limit 80KB/sec, refill period 1000 ms [ OK ] RateLimiterTest.LimitChangeTest (5002 ms) [----------] 4 tests from RateLimiterTest (27620 ms total) [----------] Global test environment tear-down [==========] 4 tests from 1 test case ran. (27621 ms total) [ PASSED ] 4 tests. ``` Reviewers: sdong, IslamAbdelRahman, yiwu, andrewkr Reviewed By: andrewkr Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D60207		9 years ago
arcanist_util	Framework for enabling continuous RocksDB build and tests	9 years ago
build_tools	Workarounds for continuous build implementation	9 years ago
coverage	Fix coverage script	10 years ago
db	Fix UBSan build break caused by variable not initialized	9 years ago
doc	Lint everything	9 years ago
examples	Fix examples/Makefile jemalloc error	9 years ago
hdfs	using java7 in runtime for hdfs env (#1072 )	9 years ago
include/rocksdb	make transaction WriteOptions modifiable	9 years ago
java	Deprectate filter_deletes	9 years ago
memtable	Updated all copyright headers to the new format.	9 years ago
port	stack_trace,cc: The current Stacktrace code does not compile for FreeBSD (#1153 )	9 years ago
table	Update SstFileWriter to use bottommost_compression if avaliable	9 years ago
third-party	fix simple typos (#1183 )	9 years ago
tools	Fixed Minor Bug on Windows Build and db_bench_tool.cc (#1189 )	9 years ago
util	fix rate limiter to avoid starvation	9 years ago
utilities	writable file close before reset	9 years ago
.arcconfig	Integrate Jenkins with Phabricator	10 years ago
.clang-format	A script that automatically reformat affected lines	11 years ago
.gitignore	Ignore db_test2	9 years ago
.travis.yml	Re-enable linux on travis	9 years ago
AUTHORS	Add AUTHORS file. Fix #203	10 years ago
CMakeLists.txt	Fixed Minor Bug on Windows Build and db_bench_tool.cc (#1189 )	9 years ago
CONTRIBUTING.md	facebook accounts are not required for CLA signers	11 years ago
DEFAULT_OPTIONS_HISTORY.md	Release RocksDB 4.8.0	9 years ago
DUMP_FORMAT.md	First version of rocksdb_dump and rocksdb_undump.	10 years ago
HISTORY.md	Detect column family from properties [CF + RepairDB part 2/3]	9 years ago
INSTALL.md	Simple changes to support builds for ppc64[le] consistent with X86	9 years ago
LANGUAGE-BINDINGS.md	Merge pull request #1056 from facebook/igorcanadi-patch-1	9 years ago
LICENSE	Updated all copyright headers to the new format.	9 years ago
Makefile	Makefile warning for invalid paths in make_config.mk	9 years ago
PATENTS	Update Patent Grant.	10 years ago
README.md	Replaced "built on on earlier work" by "built on earlier work" in README.md	10 years ago
ROCKSDB_LITE.md	Optimistic Transactions	10 years ago
USERS.md	Update USERS.md	9 years ago
Vagrantfile	RocksDB on FreeBSD support	10 years ago
WINDOWS_PORT.md	Commit both PR and internal code review changes	10 years ago
appveyor.yml	Disable long running GroupCommitTest (#1125 )	9 years ago
src.mk	Update DB::AddFile() to ingest the file to the lowest possible level	9 years ago
thirdparty.inc	Introduce XPRESS compresssion on Windows. (#1081 )	9 years ago

README.md

RocksDB: A Persistent Key-Value Store for Flash and RAM Storage

RocksDB is developed and maintained by Facebook Database Engineering Team. It is built on earlier work on LevelDB by Sanjay Ghemawat (sanjay@google.com) and Jeff Dean (jeff@google.com)

This code is a library that forms the core building block for a fast key value server, especially suited for storing data on flash drives. It has a Log-Structured-Merge-Database (LSM) design with flexible tradeoffs between Write-Amplification-Factor (WAF), Read-Amplification-Factor (RAF) and Space-Amplification-Factor (SAF). It has multi-threaded compactions, making it specially suitable for storing multiple terabytes of data in a single database.

Start with example usage here: https://github.com/facebook/rocksdb/tree/master/examples

See the github wiki for more explanation.

The public interface is in include/. Callers should not include or rely on the details of any other header files in this package. Those internal APIs may be changed without warning.

Design discussions are conducted in https://www.facebook.com/groups/rocksdb.dev/