rocksdb

fork of https://github.com/oxigraph/rocksdb and https://github.com/facebook/rocksdb for nextgraph and oxigraph

Andrew Kryczka 69c471bd9b Handle concurrent manifest update and backup creation Summary: Fixed two related race conditions in backup creation. (1) CreateNewBackup() uses DB::DisableFileDeletions() to prevent table files from being deleted while it is copying; however, the MANIFEST file could still rotate during this time. The fix is to stop deleting the old manifest in the rotation logic. It will be deleted safely later when PurgeObsoleteFiles() runs (can only happen when file deletions are enabled). (2) CreateNewBackup() did not account for the CURRENT file being mutable. This is significant because the files returned by GetLiveFiles() contain a particular manifest filename, but the manifest to which CURRENT refers can change at any time. This causes problems when CURRENT changes between the call to GetLiveFiles() and when it's copied to the backup directory. To workaround this, I manually forge a CURRENT file referring to the manifest filename returned in GetLiveFiles(). (2) also applies to the checkpointing code, so let me know if this approach is good and I'll make the same change there. Test Plan: new test for roll manifest during backup creation. running the test before this change: $ ./backupable_db_test --gtest_filter=BackupableDBTest.ChangeManifestDuringBackupCreation ... IO error: /tmp/rocksdbtest-9383/backupable_db/MANIFEST-000001: No such file or directory running the test after this change: $ ./backupable_db_test --gtest_filter=BackupableDBTest.ChangeManifestDuringBackupCreation ... [ RUN ] BackupableDBTest.ChangeManifestDuringBackupCreation [ OK ] BackupableDBTest.ChangeManifestDuringBackupCreation (2836 ms) Reviewers: IslamAbdelRahman, anthony, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D54711		8 years ago
arcanist_util	Updated all copyright headers to the new format.	9 years ago
build_tools	Add flag to forcibly disable fallocate	8 years ago
coverage	Fix coverage script	10 years ago
db	Handle concurrent manifest update and backup creation	8 years ago
doc	Lint everything	9 years ago
examples	Updated all copyright headers to the new format.	9 years ago
hdfs	Updated all copyright headers to the new format.	9 years ago
include/rocksdb	Remove stale TODO	8 years ago
java	Updated all copyright headers to the new format.	9 years ago
memtable	Updated all copyright headers to the new format.	9 years ago
port	Implement ConsistentChildrenAttribute	8 years ago
table	Revert "Revert "Fixed the bug when both whole_key_filtering and prefix_extractor are set.""	8 years ago
third-party	Updated all copyright headers to the new format.	9 years ago
tools	fix build without gflags	8 years ago
util	IOStatsContext::ToString() add option to exclude zero counters	8 years ago
utilities	Handle concurrent manifest update and backup creation	8 years ago
.arcconfig	Integrate Jenkins with Phabricator	9 years ago
.clang-format	A script that automatically reformat affected lines	11 years ago
.gitignore	New amalgamation target	9 years ago
.travis.yml	Travis CI to disable ROCKSDB_LITE tests	9 years ago
AUTHORS	Add AUTHORS file. Fix #203	10 years ago
CMakeLists.txt	IOStatsContext::ToString() add option to exclude zero counters	8 years ago
CONTRIBUTING.md	facebook accounts are not required for CLA signers	10 years ago
DUMP_FORMAT.md	First version of rocksdb_dump and rocksdb_undump.	9 years ago
HISTORY.md	Add a new compaction priority that picks file whose overlapping ratio is smallest	8 years ago
INSTALL.md	Simple changes to support builds for ppc64[le] consistent with X86	9 years ago
LICENSE	Updated all copyright headers to the new format.	9 years ago
Makefile	build: Improve -momit-leaf-frame-pointer usage	8 years ago
PATENTS	Update Patent Grant.	9 years ago
README.md	Replaced "built on on earlier work" by "built on earlier work" in README.md	10 years ago
ROCKSDB_LITE.md	Optimistic Transactions	9 years ago
USERS.md	Add Wingify to USERS.md	9 years ago
Vagrantfile	RocksDB on FreeBSD support	9 years ago
WINDOWS_PORT.md	Commit both PR and internal code review changes	9 years ago
appveyor.yml	Exclude DBTest.FileCreationRandomFailure as a long running test	9 years ago
src.mk	IOStatsContext::ToString() add option to exclude zero counters	8 years ago
thirdparty.inc	Enable override to 3rd party linkage	9 years ago

README.md

RocksDB: A Persistent Key-Value Store for Flash and RAM Storage

RocksDB is developed and maintained by Facebook Database Engineering Team. It is built on earlier work on LevelDB by Sanjay Ghemawat (sanjay@google.com) and Jeff Dean (jeff@google.com)

This code is a library that forms the core building block for a fast key value server, especially suited for storing data on flash drives. It has a Log-Structured-Merge-Database (LSM) design with flexible tradeoffs between Write-Amplification-Factor (WAF), Read-Amplification-Factor (RAF) and Space-Amplification-Factor (SAF). It has multi-threaded compactions, making it specially suitable for storing multiple terabytes of data in a single database.

Start with example usage here: https://github.com/facebook/rocksdb/tree/master/examples

See the github wiki for more explanation.

The public interface is in include/. Callers should not include or rely on the details of any other header files in this package. Those internal APIs may be changed without warning.

Design discussions are conducted in https://www.facebook.com/groups/rocksdb.dev/