rocksdb

fork of https://github.com/oxigraph/rocksdb and https://github.com/facebook/rocksdb for nextgraph and oxigraph

Peter Dillinger fd911f9655 Upgrade xxhash.h to latest dev (#11098 ) Summary: Upgrading xxhash.h to latest dev version as of 1/17/2023, which is d7197ddea81364a539051f116ca77926100fc77f This should improve performance on some ARM machines. I allowed some of our RocksDB-specific changes to be made obsolete where it seemed appropriate, for example * xxhash.h has its own fallthrough marker (which I hope works for us) * As in https://github.com/Cyan4973/xxHash/pull/549 Merging and resolving conflicts one way or the other was all that went into this diff. Except I had to mix the two sides around `defined(__loongarch64)` How I did the upgrade (for future reference), so that I could use usual merge conflict resolution: ``` # New branch to help with merging git checkout -b xxh_merge_base # Check out RocksDB revision before last xxhash.h upgrade git reset --hard 22161b7547652af82a5dc67458de9ca8946ac83d^ # Create a commit with the raw base version from xxHash repo (from xxHash repo) git show 2c611a76f914828bed675f0f342d6c4199ffee1e:xxhash.h > ../rocksdb/util/xxhash.h # In RocksDB repo git commit -a # Merge in the last xxhash.h upgrade git merge `22161b7547` # Resolve conflict using committed version git show 22161b7547652af82a5dc67458de9ca8946ac83d:util/xxhash.h > util/xxhash.h git commit -a # Catch up to upstream git merge upstream/main # Create a different branch for applying raw upgrade git checkout -b xxh_upgrade_2023 # Find the RocksDB commit we made for the raw base version from xxHash git log main..HEAD # Rewind to it git reset --hard 2428b727a9a19c6078bb75895805d7488cbdd08c # Copy in latest raw version (from xxHash repo) cat xxhash.h > ../rocksdb/util/xxhash.h # Merge in RocksDB changes, use typical tools for conflict resolution git merge xxh_merge_base ``` Branch https://github.com/facebook/rocksdb/tree/xxhash_merge_base can be used as a base for future xxhash merges. Fixes https://github.com/facebook/rocksdb/issues/11073 Pull Request resolved: https://github.com/facebook/rocksdb/pull/11098 Test Plan: existing tests (e.g. Bloom filter schema stability tests) Also seems to include a small performance boost on my Intel dev machine, using `./db_bench --benchmarks=xxh3[-X50] 2>&1 \| egrep -o 'operations;.*' \| sort` Fastest out of 50 runs, before: 15477.3 MB/s Fastest out of 50 runs, after: 15850.7 MB/s, and 11 more runs faster than the "before" number Slowest out of 50 runs, before: 12267.5 MB/s Slowest out of 50 runs, after: 13897.1 MB/s More repetitions show the distinction is repeatable Reviewed By: hx235 Differential Revision: D42560010 Pulled By: pdillinger fbshipit-source-id: c43ee52f1c5fe0ba3d6d6e4eebb22ded5f5492ea		2 years ago
.circleci	Upgrade CircleCI Windows Build (#10090 )	2 years ago
.github/workflows	ci: add GitHub token permissions for workflow (#10549 )	2 years ago
buckifier	Enable BLACK for internal_repo_rocksdb (#10710 )	2 years ago
build_tools	Print stack traces on frozen tests in CI (#10828 )	2 years ago
cache	Major Cache refactoring, CPU efficiency improvement (#10975 )	2 years ago
cmake	gcc-11 and cmake related cleanup (#9286 )	3 years ago
coverage	Enable BLACK for internal_repo_rocksdb (#10710 )	2 years ago
db	Fix asan failure caused by range tombstone start key use-after-free (#11106 )	2 years ago
db_stress_tool	fix shared state used after free (#11059 )	2 years ago
docs	Bump nokogiri from 1.13.9 to 1.13.10 in /docs (#11024 )	2 years ago
env	Added placeholders for MADV defines (#10881 )	2 years ago
examples	Add rocksdb_backup_restore_example to examples/.gitignore (#10825 )	2 years ago
file	Add a unit test for async prefetch fix in #11049 (#11084 )	2 years ago
fuzz	Add some missing headers (#10519 )	2 years ago
include/rocksdb	Major Cache refactoring, CPU efficiency improvement (#10975 )	2 years ago
java	Add missing range conflict check between file ingestion and RefitLevel() (#10988 )	2 years ago
logging	Observe and warn about misconfigured HyperClockCache (#10965 )	2 years ago
memory	Major Cache refactoring, CPU efficiency improvement (#10975 )	2 years ago
memtable	Run clang format against files under example/, memory/ and memtable/ folders (#10893 )	2 years ago
microbench	Avoid allocations/copies for large `GetMergeOperands()` results (#10458 )	2 years ago
monitoring	Add an unittest for Periodic compaction conflict with ongoing compaction (#10908 )	2 years ago
options	Major Cache refactoring, CPU efficiency improvement (#10975 )	2 years ago
plugin	Add initial CMake support to plugin (#9214 )	3 years ago
port	support loongarch64 for rocksdb (#10036 )	2 years ago
table	Revert #10802 Consider range tombstone in compaction output file cutting (#11089 )	2 years ago
test_util	~SleepingBackgroundTask() to wake up the sleeping task (#11036 )	2 years ago
third-party	Meta-internal folly integration with F14FastMap (#9546 )	3 years ago
tools	Fix crash in block_cache_trace_analyzer if reference key is null in case of MultiGet (#11042 )	2 years ago
trace_replay	Fix crash in block_cache_trace_analyzer if reference key is null in case of MultiGet (#11042 )	2 years ago
util	Upgrade xxhash.h to latest dev (#11098 )	2 years ago
utilities	support loongarch64 for rocksdb (#10036 )	2 years ago
.clang-format	A script that automatically reformat affected lines	11 years ago
.gitignore	Git ignore .clangd/ (#10817 )	2 years ago
.lgtm.yml	Create lgtm.yml for LGTM.com C/C++ analysis (#4058 )	7 years ago
.watchmanconfig	Added .watchmanconfig file to rocksdb repo (#5593 )	6 years ago
AUTHORS	Update RocksDB Authors File	7 years ago
CMakeLists.txt	Revert #10802 Consider range tombstone in compaction output file cutting (#11089 )	2 years ago
CODE_OF_CONDUCT.md	Adopt Contributor Covenant	5 years ago
CONTRIBUTING.md	Add Code of Conduct	7 years ago
COPYING	Add GPLv2 as an alternative license.	8 years ago
DEFAULT_OPTIONS_HISTORY.md	Add Options::DisableExtraChecks, clarify force_consistency_checks (#9363 )	3 years ago
DUMP_FORMAT.md	First version of rocksdb_dump and rocksdb_undump.	10 years ago
HISTORY.md	Upgrade xxhash.h to latest dev (#11098 )	2 years ago
INSTALL.md	Update supported VS versions in INSTALL.md (#9823 )	3 years ago
LANGUAGE-BINDINGS.md	Add grocksdb in Go language bindings (#10498 )	2 years ago
LICENSE.Apache	Change RocksDB License	8 years ago
LICENSE.leveldb	Add back the LevelDB license file	8 years ago
Makefile	support loongarch64 for rocksdb (#10036 )	2 years ago
PLUGINS.md	Add pmem-rocksdb-plugin link in PLUGINs.md (#9934 )	3 years ago
README.md	Remove Travis CI (#10407 )	2 years ago
ROCKSDB_LITE.md	Fix remaining uses of "backupable" (#9792 )	3 years ago
TARGETS	Revert #10802 Consider range tombstone in compaction output file cutting (#11089 )	2 years ago
USERS.md	Add Apache Spark as a user (#10993 )	2 years ago
Vagrantfile	Adding CentOS 7 Vagrantfile & build script	7 years ago
WINDOWS_PORT.md	Update branch name in WINDOWS_PORT.md (#8745 )	3 years ago
common.mk	Clean up variables for temporary directory (#9961 )	3 years ago
crash_test.mk	Allow a custom DB cleanup command to be passed to db_crashtest.py (#10883 )	2 years ago
issue_template.md	Add Google Group to Issue Template	5 years ago
rocksdb.pc.in	build: fix pkg-config file generation (#9953 )	3 years ago
src.mk	Revert #10802 Consider range tombstone in compaction output file cutting (#11089 )	2 years ago
thirdparty.inc	Fix build jemalloc api (#5470 )	6 years ago

README.md

RocksDB: A Persistent Key-Value Store for Flash and RAM Storage

RocksDB is developed and maintained by Facebook Database Engineering Team. It is built on earlier work on LevelDB by Sanjay Ghemawat (sanjay@google.com) and Jeff Dean (jeff@google.com)

This code is a library that forms the core building block for a fast key-value server, especially suited for storing data on flash drives. It has a Log-Structured-Merge-Database (LSM) design with flexible tradeoffs between Write-Amplification-Factor (WAF), Read-Amplification-Factor (RAF) and Space-Amplification-Factor (SAF). It has multi-threaded compactions, making it especially suitable for storing multiple terabytes of data in a single database.

Start with example usage here: https://github.com/facebook/rocksdb/tree/main/examples

See the github wiki for more explanation.

The public interface is in include/. Callers should not include or rely on the details of any other header files in this package. Those internal APIs may be changed without warning.

Questions and discussions are welcome on the RocksDB Developers Public Facebook group and email list on Google Groups.

License

RocksDB is dual-licensed under both the GPLv2 (found in the COPYING file in the root directory) and Apache 2.0 License (found in the LICENSE.Apache file in the root directory). You may select, at your option, one of the above-listed licenses.