Merge branch 'master' into columnfamilies

main
Igor Canadi 11 years ago
commit 20b231d712
  1. 39
      HISTORY.md
  2. 31
      INSTALL.md
  3. 37
      build_tools/make_new_version.sh
  4. 1
      db/version_set.h
  5. 12
      utilities/backupable/backupable_db_test.cc

@ -0,0 +1,39 @@
# Rocksdb Change Log
## 2.7.0 (01/28/2014)
### Public API changes
* Renamed `StackableDB::GetRawDB()` to `StackableDB::GetBaseDB()`.
* Renamed `WriteBatch::Data()` `const std::string& Data() const`.
* Renamed class `TableStats` to `TableProperties`.
* Deleted class `PrefixHashRepFactory`. Please use `NewHashSkipListRepFactory()` instead.
* Supported multi-threaded `EnableFileDeletions()` and `DisableFileDeletions()`.
* Added `DB::GetOptions()`.
* Added `DB::GetDbIdentity()`.
### New Features
* Added [BackupableDB](https://github.com/facebook/rocksdb/wiki/How-to-backup-RocksDB%3F)
* Implemented [TailingIterator](https://github.com/facebook/rocksdb/wiki/Tailing-Iterator), a special type of iterator that
doesn't create a snapshot (can be used to read newly inserted data)
and is optimized for doing sequential reads.
* Added property block for table, which allows (1) a table to store
its metadata and (2) end user to collect and store properties they
are interested in.
* Enabled caching index and filter block in block cache (turned off by default).
* Supported error report when doing manual compaction.
* Supported additional Linux platform flavors and Mac OS.
* Put with `SliceParts` - Variant of `Put()` that gathers output like `writev(2)`
* Bug fixes and code refactor for compatibility with upcoming Column
Family feature.
### Performance Improvements
* Huge benchmark performance improvements by multiple efforts. For example, increase in readonly QPS from about 530k in 2.6 release to 1.1 million in 2.7 [1]
* Speeding up a way RocksDB deleted obsolete files - no longer listing the whole directory under a lock -- decrease in p99
* Use raw pointer instead of shared pointer for statistics: [5b825d](https://github.com/facebook/rocksdb/commit/5b825d6964e26ec3b4bb6faa708ebb1787f1d7bd) -- huge increase in performance -- shared pointers are slow
* Optimized locking for `Get()` -- [1fdb3f](https://github.com/facebook/rocksdb/commit/1fdb3f7dc60e96394e3e5b69a46ede5d67fb976c) -- 1.5x QPS increase for some workloads
* Cache speedup - [e8d40c3](https://github.com/facebook/rocksdb/commit/e8d40c31b3cca0c3e1ae9abe9b9003b1288026a9)
* Implemented autovector, which allocates first N elements on stack. Most of vectors in RocksDB are small. Also, we never want to allocate heap objects while holding a mutex. -- [c01676e4](https://github.com/facebook/rocksdb/commit/c01676e46d3be08c3c140361ef1f5884f47d3b3c)
* Lots of efforts to move malloc, memcpy and IO outside of locks

@ -17,15 +17,42 @@ libraries. You are on your own.
## Supported platforms
* **Linux**
* **Linux - Ubuntu**
* Upgrade your gcc to version at least 4.7 to get C++11 support.
* Install gflags. First, try: `sudo apt-get install libgflags-dev`.
* Install gflags. First, try: `sudo apt-get install libgflags-dev`
If this doesn't work and you're using Ubuntu, here's a nice tutorial:
(http://askubuntu.com/questions/312173/installing-gflags-12-04)
* Install snappy. This is usually as easy as:
`sudo apt-get install libsnappy-dev`.
* Install zlib. Try: `sudo apt-get install zlib1g-dev`.
* Install bzip2: `sudo apt-get install libbz2-dev`.
* **Linux - CentOS**
* Upgrade your gcc to version at least 4.7 to get C++11 support:
`yum install gcc47-c++`
* Install gflags:
wget https://gflags.googlecode.com/files/gflags-2.0-no-svn-files.tar.gz
tar -xzvf gflags-2.0-no-svn-files.tar.gz
cd gflags-2.0
./configure && make && sudo make install
* Install snappy:
wget https://snappy.googlecode.com/files/snappy-1.1.1.tar.gz
tar -xzvf snappy-1.1.1.tar.gz
cd snappy-1.1.1
./configure && make && sudo make install
* Install zlib:
sudo yum install zlib
sudo yum install zlib-devel
* Install bzip2:
sudo yum install bzip2
sudo yum install bzip2-devel
* **OS X**:
* Install latest C++ compiler that supports C++ 11:
* Update XCode: run `xcode-select --install` (or install it from XCode App's settting).

@ -5,6 +5,11 @@
# of patent rights can be found in the PATENTS file in the same directory.
set -e
if [ -z "$GIT" ]
then
GIT="git"
fi
# Print out the colored progress info so that it can be brainlessly
# distinguished by users.
function title() {
@ -27,35 +32,13 @@ if [ $GIT_BRANCH != "master" ]; then
echo "Error: Current branch is '$GIT_BRANCH', Please switch to master branch."
fi
# --Step 1: cutting new tag
title "Adding new tag for this release ..."
git tag -a "$ROCKSDB_VERSION.fb" -m "Rocksdb $ROCKSDB_VERSION"
$TAG="$ROCKSDB_VERSION.fb"
$GIT tag -a "$TAG" -m "Rocksdb $ROCKSDB_VERSION"
# Setting up the proxy for remote repo access
export http_proxy=http://172.31.255.99:8080
export https_proxy="$http_proxy";
title "Pushing new tag to remote repo ..."
proxycmd.sh git push origin --tags
# --Step 2: Update README.fb
title "Updating the latest version info in README.fb ..."
sed -i "s/Latest release is [0-9]\+.[0-9]\+.fb/Latest release is $ROCKSDB_VERSION.fb/" README.fb
git commit README.fb -m "update the latest version in README.fb to $ROCKSDB_VERSION"
proxycmd.sh git push
# --Step 3: Prepare this repo for 3rd release
title "Cleaning up repo ..."
make clean
git clean -fxd
title "Generating the build info ..."
# Comment out the call of `build_detection_version` so that the SHA number and build date of this
# release will remain constant. Otherwise everytime we run "make" util/build_version.cc will be
# overridden.
sed -i 's/^\$PWD\/build_tools\/build_detect_version$//' build_tools/build_detect_platform
# Generate util/build_version.cc
build_tools/build_detect_version
$GIT push origin --tags
title "Done!"
title "Tag $TAG is pushed to github; if you want to delete it, please run"
title "git tags -d $TAG && git push origin :refs/tags/$TAG"

@ -23,6 +23,7 @@
#include <set>
#include <vector>
#include <deque>
#include <atomic>
#include "db/dbformat.h"
#include "db/version_edit.h"
#include "port/port.h"

@ -252,17 +252,15 @@ class FileManager : public EnvWrapper {
return s;
}
std::vector<int64_t> positions;
auto pos = metadata.find(" crc32 ");
auto pos = metadata.find("private");
if (pos == std::string::npos) {
return Status::Corruption("private file is expected");
}
pos = metadata.find(" crc32 ", pos + 6);
if (pos == std::string::npos) {
return Status::Corruption("checksum not found");
}
do {
positions.push_back(pos);
pos = metadata.find(" crc32 ", pos + 6);
} while (pos != std::string::npos);
pos = positions[rnd_.Next() % positions.size()];
if (metadata.size() < pos + 7) {
return Status::Corruption("bad CRC32 checksum value");
}

Loading…
Cancel
Save