|
|
|
#!/bin/sh
|
|
|
|
#
|
|
|
|
# Detects OS we're compiling on and outputs a file specified by the first
|
|
|
|
# argument, which in turn gets read while processing Makefile.
|
|
|
|
#
|
|
|
|
# The output will set the following variables:
|
|
|
|
# CC C Compiler path
|
|
|
|
# CXX C++ Compiler path
|
|
|
|
# PLATFORM_LDFLAGS Linker flags
|
|
|
|
# PLATFORM_SHARED_EXT Extension for shared libraries
|
|
|
|
# PLATFORM_SHARED_LDFLAGS Flags for building shared library
|
|
|
|
# PLATFORM_SHARED_CFLAGS Flags for compiling objects for shared library
|
|
|
|
# PLATFORM_CCFLAGS C compiler flags
|
|
|
|
# PLATFORM_CXXFLAGS C++ compiler flags. Will contain:
|
|
|
|
# PLATFORM_SHARED_VERSIONED Set to 'true' if platform supports versioned
|
|
|
|
# shared libraries, empty otherwise.
|
|
|
|
#
|
|
|
|
# The PLATFORM_CCFLAGS and PLATFORM_CXXFLAGS might include the following:
|
|
|
|
#
|
|
|
|
# -DLEVELDB_PLATFORM_POSIX if cstdatomic is present
|
|
|
|
# -DLEVELDB_PLATFORM_NOATOMIC if it is not
|
|
|
|
# -DSNAPPY if the Snappy library is present
|
|
|
|
#
|
|
|
|
# Using gflags in rocksdb:
|
|
|
|
# Our project depends on gflags, which requires users to take some extra steps
|
|
|
|
# before they can compile the whole repository:
|
|
|
|
# 1. Install gflags. You may download it from here:
|
|
|
|
# https://code.google.com/p/gflags/
|
|
|
|
# 2. Once install, add the include path/lib path for gflags to CPATH and
|
|
|
|
# LIBRARY_PATH respectively. If installed with default mode, the
|
|
|
|
# lib and include path will be /usr/local/lib and /usr/local/include
|
|
|
|
# Mac user can do this by running build_tools/mac-install-gflags.sh
|
|
|
|
|
|
|
|
OUTPUT=$1
|
|
|
|
if test -z "$OUTPUT"; then
|
|
|
|
echo "usage: $0 <output-filename>" >&2
|
|
|
|
exit 1
|
|
|
|
fi
|
|
|
|
|
|
|
|
# we depend on C++11
|
|
|
|
PLATFORM_CXXFLAGS="-std=gnu++11"
|
|
|
|
# we currently depend on POSIX platform
|
|
|
|
COMMON_FLAGS="-DROCKSDB_PLATFORM_POSIX"
|
|
|
|
|
|
|
|
# Default to fbcode gcc on internal fb machines
|
|
|
|
if [ -d /mnt/gvfs/third-party -a -z "$CXX" ]; then
|
|
|
|
FBCODE_BUILD="true"
|
|
|
|
if [ -z "$USE_CLANG" ]; then
|
|
|
|
CENTOS_VERSION=`rpm -q --qf "%{VERSION}" \
|
|
|
|
$(rpm -q --whatprovides redhat-release)`
|
|
|
|
if [ "$CENTOS_VERSION" = "6" ]; then
|
|
|
|
source $PWD/build_tools/fbcode.gcc481.sh
|
|
|
|
else
|
|
|
|
source $PWD/build_tools/fbcode.gcc471.sh
|
|
|
|
fi
|
|
|
|
else
|
|
|
|
source $PWD/build_tools/fbcode.clang31.sh
|
|
|
|
fi
|
|
|
|
fi
|
|
|
|
|
|
|
|
# Delete existing output, if it exists
|
|
|
|
rm -f $OUTPUT
|
|
|
|
touch $OUTPUT
|
|
|
|
|
|
|
|
if test -z "$CC"; then
|
|
|
|
CC=cc
|
|
|
|
fi
|
|
|
|
|
|
|
|
if test -z "$CXX"; then
|
|
|
|
CXX=g++
|
|
|
|
fi
|
|
|
|
|
|
|
|
# Detect OS
|
|
|
|
if test -z "$TARGET_OS"; then
|
|
|
|
TARGET_OS=`uname -s`
|
|
|
|
fi
|
|
|
|
|
fPIC in x64 environment
Summary:
Check https://github.com/facebook/rocksdb/pull/15 for context.
Apparently [1], we need -fPIC in x64 environments (this is added only in non-fbcode).
In fbcode, I removed -fPIC per @dhruba's suggestion, since it introduces perf regression. I'm not sure what would are the implications of doing that, but looks like it works, and when releasing to the third-party, we're disabling -fPIC either way [2].
Would love a suggestion from someone who knows more about this
[1] http://eli.thegreenplace.net/2011/11/11/position-independent-code-pic-in-shared-libraries-on-x64/
[2] https://our.intern.facebook.com/intern/wiki/index.php/Database/RocksDB/Third_Party
Test Plan: make check works
Reviewers: dhruba, emayanke, kailiu
Reviewed By: dhruba
CC: leveldb, dhruba, reconnect.grayhat
Differential Revision: https://reviews.facebook.net/D14337
11 years ago
|
|
|
COMMON_FLAGS="$COMMON_FLAGS ${CFLAGS}"
|
|
|
|
CROSS_COMPILE=
|
|
|
|
PLATFORM_CCFLAGS=
|
|
|
|
PLATFORM_CXXFLAGS="$PLATFORM_CXXFLAGS ${CXXFLAGS}"
|
|
|
|
PLATFORM_LDFLAGS="$PLATFORM_LDFLAGS"
|
|
|
|
PLATFORM_SHARED_EXT="so"
|
|
|
|
PLATFORM_SHARED_LDFLAGS="${EXEC_LDFLAGS_SHARED} -shared -Wl,-soname -Wl,"
|
|
|
|
PLATFORM_SHARED_CFLAGS="-fPIC"
|
|
|
|
PLATFORM_SHARED_VERSIONED=true
|
|
|
|
|
|
|
|
# generic port files (working on all platform by #ifdef) go directly in /port
|
|
|
|
GENERIC_PORT_FILES=`find $ROCKSDB_ROOT/port -name '*.cc' | tr "\n" " "`
|
|
|
|
|
|
|
|
# On GCC, we pick libc's memcmp over GCC's memcmp via -fno-builtin-memcmp
|
|
|
|
case "$TARGET_OS" in
|
|
|
|
Darwin)
|
|
|
|
PLATFORM=OS_MACOSX
|
|
|
|
COMMON_FLAGS="$COMMON_FLAGS -DOS_MACOSX"
|
|
|
|
PLATFORM_SHARED_EXT=dylib
|
|
|
|
PLATFORM_SHARED_LDFLAGS="-dynamiclib -install_name "
|
|
|
|
# PORT_FILES=port/darwin/darwin_specific.cc
|
|
|
|
;;
|
|
|
|
Linux)
|
|
|
|
PLATFORM=OS_LINUX
|
|
|
|
COMMON_FLAGS="$COMMON_FLAGS -DOS_LINUX"
|
|
|
|
if [ -z "$USE_CLANG" ]; then
|
|
|
|
COMMON_FLAGS="$COMMON_FLAGS -fno-builtin-memcmp"
|
|
|
|
fi
|
[RocksDB] Added nano second stopwatch and new perf counters to track block read cost
Summary: The pupose of this diff is to expose per user-call level precise timing of block read, so that we can answer questions like: a Get() costs me 100ms, is that somehow related to loading blocks from file system, or sth else? We will answer that with EXACTLY how many blocks have been read, how much time was spent on transfering the bytes from os, how much time was spent on checksum verification and how much time was spent on block decompression, just for that one Get. A nano second stopwatch was introduced to track time with higher precision. The cost/precision of the stopwatch is also measured in unit-test. On my dev box, retrieving one time instance costs about 30ns, on average. The deviation of timing results is good enough to track 100ns-1us level events. And the overhead could be safely ignored for 100us level events (10000 instances/s), for example, a viewstate thrift call.
Test Plan: perf_context_test, also testing with viewstate shadow traffic.
Reviewers: dhruba
Reviewed By: dhruba
CC: leveldb, xjin
Differential Revision: https://reviews.facebook.net/D12351
12 years ago
|
|
|
PLATFORM_LDFLAGS="$PLATFORM_LDFLAGS -lpthread -lrt"
|
|
|
|
# PORT_FILES=port/linux/linux_specific.cc
|
|
|
|
;;
|
|
|
|
SunOS)
|
|
|
|
PLATFORM=OS_SOLARIS
|
|
|
|
COMMON_FLAGS="$COMMON_FLAGS -fno-builtin-memcmp -D_REENTRANT -DOS_SOLARIS"
|
|
|
|
PLATFORM_LDFLAGS="$PLATFORM_LDFLAGS -lpthread -lrt"
|
|
|
|
# PORT_FILES=port/sunos/sunos_specific.cc
|
|
|
|
;;
|
|
|
|
FreeBSD)
|
|
|
|
PLATFORM=OS_FREEBSD
|
|
|
|
COMMON_FLAGS="$COMMON_FLAGS -fno-builtin-memcmp -D_REENTRANT -DOS_FREEBSD"
|
|
|
|
PLATFORM_LDFLAGS="$PLATFORM_LDFLAGS -lpthread"
|
|
|
|
# PORT_FILES=port/freebsd/freebsd_specific.cc
|
|
|
|
;;
|
|
|
|
NetBSD)
|
|
|
|
PLATFORM=OS_NETBSD
|
|
|
|
COMMON_FLAGS="$COMMON_FLAGS -fno-builtin-memcmp -D_REENTRANT -DOS_NETBSD"
|
|
|
|
PLATFORM_LDFLAGS="$PLATFORM_LDFLAGS -lpthread -lgcc_s"
|
|
|
|
# PORT_FILES=port/netbsd/netbsd_specific.cc
|
|
|
|
;;
|
|
|
|
OpenBSD)
|
|
|
|
PLATFORM=OS_OPENBSD
|
|
|
|
COMMON_FLAGS="$COMMON_FLAGS -fno-builtin-memcmp -D_REENTRANT -DOS_OPENBSD"
|
|
|
|
PLATFORM_LDFLAGS="$PLATFORM_LDFLAGS -pthread"
|
|
|
|
# PORT_FILES=port/openbsd/openbsd_specific.cc
|
|
|
|
;;
|
|
|
|
DragonFly)
|
|
|
|
PLATFORM=OS_DRAGONFLYBSD
|
|
|
|
COMMON_FLAGS="$COMMON_FLAGS -fno-builtin-memcmp -D_REENTRANT -DOS_DRAGONFLYBSD"
|
|
|
|
PLATFORM_LDFLAGS="$PLATFORM_LDFLAGS -lpthread"
|
|
|
|
# PORT_FILES=port/dragonfly/dragonfly_specific.cc
|
|
|
|
;;
|
|
|
|
OS_ANDROID_CROSSCOMPILE)
|
|
|
|
PLATFORM=OS_ANDROID
|
|
|
|
COMMON_FLAGS="$COMMON_FLAGS -fno-builtin-memcmp -D_REENTRANT -DOS_ANDROID -DLEVELDB_PLATFORM_POSIX"
|
|
|
|
PLATFORM_LDFLAGS="$PLATFORM_LDFLAGS " # All pthread features are in the Android C library
|
|
|
|
# PORT_FILES=port/android/android.cc
|
|
|
|
CROSS_COMPILE=true
|
|
|
|
;;
|
|
|
|
*)
|
|
|
|
echo "Unknown platform!" >&2
|
|
|
|
exit 1
|
|
|
|
esac
|
|
|
|
|
|
|
|
$PWD/build_tools/build_detect_version
|
|
|
|
|
|
|
|
# We want to make a list of all cc files within util, db, table, and helpers
|
|
|
|
# except for the test and benchmark files. By default, find will output a list
|
|
|
|
# of all files matching either rule, so we need to append -print to make the
|
|
|
|
# prune take effect.
|
Timestamp and TTL Wrapper for rocksdb
Summary:
When opened with DBTimestamp::Open call, timestamps are prepended to and stripped from the value during subsequent Put and Get calls respectively. The Timestamp is used to discard values in Get and custom compaction filter which have exceeded their TTL which is specified during Open.
Have made a temporary change to Makefile to let us test with the temporary file TestTime.cc. Have also changed the private members of db_impl.h to protected to let them be inherited by the new class DBTimestamp
Test Plan: make db_timestamp; TestTime.cc(will not check it in) shows how to use the apis currently, but I will write unit-tests shortly
Reviewers: dhruba, vamsi, haobo, sheki, heyongqiang, vkrest
Reviewed By: vamsi
CC: zshao, xjin, vkrest, MarkCallaghan
Differential Revision: https://reviews.facebook.net/D10311
12 years ago
|
|
|
DIRS="util db table utilities"
|
|
|
|
|
|
|
|
set -f # temporarily disable globbing so that our patterns arent expanded
|
|
|
|
PRUNE_TEST="-name *test*.cc -prune"
|
|
|
|
PRUNE_BENCH="-name *_bench.cc -prune"
|
|
|
|
PORTABLE_FILES=`cd $ROCKSDB_ROOT; find $DIRS $PRUNE_TEST -o $PRUNE_BENCH -o -name '*.cc' -print | sort | tr "\n" " "`
|
|
|
|
PORTABLE_CPP=`cd $ROCKSDB_ROOT; find $DIRS $PRUNE_TEST -o $PRUNE_BENCH -o -name '*.cpp' -print | sort | tr "\n" " "`
|
|
|
|
set +f # re-enable globbing
|
|
|
|
|
|
|
|
# The sources consist of the portable files, plus the platform-specific port
|
|
|
|
# file.
|
|
|
|
echo "SOURCES=$PORTABLE_FILES $GENERIC_PORT_FILES $PORT_FILES" >> $OUTPUT
|
|
|
|
echo "SOURCESCPP=$PORTABLE_CPP" >> $OUTPUT
|
|
|
|
echo "MEMENV_SOURCES=helpers/memenv/memenv.cc" >> $OUTPUT
|
|
|
|
|
|
|
|
if [ "$CROSS_COMPILE" = "true" -o "$FBCODE_BUILD" = "true" ]; then
|
|
|
|
# Cross-compiling; do not try any compilation tests.
|
|
|
|
# Also don't need any compilation tests if compiling on fbcode
|
|
|
|
# TODO(kailiu) For some fb-supported environments, they have trouble finding
|
|
|
|
# symbols from gflags if -lgflags wasn't specified. As a temporary resort,
|
|
|
|
# we added it to PLATFORM_LDFLAGS although for some environments it is
|
|
|
|
# unnecessary.
|
|
|
|
PLATFORM_LDFLAGS="$PLATFORM_LDFLAGS -gflags"
|
|
|
|
else
|
fPIC in x64 environment
Summary:
Check https://github.com/facebook/rocksdb/pull/15 for context.
Apparently [1], we need -fPIC in x64 environments (this is added only in non-fbcode).
In fbcode, I removed -fPIC per @dhruba's suggestion, since it introduces perf regression. I'm not sure what would are the implications of doing that, but looks like it works, and when releasing to the third-party, we're disabling -fPIC either way [2].
Would love a suggestion from someone who knows more about this
[1] http://eli.thegreenplace.net/2011/11/11/position-independent-code-pic-in-shared-libraries-on-x64/
[2] https://our.intern.facebook.com/intern/wiki/index.php/Database/RocksDB/Third_Party
Test Plan: make check works
Reviewers: dhruba, emayanke, kailiu
Reviewed By: dhruba
CC: leveldb, dhruba, reconnect.grayhat
Differential Revision: https://reviews.facebook.net/D14337
11 years ago
|
|
|
# do fPIC on 64 bit in non-fbcode environment
|
|
|
|
case "$TARGET_OS" in
|
|
|
|
x86_64)
|
|
|
|
PLATFORM_CXXFLAGS="$PLATFORM_CXXFLAGS -fPIC"
|
|
|
|
esac
|
|
|
|
|
|
|
|
# If -std=c++0x works, use <atomic>. Otherwise use port_posix.h.
|
|
|
|
$CXX $CFLAGS -std=c++0x -x c++ - -o /dev/null 2>/dev/null <<EOF
|
|
|
|
#include <atomic>
|
|
|
|
int main() {}
|
|
|
|
EOF
|
|
|
|
if [ "$?" = 0 ]; then
|
|
|
|
COMMON_FLAGS="$COMMON_FLAGS -DROCKSDB_ATOMIC_PRESENT"
|
|
|
|
fi
|
|
|
|
|
|
|
|
# Test whether fallocate is available
|
|
|
|
$CXX $CFLAGS -x c++ - -o /dev/null 2>/dev/null <<EOF
|
|
|
|
#include <fcntl.h>
|
|
|
|
int main() {
|
|
|
|
int fd = open("/dev/null", 0);
|
|
|
|
fallocate(fd, 0, 0, 1024);
|
|
|
|
}
|
|
|
|
EOF
|
|
|
|
if [ "$?" = 0 ]; then
|
|
|
|
COMMON_FLAGS="$COMMON_FLAGS -DROCKSDB_FALLOCATE_PRESENT"
|
|
|
|
fi
|
|
|
|
|
|
|
|
# Test whether Snappy library is installed
|
|
|
|
# http://code.google.com/p/snappy/
|
|
|
|
$CXX $CFLAGS -x c++ - -o /dev/null 2>/dev/null <<EOF
|
|
|
|
#include <snappy.h>
|
|
|
|
int main() {}
|
|
|
|
EOF
|
|
|
|
if [ "$?" = 0 ]; then
|
|
|
|
COMMON_FLAGS="$COMMON_FLAGS -DSNAPPY"
|
|
|
|
PLATFORM_LDFLAGS="$PLATFORM_LDFLAGS -lsnappy"
|
|
|
|
fi
|
|
|
|
|
|
|
|
|
|
|
|
# Test whether gflags library is installed
|
|
|
|
# http://code.google.com/p/gflags/
|
|
|
|
$CXX $CFLAGS -x c++ - -o /dev/null 2>/dev/null <<EOF
|
|
|
|
#include <gflags/gflags.h>
|
|
|
|
int main() {}
|
|
|
|
EOF
|
|
|
|
if [ "$?" = 0 ]; then
|
|
|
|
COMMON_FLAGS="$COMMON_FLAGS -DGFLAGS"
|
|
|
|
PLATFORM_LDFLAGS="$PLATFORM_LDFLAGS -lgflags"
|
|
|
|
fi
|
|
|
|
|
|
|
|
# Test whether zlib library is installed
|
|
|
|
$CXX $CFLAGS $COMMON_FLAGS -x c++ - -o /dev/null 2>/dev/null <<EOF
|
|
|
|
#include <zlib.h>
|
|
|
|
int main() {}
|
|
|
|
EOF
|
|
|
|
if [ "$?" = 0 ]; then
|
|
|
|
COMMON_FLAGS="$COMMON_FLAGS -DZLIB"
|
|
|
|
PLATFORM_LDFLAGS="$PLATFORM_LDFLAGS -lz"
|
|
|
|
fi
|
|
|
|
|
|
|
|
# Test whether bzip library is installed
|
|
|
|
$CXX $CFLAGS $COMMON_FLAGS -x c++ - -o /dev/null 2>/dev/null <<EOF
|
|
|
|
#include <bzlib.h>
|
|
|
|
int main() {}
|
|
|
|
EOF
|
|
|
|
if [ "$?" = 0 ]; then
|
|
|
|
COMMON_FLAGS="$COMMON_FLAGS -DBZIP2"
|
|
|
|
PLATFORM_LDFLAGS="$PLATFORM_LDFLAGS -lbz2"
|
|
|
|
fi
|
|
|
|
|
|
|
|
# Test whether tcmalloc is available
|
|
|
|
$CXX $CFLAGS -x c++ - -o /dev/null -ltcmalloc 2>/dev/null <<EOF
|
|
|
|
int main() {}
|
|
|
|
EOF
|
|
|
|
if [ "$?" = 0 ]; then
|
|
|
|
PLATFORM_LDFLAGS="$PLATFORM_LDFLAGS -ltcmalloc"
|
|
|
|
fi
|
|
|
|
fi
|
|
|
|
|
|
|
|
# shall we use HDFS?
|
|
|
|
|
|
|
|
if test "$USE_HDFS"; then
|
|
|
|
if test -z "$JAVA_HOME"; then
|
|
|
|
echo "JAVA_HOME has to be set for HDFS usage."
|
|
|
|
exit 1
|
|
|
|
fi
|
|
|
|
HDFS_CCFLAGS="$HDFS_CCFLAGS -I$JAVA_HOME/include -I$JAVA_HOME/include/linux -DUSE_HDFS"
|
|
|
|
HDFS_LDFLAGS="$HDFS_LDFLAGS -Wl,--no-whole-archive hdfs/libhdfs.a -L$JAVA_HOME/jre/lib/amd64"
|
|
|
|
HDFS_LDFLAGS="$HDFS_LDFLAGS -L$JAVA_HOME/jre/lib/amd64/server -L$GLIBC_RUNTIME_PATH/lib"
|
|
|
|
HDFS_LDFLAGS="$HDFS_LDFLAGS -ldl -lverify -ljava -ljvm"
|
|
|
|
COMMON_FLAGS="$COMMON_FLAGS $HDFS_CCFLAGS"
|
|
|
|
PLATFORM_LDFLAGS="$PLATFORM_LDFLAGS $HDFS_LDFLAGS"
|
|
|
|
fi
|
|
|
|
|
|
|
|
# if Intel SSE instruction set is supported, set USE_SSE=" -msse -msse4.2 "
|
|
|
|
COMMON_FLAGS="$COMMON_FLAGS $USE_SSE"
|
|
|
|
|
|
|
|
PLATFORM_CCFLAGS="$PLATFORM_CCFLAGS $COMMON_FLAGS"
|
|
|
|
PLATFORM_CXXFLAGS="$PLATFORM_CXXFLAGS $COMMON_FLAGS"
|
|
|
|
|
|
|
|
VALGRIND_VER="$VALGRIND_VER"
|
|
|
|
|
|
|
|
echo "CC=$CC" >> $OUTPUT
|
|
|
|
echo "CXX=$CXX" >> $OUTPUT
|
|
|
|
echo "PLATFORM=$PLATFORM" >> $OUTPUT
|
|
|
|
echo "PLATFORM_LDFLAGS=$PLATFORM_LDFLAGS" >> $OUTPUT
|
|
|
|
echo "VALGRIND_VER=$VALGRIND_VER" >> $OUTPUT
|
|
|
|
echo "PLATFORM_CCFLAGS=$PLATFORM_CCFLAGS" >> $OUTPUT
|
|
|
|
echo "PLATFORM_CXXFLAGS=$PLATFORM_CXXFLAGS" >> $OUTPUT
|
|
|
|
echo "PLATFORM_SHARED_CFLAGS=$PLATFORM_SHARED_CFLAGS" >> $OUTPUT
|
|
|
|
echo "PLATFORM_SHARED_EXT=$PLATFORM_SHARED_EXT" >> $OUTPUT
|
|
|
|
echo "PLATFORM_SHARED_LDFLAGS=$PLATFORM_SHARED_LDFLAGS" >> $OUTPUT
|
|
|
|
echo "PLATFORM_SHARED_VERSIONED=$PLATFORM_SHARED_VERSIONED" >> $OUTPUT
|
|
|
|
echo "EXEC_LDFLAGS=$EXEC_LDFLAGS" >> $OUTPUT
|
|
|
|
echo "JEMALLOC_INCLUDE=$JEMALLOC_INCLUDE" >> $OUTPUT
|
|
|
|
echo "JEMALLOC_LIB=$JEMALLOC_LIB" >> $OUTPUT
|