Summary:
Instead of existing calls to ps from gnu_parallel, call a new wrapper that does ps, looks for unit test like processes, and uses pstack or gdb to print thread stack traces. Also, using `ps -wwf` instead of `ps -wf` ensures output is not cut off.
For security, CircleCI runs with security restrictions on ptrace (/proc/sys/kernel/yama/ptrace_scope = 1), and this change adds a work-around to `InstallStackTraceHandler()` (only used by testing tools) to allow any process from the same user to debug it. (I've also touched >100 files to ensure all the unit tests call this function.)
Pull Request resolved: https://github.com/facebook/rocksdb/pull/10828
Test Plan: local manual + temporary infinite loop in a unit test to observe in CircleCI
Reviewed By: hx235
Differential Revision: D40447634
Pulled By: pdillinger
fbshipit-source-id: 718a4c4a5b54fa0f9af2d01a446162b45e5e84e1
Summary:
In https://github.com/facebook/rocksdb/issues/8539 I accidentally only checked for GCC TSAN, which is
what I tested locally, while CircleCI and FB CI use clang TSAN. Related:
other existing code like in stack_trace.cc only check for clang TSAN.
I've now standardized these to the GCC convention in port/lang.h, so now
#ifdef __SANITIZE_THREAD__
can check for any TSAN (assuming lang.h include)
Pull Request resolved: https://github.com/facebook/rocksdb/pull/8543
Test Plan:
Put an assert(false) in slice_test and look for the NOTE
about "signal-unsafe call", both GCC and clang. Eventually, CircleCI
TSAN in https://github.com/facebook/rocksdb/issues/8538
Reviewed By: zhichao-cao
Differential Revision: D29728483
Pulled By: pdillinger
fbshipit-source-id: 8a3b8015c2ed48078214c3ee17146a2c3f11c9f7
Summary:
TSAN reports that our stack trace handler makes unsafe calls
during a signal handler. I just tried fixing some of them and I don't
think it's fixable unless we can get away from using FILE stdio. Even if
we can use lower level functions only, I'm not sure it's fixed.
I also tried suppressing the reports with function and file level TSAN
suppression, but that doesn't seem to work, perhaps because the
violation is reported on the callee, not the caller.
So I added a warning to be printed whenever these violations would be
reported that they are practically ignorable.
Internal ref: T77844138
Pull Request resolved: https://github.com/facebook/rocksdb/pull/7723
Test Plan:
run external_sst_file_test with seeded abort(), with TSAN
(TSAN warnings + new warning) and without TSAN (no warning, just stack
trace).
Reviewed By: akankshamahajan15
Differential Revision: D25228011
Pulled By: pdillinger
fbshipit-source-id: 3eda1d6e7ca3cdc64076cf99ae954168837d2818
Summary:
While rocksdb can compile on both macOS and Linux with Buck, it couldn't be
compiled on Windows. The only way to compile it on Windows was with the CMake
build.
To keep the multi-platform complexity low, I've simply included all the Windows
bits in the TARGETS file, and added large #if blocks when not on Windows, the
same was done on the posix specific files.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/7406
Test Plan:
On my devserver:
buck test //rocksdb/...
On Windows:
buck build mode/win //rocksdb/src:rocksdb_lib
Reviewed By: pdillinger
Differential Revision: D23874358
Pulled By: xavierd
fbshipit-source-id: 8768b5d16d7e8f44b5ca1e2483881ca4b24bffbe
Summary:
This PR implements a fault injection mechanism for injecting errors in reads in db_stress. The FaultInjectionTestFS is used for this purpose. A thread local structure is used to track the errors, so that each db_stress thread can independently enable/disable error injection and verify observed errors against expected errors. This is initially enabled only for Get and MultiGet, but can be extended to iterator as well once its proven stable.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/6538
Test Plan:
crash_test
make check
Reviewed By: riversand963
Differential Revision: D20714347
Pulled By: anand1976
fbshipit-source-id: d7598321d4a2d72bda0ced57411a337a91d87dc7
Summary:
When dynamically linking two binaries together, different builds of RocksDB from two sources might cause errors. To provide a tool for user to solve the problem, the RocksDB namespace is changed to a flag which can be overridden in build time.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/6433
Test Plan: Build release, all and jtest. Try to build with ROCKSDB_NAMESPACE with another flag.
Differential Revision: D19977691
fbshipit-source-id: aa7f2d0972e1c31d75339ac48478f34f6cfcfb3e
Summary:
This reverts the previous commit 1d7048c598, which broke the build.
Did a `git revert 1d7048c`.
Closes https://github.com/facebook/rocksdb/pull/2627
Differential Revision: D5476473
Pulled By: sagar0
fbshipit-source-id: 4756ff5c0dfc88c17eceb00e02c36176de728d06
Summary: This uses `clang-tidy` to comment out unused parameters (in functions, methods and lambdas) in fbcode. Cases that the tool failed to handle are fixed manually.
Reviewed By: igorsugak
Differential Revision: D5454343
fbshipit-source-id: 5dee339b4334e25e963891b519a5aa81fbf627b2
Summary:
Replacement of #2147
The change was squashed due to a lot of conflicts.
Closes https://github.com/facebook/rocksdb/pull/2194
Differential Revision: D4929799
Pulled By: siying
fbshipit-source-id: 5cd49c254737a1d5ac13f3c035f128e86524c581
Summary:
The address of the array of string pointers is returned as the function result of backtrace_symbols(). This array is malloced by backtrace_symbols(), and must be freed by the caller.
Closes https://github.com/facebook/rocksdb/pull/1692
Differential Revision: D4355737
Pulled By: IslamAbdelRahman
fbshipit-source-id: 5742035
* stack_trace,cc: The current Stacktrace code does not compile for FreeBSD
So set it to generate empty routines
* stack_trace,cc: The current Stacktrace code does not compile for FreeBSD
Use the definition also used in other commits
* Musl libc does not provide adaptive mutex. Added feature test for PTHREAD_MUTEX_ADAPTIVE_NP.
* Musl libc does not provide backtrace(3). Added a feature check for backtrace(3).
* Fixed compiler error.
* Musl libc does not implement backtrace(3). Added platform check for libexecinfo.
* Alpine does not appear to support gcc -pg option. By default (gcc has PIE option enabled) it fails with:
gcc: error: -pie and -pg|p|profile are incompatible when linking
When -fno-PIE and -nopie are used it fails with:
/usr/lib/gcc/x86_64-alpine-linux-musl/5.3.0/../../../../x86_64-alpine-linux-musl/bin/ld: cannot find gcrt1.o: No such file or directory
Added gcc -pg platform test and output PROFILING_FLAGS accordingly. Replaced pg var in Makefile with PROFILING_FLAGS.
* fix segfault when TEST_IOCTL_FRIENDLY_TMPDIR is undefined and default candidates are not suitable
* use ASSERT_DOUBLE_EQ instead of ASSERT_EQ
* When compiled with ROCKSDB_MALLOC_USABLE_SIZE UniversalCompactionFourPaths and UniversalCompactionSecondPathRatio tests fail due to premature memtable flushes on systems with 16-byte alignment. Arena runs out of block space before GenerateNewFile() completes.
Increased options.write_buffer_size.
Summary:
Make it build for CYGWIN.
Need to define "-std=gnu++11" instead of "-std=c++11" and use some replacement functions.
Test Plan: Build it and run some unit tests in CYGWIN
Reviewers: yhchiang, rven, anthony, kradhakrishnan, igor
Reviewed By: igor
Subscribers: leveldb, dhruba
Differential Revision: https://reviews.facebook.net/D37605
Summary:
So glibc is not -Wshadow-safe, so we need to turn it off :(
error: ‘int sigaction(int, const sigaction*, sigaction*)’ hides
constructor for ‘struct sigaction’
The rest of the changes in this diff is that we include .h files under rocksdb namespace, which is a no-no.
Test Plan: compiles now
Reviewers: ljin, yhchiang, rven, sdong
Reviewed By: sdong
Subscribers: dhruba, leveldb
Differential Revision: https://reviews.facebook.net/D28413
Summary:
Abstract out FlushProcess and take it out of DBImpl.
This also includes taking DeletionState outside of DBImpl.
Currently this diff is only doing the refactoring. Future work includes:
1. Decoupling flush_process.cc, make it depend on less state
2. Write flush_process_test, which will mock out everything that FlushProcess depends on and test it in isolation
Test Plan: make check
Reviewers: rven, yhchiang, sdong, ljin
Reviewed By: ljin
Subscribers: dhruba, leveldb
Differential Revision: https://reviews.facebook.net/D27561
Summary:
Now this gives us the real deal stack trace:
Assertion failed: (false), function GetProperty, file db/db_impl.cc, line 4072.
Received signal 6 (Abort trap: 6)
#0 0x7fff57ce39b9
#1 abort (in libsystem_c.dylib) + 125
#2 basename (in libsystem_c.dylib) + 0
#3 rocksdb::DBImpl::GetProperty(rocksdb::ColumnFamilyHandle*, rocksdb::Slice const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >*) (in db_test) (db_impl.cc:4072)
#4 rocksdb::_Test_Empty::_Run() (in db_test) (testharness.h:68)
#5 rocksdb::_Test_Empty::_RunIt() (in db_test) (db_test.cc:1005)
#6 rocksdb::test::RunAllTests() (in db_test) (testharness.cc:60)
#7 main (in db_test) (db_test.cc:6697)
#8 start (in libdyld.dylib) + 1
Test Plan: added artificial assert, saw great stack trace
Reviewers: haobo, dhruba, ljin
Reviewed By: haobo
CC: leveldb
Differential Revision: https://reviews.facebook.net/D18309
Summary: Sometimes, our tests fail because of normal `assert` call. It would be helpful to see stack trace in that case, too.
Test Plan: Added `assert(false)` and verified it prints out stack trace
Reviewers: haobo, dhruba, sdong, ljin, yhchiang
Reviewed By: haobo
CC: leveldb
Differential Revision: https://reviews.facebook.net/D18291
Summary: While debugging Mac-only issue with ThreadLocalPtr, this was very useful. Let's print out stack trace in MAC OS, too.
Test Plan: Verified that somewhat useful stack trace was generated on mac. Will run PrintStack() on linux, too.
Reviewers: ljin, haobo
Reviewed By: haobo
CC: leveldb
Differential Revision: https://reviews.facebook.net/D18189
Summary:
This will help me a lot! When we hit an assertion in unittest, we get the whole stack trace now.
Also, changed stack trace a bit, we now include actual demangled C++ class::function symbols!
Test Plan: Added ASSERT_TRUE(false) to a test, observed a stack trace
Reviewers: haobo, dhruba, kailiu
Reviewed By: kailiu
CC: leveldb
Differential Revision: https://reviews.facebook.net/D14499
Summary:
Change namespace from leveldb to rocksdb. This allows a single
application to link in open-source leveldb code as well as
rocksdb code into the same process.
Test Plan: compile rocksdb
Reviewers: emayanke
Reviewed By: emayanke
CC: leveldb
Differential Revision: https://reviews.facebook.net/D13287
Summary: Some scripts (like regression_build_test.sh) redirect stdio to a tmp file and delete it on exit. This would miss the stack trace output on segfault. Output to stderr would hopefully show us the stack trace in the continuous build output.
Test Plan: ./signal_test, make check
Reviewers: dhruba
Reviewed By: dhruba
CC: leveldb
Differential Revision: https://reviews.facebook.net/D10485
Summary:
This diff provides the ability to print out a stacktrace when the process receives certain signals.
Currently, we enable this for the following signals (program error related):
SIGILL SIGSEGV SIGBUS SIGABRT
Application simply #include "util/stack_trace.h" and call leveldb::InstallStackTraceHandler() during initialization, if signal handler is needed. It's not done automatically when openning db, because it's the application(process)'s responsibility to install signal handler and some applications might already have their own (like fbcode).
Sample output:
Received signal 11 (Segmentation fault)
#0 0x408ff0 ./signal_test() [0x408ff0] /home/haobo/rocksdb/util/signal_test.cc:4
#1 0x40827d ./signal_test() [0x40827d] /home/haobo/rocksdb/util/signal_test.cc:24
#2 0x7f8bb183172e /usr/local/fbcode/gcc-4.7.1-glibc-2.14.1/lib/libc.so.6(__libc_start_main+0x10e) [0x7f8bb183172e] ??:0
#3 0x408ebc ./signal_test() [0x408ebc] /home/engshare/third-party/src/glibc/glibc-2.14.1/glibc-2.14.1/csu/../sysdeps/x86_64/elf/start.S:113
Segmentation fault (core dumped)
For each frame, we print the raw pointer, the symbol provided by backtrace_symbols (still not good enough), and the source file/line. Note that address translation is done by directly shell out to addr2line. ??:0 means addr2line fails to do the translation. Hacky, but I think it's good for now.
Test Plan: signal_test.cc
Reviewers: dhruba, MarkCallaghan
Reviewed By: dhruba
CC: leveldb
Differential Revision: https://reviews.facebook.net/D10173
Peter Dillinger