Major CircleCI/Linux fixes / tweaks / enhancements (#7078)
Summary: Primarily, this change adds a way to work around a bug limiting the effective output (and therefore debugability) of the Linux builds using parallel make. We would get make[1]: write error: stdout probably due to a kernel bug, apparently affecting both available ubuntu 16 machine images (maybe not affecting docker images, less horsepower). https://bugs.launchpad.net/ubuntu/+source/linux-signed/+bug/1814393 Now in the CircleCI config, make output on Ubuntu is piped through a custom 'cat' that ignores EAGAIN errors, which seems to fix the problem. Significant other changes: * Add another linux build that combines * LIB_MODE=shared, to ensure this works with compile and unit test execution * Alternative rocksdb namespace, to ensure this works (not rely on Travis) * ASSERT_STATUS_CHECKED=1, but with building all unit tests and running those expected to pass with it * Run release build with and without gflags. (Was running only without, ignore large swaths of code in a normal release build! Two regressions in this build, only with gflags, in the last week not caught by CI!) * Use gflags with unity and LITE build, as typical case. Debugability improvements: * Use V=1 to show commands being executed (thanks to EAGAIN work-around) * Print kernel version and compiler versions as part of V=1 output from Makefile Cosmetic other changes: * Put more commands on one line, for less clutter in CircleCI output pages * Remove redundant "all" in "make all check" and put make command options before targets * Change some recursive "make clean" into dependency on "clean," toward minimizing unnecessary overhead (detect platform, build version, etc.) of extra recursive makes Pull Request resolved: https://github.com/facebook/rocksdb/pull/7078 Reviewed By: siying Differential Revision: D22391647 Pulled By: pdillinger fbshipit-source-id: d446fccf5a8c568b37dc8748621c8a5c546fe135main
Peter Dillinger
4 years ago
committed by
Facebook GitHub Bot
parent
a693341604
commit
92731b6b4a
@ -0,0 +1,54 @@ |
||||
#! /bin/bash |
||||
|
||||
# Work around issue with parallel make output causing random error, as in |
||||
# make[1]: write error: stdout |
||||
# Probably due to a kernel bug: |
||||
# https://bugs.launchpad.net/ubuntu/+source/linux-signed/+bug/1814393 |
||||
# Seems to affect image ubuntu-1604:201903-01 and ubuntu-1604:202004-01 |
||||
|
||||
cd "$(dirname $0)" |
||||
|
||||
if [ ! -x cat_ignore_eagain.out ]; then |
||||
cc -x c -o cat_ignore_eagain.out - << EOF |
||||
#include <unistd.h> |
||||
#include <errno.h> |
||||
#include <stdio.h> |
||||
int main() { |
||||
int n, m, p; |
||||
char buf[1024]; |
||||
for (;;) { |
||||
n = read(STDIN_FILENO, buf, 1024); |
||||
if (n > 0 && n <= 1024) { |
||||
for (m = 0; m < n;) { |
||||
p = write(STDOUT_FILENO, buf + m, n - m); |
||||
if (p < 0) { |
||||
if (errno == EAGAIN) { |
||||
// ignore but pause a bit |
||||
usleep(100); |
||||
} else { |
||||
perror("write failed"); |
||||
return 42; |
||||
} |
||||
} else { |
||||
m += p; |
||||
} |
||||
} |
||||
} else if (n < 0) { |
||||
if (errno == EAGAIN) { |
||||
// ignore but pause a bit |
||||
usleep(100); |
||||
} else { |
||||
// Some non-ignorable error |
||||
perror("read failed"); |
||||
return 43; |
||||
} |
||||
} else { |
||||
// EOF |
||||
return 0; |
||||
} |
||||
} |
||||
} |
||||
EOF |
||||
fi |
||||
|
||||
exec ./cat_ignore_eagain.out |
||||
Loading…
Reference in new issue