Major CircleCI/Linux fixes / tweaks / enhancements (#7078)
Summary: Primarily, this change adds a way to work around a bug limiting the effective output (and therefore debugability) of the Linux builds using parallel make. We would get make[1]: write error: stdout probably due to a kernel bug, apparently affecting both available ubuntu 16 machine images (maybe not affecting docker images, less horsepower). https://bugs.launchpad.net/ubuntu/+source/linux-signed/+bug/1814393 Now in the CircleCI config, make output on Ubuntu is piped through a custom 'cat' that ignores EAGAIN errors, which seems to fix the problem. Significant other changes: * Add another linux build that combines * LIB_MODE=shared, to ensure this works with compile and unit test execution * Alternative rocksdb namespace, to ensure this works (not rely on Travis) * ASSERT_STATUS_CHECKED=1, but with building all unit tests and running those expected to pass with it * Run release build with and without gflags. (Was running only without, ignore large swaths of code in a normal release build! Two regressions in this build, only with gflags, in the last week not caught by CI!) * Use gflags with unity and LITE build, as typical case. Debugability improvements: * Use V=1 to show commands being executed (thanks to EAGAIN work-around) * Print kernel version and compiler versions as part of V=1 output from Makefile Cosmetic other changes: * Put more commands on one line, for less clutter in CircleCI output pages * Remove redundant "all" in "make all check" and put make command options before targets * Change some recursive "make clean" into dependency on "clean," toward minimizing unnecessary overhead (detect platform, build version, etc.) of extra recursive makes Pull Request resolved: https://github.com/facebook/rocksdb/pull/7078 Reviewed By: siying Differential Revision: D22391647 Pulled By: pdillinger fbshipit-source-id: d446fccf5a8c568b37dc8748621c8a5c546fe135main
Peter Dillinger
5 years ago
committed by
Facebook GitHub Bot
parent
a693341604
commit
92731b6b4a
@ -0,0 +1,54 @@ |
|||||||
|
#! /bin/bash |
||||||
|
|
||||||
|
# Work around issue with parallel make output causing random error, as in |
||||||
|
# make[1]: write error: stdout |
||||||
|
# Probably due to a kernel bug: |
||||||
|
# https://bugs.launchpad.net/ubuntu/+source/linux-signed/+bug/1814393 |
||||||
|
# Seems to affect image ubuntu-1604:201903-01 and ubuntu-1604:202004-01 |
||||||
|
|
||||||
|
cd "$(dirname $0)" |
||||||
|
|
||||||
|
if [ ! -x cat_ignore_eagain.out ]; then |
||||||
|
cc -x c -o cat_ignore_eagain.out - << EOF |
||||||
|
#include <unistd.h> |
||||||
|
#include <errno.h> |
||||||
|
#include <stdio.h> |
||||||
|
int main() { |
||||||
|
int n, m, p; |
||||||
|
char buf[1024]; |
||||||
|
for (;;) { |
||||||
|
n = read(STDIN_FILENO, buf, 1024); |
||||||
|
if (n > 0 && n <= 1024) { |
||||||
|
for (m = 0; m < n;) { |
||||||
|
p = write(STDOUT_FILENO, buf + m, n - m); |
||||||
|
if (p < 0) { |
||||||
|
if (errno == EAGAIN) { |
||||||
|
// ignore but pause a bit |
||||||
|
usleep(100); |
||||||
|
} else { |
||||||
|
perror("write failed"); |
||||||
|
return 42; |
||||||
|
} |
||||||
|
} else { |
||||||
|
m += p; |
||||||
|
} |
||||||
|
} |
||||||
|
} else if (n < 0) { |
||||||
|
if (errno == EAGAIN) { |
||||||
|
// ignore but pause a bit |
||||||
|
usleep(100); |
||||||
|
} else { |
||||||
|
// Some non-ignorable error |
||||||
|
perror("read failed"); |
||||||
|
return 43; |
||||||
|
} |
||||||
|
} else { |
||||||
|
// EOF |
||||||
|
return 0; |
||||||
|
} |
||||||
|
} |
||||||
|
} |
||||||
|
EOF |
||||||
|
fi |
||||||
|
|
||||||
|
exec ./cat_ignore_eagain.out |
||||||
Loading…
Reference in new issue