You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
rocksdb/build_tools/cont_integration.sh

94 lines
2.5 KiB

Framework for enabling continuous RocksDB build and tests Summary: The main PHP code churn is caused by extracting the common code from `FacebookArcanistConfiguration.php` and `FacebookOldArcanistConfiguration.php` into `RocksDBCommonDeterminator.php`. This is necessary both for reducing the duplication of code and making sure that we can execute the common core logic separately from continuous runs. The main logic in `RocksDBCommonDeterminator.php` remains quite the same with the exception of some things: - Adding separation between the cases when a diff is submitted //vs.// when the code is triggered from a continuous run. There are certain actions which we should do in a case of diff only. - Adding reporting - now the person who authored the diff will receive e-mail notifications if any of the jobs have failed. - Enabling assertions and making sure that we'll terminate on failure. This is an internal code used by competent engineers, so instead of `if (!condition) { echo "Something"; exit(1); }` for every invariant I think that `assert(condition)` provides better readability with the same behavior. Especially taking into account that we're talking about things which shouldn't ever happen. Enabling this entire process will be triggered internally and will be a subject of a separate code review. We should discuss the details of triggering continuous RocksDB build and tests on that diff. Test Plan: - Make sure that `[p]arc diff` scenario isn't broken by verifying that tests validating this diff will pass. - Private testing of triggering the continuous build script. - Once the changes will land then author an internal job which will use the script and verify its validity. Reviewers: sdong, yhchiang, kradhakrishnan Reviewed By: kradhakrishnan Subscribers: andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D59811
8 years ago
#!/bin/bash
#
# Copyright (c) 2016, Facebook. All rights reserved.
#
# Overall wrapper script for RocksDB continuous builds. The implementation is a
# trivial pulling scheme. We loop infinitely, check if any new changes have been
# committed, if yes then trigger a Sandcastle run, and finally go to sleep again
# for a certain interval.
#
function log {
DATE=`date +%Y-%m-%d:%H:%M:%S`
echo $DATE $@
}
function log_err {
log "ERROR: $@ Error code: $?."
Framework for enabling continuous RocksDB build and tests Summary: The main PHP code churn is caused by extracting the common code from `FacebookArcanistConfiguration.php` and `FacebookOldArcanistConfiguration.php` into `RocksDBCommonDeterminator.php`. This is necessary both for reducing the duplication of code and making sure that we can execute the common core logic separately from continuous runs. The main logic in `RocksDBCommonDeterminator.php` remains quite the same with the exception of some things: - Adding separation between the cases when a diff is submitted //vs.// when the code is triggered from a continuous run. There are certain actions which we should do in a case of diff only. - Adding reporting - now the person who authored the diff will receive e-mail notifications if any of the jobs have failed. - Enabling assertions and making sure that we'll terminate on failure. This is an internal code used by competent engineers, so instead of `if (!condition) { echo "Something"; exit(1); }` for every invariant I think that `assert(condition)` provides better readability with the same behavior. Especially taking into account that we're talking about things which shouldn't ever happen. Enabling this entire process will be triggered internally and will be a subject of a separate code review. We should discuss the details of triggering continuous RocksDB build and tests on that diff. Test Plan: - Make sure that `[p]arc diff` scenario isn't broken by verifying that tests validating this diff will pass. - Private testing of triggering the continuous build script. - Once the changes will land then author an internal job which will use the script and verify its validity. Reviewers: sdong, yhchiang, kradhakrishnan Reviewed By: kradhakrishnan Subscribers: andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D59811
8 years ago
}
function update_repo_status {
}
Framework for enabling continuous RocksDB build and tests Summary: The main PHP code churn is caused by extracting the common code from `FacebookArcanistConfiguration.php` and `FacebookOldArcanistConfiguration.php` into `RocksDBCommonDeterminator.php`. This is necessary both for reducing the duplication of code and making sure that we can execute the common core logic separately from continuous runs. The main logic in `RocksDBCommonDeterminator.php` remains quite the same with the exception of some things: - Adding separation between the cases when a diff is submitted //vs.// when the code is triggered from a continuous run. There are certain actions which we should do in a case of diff only. - Adding reporting - now the person who authored the diff will receive e-mail notifications if any of the jobs have failed. - Enabling assertions and making sure that we'll terminate on failure. This is an internal code used by competent engineers, so instead of `if (!condition) { echo "Something"; exit(1); }` for every invariant I think that `assert(condition)` provides better readability with the same behavior. Especially taking into account that we're talking about things which shouldn't ever happen. Enabling this entire process will be triggered internally and will be a subject of a separate code review. We should discuss the details of triggering continuous RocksDB build and tests on that diff. Test Plan: - Make sure that `[p]arc diff` scenario isn't broken by verifying that tests validating this diff will pass. - Private testing of triggering the continuous build script. - Once the changes will land then author an internal job which will use the script and verify its validity. Reviewers: sdong, yhchiang, kradhakrishnan Reviewed By: kradhakrishnan Subscribers: andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D59811
8 years ago
#
# Execution starts here.
#
# Path to the determinator from the root of the RocksDB repo.
CONTRUN_DETERMINATOR=./arcanist_util/config/RocksDBCommonHelper.php
Framework for enabling continuous RocksDB build and tests Summary: The main PHP code churn is caused by extracting the common code from `FacebookArcanistConfiguration.php` and `FacebookOldArcanistConfiguration.php` into `RocksDBCommonDeterminator.php`. This is necessary both for reducing the duplication of code and making sure that we can execute the common core logic separately from continuous runs. The main logic in `RocksDBCommonDeterminator.php` remains quite the same with the exception of some things: - Adding separation between the cases when a diff is submitted //vs.// when the code is triggered from a continuous run. There are certain actions which we should do in a case of diff only. - Adding reporting - now the person who authored the diff will receive e-mail notifications if any of the jobs have failed. - Enabling assertions and making sure that we'll terminate on failure. This is an internal code used by competent engineers, so instead of `if (!condition) { echo "Something"; exit(1); }` for every invariant I think that `assert(condition)` provides better readability with the same behavior. Especially taking into account that we're talking about things which shouldn't ever happen. Enabling this entire process will be triggered internally and will be a subject of a separate code review. We should discuss the details of triggering continuous RocksDB build and tests on that diff. Test Plan: - Make sure that `[p]arc diff` scenario isn't broken by verifying that tests validating this diff will pass. - Private testing of triggering the continuous build script. - Once the changes will land then author an internal job which will use the script and verify its validity. Reviewers: sdong, yhchiang, kradhakrishnan Reviewed By: kradhakrishnan Subscribers: andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D59811
8 years ago
# Value of the previous commit.
PREV_COMMIT=
log "Starting to monitor for new RocksDB changes ..."
log "Running under `pwd` as `whoami`."
git remote -v
git status
Framework for enabling continuous RocksDB build and tests Summary: The main PHP code churn is caused by extracting the common code from `FacebookArcanistConfiguration.php` and `FacebookOldArcanistConfiguration.php` into `RocksDBCommonDeterminator.php`. This is necessary both for reducing the duplication of code and making sure that we can execute the common core logic separately from continuous runs. The main logic in `RocksDBCommonDeterminator.php` remains quite the same with the exception of some things: - Adding separation between the cases when a diff is submitted //vs.// when the code is triggered from a continuous run. There are certain actions which we should do in a case of diff only. - Adding reporting - now the person who authored the diff will receive e-mail notifications if any of the jobs have failed. - Enabling assertions and making sure that we'll terminate on failure. This is an internal code used by competent engineers, so instead of `if (!condition) { echo "Something"; exit(1); }` for every invariant I think that `assert(condition)` provides better readability with the same behavior. Especially taking into account that we're talking about things which shouldn't ever happen. Enabling this entire process will be triggered internally and will be a subject of a separate code review. We should discuss the details of triggering continuous RocksDB build and tests on that diff. Test Plan: - Make sure that `[p]arc diff` scenario isn't broken by verifying that tests validating this diff will pass. - Private testing of triggering the continuous build script. - Once the changes will land then author an internal job which will use the script and verify its validity. Reviewers: sdong, yhchiang, kradhakrishnan Reviewed By: kradhakrishnan Subscribers: andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D59811
8 years ago
# Paranoia. Make sure that we're using the right branch.
git checkout master
if [ ! $? -eq 0 ]; then
log_err "This is not good. Can't checkout master. Bye-bye!"
exit 1
fi
# We'll run forever and let the execution environment terminate us if we'll
# exceed whatever timeout is set for the job.
while true;
do
# Get the latest changes committed.
git pull --rebase
if [ $? -eq 0 ]; then
LAST_COMMIT=`git log -1 | head -1 | grep commit | awk '{ print $2; }'`
log "Last commit is '$LAST_COMMIT', previous commit is '$PREV_COMMIT'."
if [ "$PREV_COMMIT" == "$LAST_COMMIT" ]; then
log "There were no changes since the last time I checked. Going to sleep."
else
if [ ! -z "$LAST_COMMIT" ]; then
log "New code has been committed or previous commit not known. " \
Framework for enabling continuous RocksDB build and tests Summary: The main PHP code churn is caused by extracting the common code from `FacebookArcanistConfiguration.php` and `FacebookOldArcanistConfiguration.php` into `RocksDBCommonDeterminator.php`. This is necessary both for reducing the duplication of code and making sure that we can execute the common core logic separately from continuous runs. The main logic in `RocksDBCommonDeterminator.php` remains quite the same with the exception of some things: - Adding separation between the cases when a diff is submitted //vs.// when the code is triggered from a continuous run. There are certain actions which we should do in a case of diff only. - Adding reporting - now the person who authored the diff will receive e-mail notifications if any of the jobs have failed. - Enabling assertions and making sure that we'll terminate on failure. This is an internal code used by competent engineers, so instead of `if (!condition) { echo "Something"; exit(1); }` for every invariant I think that `assert(condition)` provides better readability with the same behavior. Especially taking into account that we're talking about things which shouldn't ever happen. Enabling this entire process will be triggered internally and will be a subject of a separate code review. We should discuss the details of triggering continuous RocksDB build and tests on that diff. Test Plan: - Make sure that `[p]arc diff` scenario isn't broken by verifying that tests validating this diff will pass. - Private testing of triggering the continuous build script. - Once the changes will land then author an internal job which will use the script and verify its validity. Reviewers: sdong, yhchiang, kradhakrishnan Reviewed By: kradhakrishnan Subscribers: andrewkr, dhruba Differential Revision: https://reviews.facebook.net/D59811
8 years ago
"Will trigger the tests."
PREV_COMMIT=$LAST_COMMIT
log "Updated previous commit to '$PREV_COMMIT'."
#
# This is where we'll trigger the Sandcastle run. The values for
# HTTPS_APP_VALUE and HTTPS_APP_VALUE will be set in the container we're
# running in.
#
POST_RECEIVE_HOOK=1 php $CONTRUN_DETERMINATOR
if [ $? -eq 0 ]; then
log "Sandcastle run successfully triggered."
else
log_err "Failed to trigger Sandcastle run."
fi
else
log_err "Previous commit not updated. Don't know what the last one is."
fi
fi
else
log_err "git pull --rebase failed. Will skip running tests for now."
fi
# Always sleep, even if errors happens while trying to determine the latest
# commit. This will prevent us terminating in case of transient errors.
log "Will go to sleep for 5 minutes."
sleep 5m
done