add call to install superversion and schedule work in enableautocompactions

Summary:
This patch fixes https://github.com/facebook/mysql-5.6/issues/121

There is a recent change in rocksdb to disable auto compactions on startup: https://reviews.facebook.net/D51147. However, there is a small timing window where a column family needs to be compacted and schedules a compaction, but the scheduled compaction fails when it checks the disable_auto_compactions setting. The expectation is once the application is ready, it will call EnableAutoCompactions() to allow new compactions to go through. However, if the Column family is stalled because L0 is full, and no writes can go through, it is possible the column family may never have a new compaction request get scheduled. EnableAutoCompaction() should probably schedule an new flush and compaction event when it resets disable_auto_compaction.

Using InstallSuperVersionAndScheduleWork, we call SchedulePendingFlush,
SchedulePendingCompaction, as well as MaybeScheduleFlushOrcompaction on all the
column families to avoid the situation above.

This is still a first pass for feedback.
Could also just call SchedePendingFlush and SchedulePendingCompaction directly.

Test Plan:
Run on Asan build
cd _build-5.6-ASan/ && ./mysql-test/mtr --mem --big --testcase-timeout=36000 --suite-timeout=12000 --parallel=16 --suite=rocksdb,rocksdb_rpl,rocksdb_sys_vars --mysqld=--default-storage-engine=rocksdb --mysqld=--skip-innodb --mysqld=--default-tmp-storage-engine=MyISAM --mysqld=--rocksdb rocksdb_rpl.rpl_rocksdb_stress_crash --repeat=1000

Ensure that it no longer hangs during the test.

Reviewers: hermanlee4, yhchiang, anthony

Reviewed By: anthony

Subscribers: leveldb, yhchiang, dhruba

Differential Revision: https://reviews.facebook.net/D51747
main
Alex Yang 9 years ago
parent 22c6b50ee8
commit 33e09c0e19
  1. 18
      db/db_impl.cc
  2. 10
      include/rocksdb/db.h

@ -2393,17 +2393,19 @@ Status DBImpl::EnableAutoCompaction(
const std::vector<ColumnFamilyHandle*>& column_family_handles) { const std::vector<ColumnFamilyHandle*>& column_family_handles) {
Status s; Status s;
for (auto cf_ptr : column_family_handles) { for (auto cf_ptr : column_family_handles) {
// check options here, enable only if didn't initially disable Status status =
if (s.ok()) { this->SetOptions(cf_ptr, {{"disable_auto_compactions", "false"}});
s = this->SetOptions(cf_ptr, {{"disable_auto_compactions", "false"}}); if (status.ok()) {
ColumnFamilyData* cfd =
reinterpret_cast<ColumnFamilyHandleImpl*>(cf_ptr)->cfd();
InstrumentedMutexLock guard_lock(&mutex_);
delete this->InstallSuperVersionAndScheduleWork(
cfd, nullptr, *cfd->GetLatestMutableCFOptions());
} else {
s = status;
} }
} }
if (s.ok()) {
InstrumentedMutexLock guard_lock(&mutex_);
MaybeScheduleFlushOrCompaction();
}
return s; return s;
} }

@ -559,8 +559,14 @@ class DB {
virtual Status ContinueBackgroundWork() = 0; virtual Status ContinueBackgroundWork() = 0;
// This function will enable automatic compactions for the given column // This function will enable automatic compactions for the given column
// families if they were previously disabled via the disable_auto_compactions // families if they were previously disabled. The function will first set the
// option. // disable_auto_compactions option for each column family to 'false', after
// which it will schedule a flush/compaction.
//
// NOTE: Setting disable_auto_compactions to 'false' through SetOptions() API
// does NOT schedule a flush/compaction afterwards, and only changes the
// parameter itself within the column family option.
//
virtual Status EnableAutoCompaction( virtual Status EnableAutoCompaction(
const std::vector<ColumnFamilyHandle*>& column_family_handles) = 0; const std::vector<ColumnFamilyHandle*>& column_family_handles) = 0;

Loading…
Cancel
Save