Fix bug in Logger creation if dbname and db_log_dir are on different filesystem (#10292)

Summary:
If dbname and db_log_dir are at different filesystems (one
local and one remote), creation of dbname will fail because that path
doesn't exist wrt to db_log_dir.
This patch will ignore the error returned on creation of dbname. If they
are on same filesystem, db_log_dir creation will automatically return
the error in case there is any error in creation of dbname.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/10292

Test Plan: Existing unit tests

Reviewed By: riversand963

Differential Revision: D37567773

Pulled By: akankshamahajan15

fbshipit-source-id: 005d28c536208d4c126c8cb8e196d1d85b881100
main
Akanksha Mahajan 2 years ago committed by Facebook GitHub Bot
parent 4428c76181
commit 11215e0f3a
  1. 1
      HISTORY.md
  2. 12
      logging/auto_roll_logger.cc

@ -15,6 +15,7 @@
### Bug Fixes ### Bug Fixes
* Fix a bug in which backup/checkpoint can include a WAL deleted by RocksDB. * Fix a bug in which backup/checkpoint can include a WAL deleted by RocksDB.
* Fix a bug where concurrent compactions might cause unnecessary further write stalling. In some cases, this might cause write rate to drop to minimum. * Fix a bug where concurrent compactions might cause unnecessary further write stalling. In some cases, this might cause write rate to drop to minimum.
* Fix a bug in Logger where if dbname and db_log_dir are on different filesystems, dbname creation would fail wrt to db_log_dir path returning an error and fails to open the DB.
## Behavior Change ## Behavior Change
* In leveled compaction with dynamic levelling, level multiplier is not anymore adjusted due to oversized L0. Instead, compaction score is adjusted by increasing size level target by adding incoming bytes from upper levels. This would deprioritize compactions from upper levels if more data from L0 is coming. This is to fix some unnecessary full stalling due to drastic change of level targets, while not wasting write bandwidth for compaction while writes are overloaded. * In leveled compaction with dynamic levelling, level multiplier is not anymore adjusted due to oversized L0. Instead, compaction score is adjusted by increasing size level target by adding incoming bytes from upper levels. This would deprioritize compactions from upper levels if more data from L0 is coming. This is to fix some unnecessary full stalling due to drastic change of level targets, while not wasting write bandwidth for compaction while writes are overloaded.

@ -278,11 +278,21 @@ Status CreateLoggerFromOptions(const std::string& dbname,
InfoLogFileName(dbname, db_absolute_path, options.db_log_dir); InfoLogFileName(dbname, db_absolute_path, options.db_log_dir);
const auto& clock = env->GetSystemClock(); const auto& clock = env->GetSystemClock();
// In case it does not exist // In case it does not exist.
s = env->CreateDirIfMissing(dbname); s = env->CreateDirIfMissing(dbname);
if (!s.ok()) { if (!s.ok()) {
if (options.db_log_dir.empty()) {
return s; return s;
} else {
// Ignore the error returned during creation of dbname because dbname and
// db_log_dir can be on different filesystems in which case dbname will
// not exist and error should be ignored. db_log_dir creation will handle
// the error in case there is any error in the creation of dbname on same
// filesystem.
s = Status::OK();
}
} }
assert(s.ok());
if (!options.db_log_dir.empty()) { if (!options.db_log_dir.empty()) {
s = env->CreateDirIfMissing(options.db_log_dir); s = env->CreateDirIfMissing(options.db_log_dir);

Loading…
Cancel
Save