CompactFiles, EventListener and GetDatabaseMetaData
Summary:
This diff adds three sets of APIs to RocksDB.
= GetColumnFamilyMetaData =
* This APIs allow users to obtain the current state of a RocksDB instance on one column family.
* See GetColumnFamilyMetaData in include/rocksdb/db.h
= EventListener =
* A virtual class that allows users to implement a set of
call-back functions which will be called when specific
events of a RocksDB instance happens.
* To register EventListener, simply insert an EventListener to ColumnFamilyOptions::listeners
= CompactFiles =
* CompactFiles API inputs a set of file numbers and an output level, and RocksDB
will try to compact those files into the specified level.
= Example =
* Example code can be found in example/compact_files_example.cc, which implements
a simple external compactor using EventListener, GetColumnFamilyMetaData, and
CompactFiles API.
Test Plan:
listener_test
compactor_test
example/compact_files_example
export ROCKSDB_TESTS=CompactFiles
db_test
export ROCKSDB_TESTS=MetaData
db_test
Reviewers: ljin, igor, rven, sdong
Reviewed By: sdong
Subscribers: MarkCallaghan, dhruba, leveldb
Differential Revision: https://reviews.facebook.net/D24705
10 years ago
|
|
|
// Copyright (c) 2014, Facebook, Inc. All rights reserved.
|
|
|
|
// This source code is licensed under the BSD-style license found in the
|
|
|
|
// LICENSE file in the root directory of this source tree. An additional grant
|
|
|
|
// of patent rights can be found in the PATENTS file in the same directory.
|
|
|
|
|
|
|
|
#pragma once
|
|
|
|
|
|
|
|
#include <stdint.h>
|
|
|
|
|
CompactFiles, EventListener and GetDatabaseMetaData
Summary:
This diff adds three sets of APIs to RocksDB.
= GetColumnFamilyMetaData =
* This APIs allow users to obtain the current state of a RocksDB instance on one column family.
* See GetColumnFamilyMetaData in include/rocksdb/db.h
= EventListener =
* A virtual class that allows users to implement a set of
call-back functions which will be called when specific
events of a RocksDB instance happens.
* To register EventListener, simply insert an EventListener to ColumnFamilyOptions::listeners
= CompactFiles =
* CompactFiles API inputs a set of file numbers and an output level, and RocksDB
will try to compact those files into the specified level.
= Example =
* Example code can be found in example/compact_files_example.cc, which implements
a simple external compactor using EventListener, GetColumnFamilyMetaData, and
CompactFiles API.
Test Plan:
listener_test
compactor_test
example/compact_files_example
export ROCKSDB_TESTS=CompactFiles
db_test
export ROCKSDB_TESTS=MetaData
db_test
Reviewers: ljin, igor, rven, sdong
Reviewed By: sdong
Subscribers: MarkCallaghan, dhruba, leveldb
Differential Revision: https://reviews.facebook.net/D24705
10 years ago
|
|
|
#include <limits>
|
|
|
|
#include <string>
|
|
|
|
#include <vector>
|
|
|
|
|
|
|
|
#include "rocksdb/types.h"
|
|
|
|
|
|
|
|
namespace rocksdb {
|
|
|
|
struct ColumnFamilyMetaData;
|
|
|
|
struct LevelMetaData;
|
|
|
|
struct SstFileMetaData;
|
|
|
|
|
|
|
|
// The metadata that describes a column family.
|
|
|
|
struct ColumnFamilyMetaData {
|
|
|
|
ColumnFamilyMetaData() : size(0), name("") {}
|
|
|
|
ColumnFamilyMetaData(const std::string& _name, uint64_t _size,
|
|
|
|
const std::vector<LevelMetaData>&& _levels) :
|
|
|
|
size(_size), name(_name), levels(_levels) {}
|
CompactFiles, EventListener and GetDatabaseMetaData
Summary:
This diff adds three sets of APIs to RocksDB.
= GetColumnFamilyMetaData =
* This APIs allow users to obtain the current state of a RocksDB instance on one column family.
* See GetColumnFamilyMetaData in include/rocksdb/db.h
= EventListener =
* A virtual class that allows users to implement a set of
call-back functions which will be called when specific
events of a RocksDB instance happens.
* To register EventListener, simply insert an EventListener to ColumnFamilyOptions::listeners
= CompactFiles =
* CompactFiles API inputs a set of file numbers and an output level, and RocksDB
will try to compact those files into the specified level.
= Example =
* Example code can be found in example/compact_files_example.cc, which implements
a simple external compactor using EventListener, GetColumnFamilyMetaData, and
CompactFiles API.
Test Plan:
listener_test
compactor_test
example/compact_files_example
export ROCKSDB_TESTS=CompactFiles
db_test
export ROCKSDB_TESTS=MetaData
db_test
Reviewers: ljin, igor, rven, sdong
Reviewed By: sdong
Subscribers: MarkCallaghan, dhruba, leveldb
Differential Revision: https://reviews.facebook.net/D24705
10 years ago
|
|
|
|
|
|
|
// The size of this column family in bytes, which is equal to the sum of
|
|
|
|
// the file size of its "levels".
|
|
|
|
uint64_t size;
|
|
|
|
// The number of files in this column family.
|
|
|
|
size_t file_count;
|
|
|
|
// The name of the column family.
|
|
|
|
std::string name;
|
|
|
|
// The metadata of all levels in this column family.
|
|
|
|
std::vector<LevelMetaData> levels;
|
|
|
|
};
|
|
|
|
|
|
|
|
// The metadata that describes a level.
|
|
|
|
struct LevelMetaData {
|
|
|
|
LevelMetaData(int _level, uint64_t _size,
|
|
|
|
const std::vector<SstFileMetaData>&& _files) :
|
|
|
|
level(_level), size(_size),
|
|
|
|
files(_files) {}
|
CompactFiles, EventListener and GetDatabaseMetaData
Summary:
This diff adds three sets of APIs to RocksDB.
= GetColumnFamilyMetaData =
* This APIs allow users to obtain the current state of a RocksDB instance on one column family.
* See GetColumnFamilyMetaData in include/rocksdb/db.h
= EventListener =
* A virtual class that allows users to implement a set of
call-back functions which will be called when specific
events of a RocksDB instance happens.
* To register EventListener, simply insert an EventListener to ColumnFamilyOptions::listeners
= CompactFiles =
* CompactFiles API inputs a set of file numbers and an output level, and RocksDB
will try to compact those files into the specified level.
= Example =
* Example code can be found in example/compact_files_example.cc, which implements
a simple external compactor using EventListener, GetColumnFamilyMetaData, and
CompactFiles API.
Test Plan:
listener_test
compactor_test
example/compact_files_example
export ROCKSDB_TESTS=CompactFiles
db_test
export ROCKSDB_TESTS=MetaData
db_test
Reviewers: ljin, igor, rven, sdong
Reviewed By: sdong
Subscribers: MarkCallaghan, dhruba, leveldb
Differential Revision: https://reviews.facebook.net/D24705
10 years ago
|
|
|
|
|
|
|
// The level which this meta data describes.
|
|
|
|
const int level;
|
|
|
|
// The size of this level in bytes, which is equal to the sum of
|
|
|
|
// the file size of its "files".
|
|
|
|
const uint64_t size;
|
|
|
|
// The metadata of all sst files in this level.
|
|
|
|
const std::vector<SstFileMetaData> files;
|
|
|
|
};
|
|
|
|
|
|
|
|
// The metadata that describes a SST file.
|
|
|
|
struct SstFileMetaData {
|
|
|
|
SstFileMetaData() {}
|
|
|
|
SstFileMetaData(const std::string& _file_name,
|
|
|
|
const std::string& _path, uint64_t _size,
|
|
|
|
SequenceNumber _smallest_seqno,
|
|
|
|
SequenceNumber _largest_seqno,
|
|
|
|
const std::string& _smallestkey,
|
|
|
|
const std::string& _largestkey,
|
|
|
|
bool _being_compacted) :
|
|
|
|
size(_size), name(_file_name),
|
|
|
|
db_path(_path), smallest_seqno(_smallest_seqno), largest_seqno(_largest_seqno),
|
|
|
|
smallestkey(_smallestkey), largestkey(_largestkey),
|
|
|
|
being_compacted(_being_compacted) {}
|
CompactFiles, EventListener and GetDatabaseMetaData
Summary:
This diff adds three sets of APIs to RocksDB.
= GetColumnFamilyMetaData =
* This APIs allow users to obtain the current state of a RocksDB instance on one column family.
* See GetColumnFamilyMetaData in include/rocksdb/db.h
= EventListener =
* A virtual class that allows users to implement a set of
call-back functions which will be called when specific
events of a RocksDB instance happens.
* To register EventListener, simply insert an EventListener to ColumnFamilyOptions::listeners
= CompactFiles =
* CompactFiles API inputs a set of file numbers and an output level, and RocksDB
will try to compact those files into the specified level.
= Example =
* Example code can be found in example/compact_files_example.cc, which implements
a simple external compactor using EventListener, GetColumnFamilyMetaData, and
CompactFiles API.
Test Plan:
listener_test
compactor_test
example/compact_files_example
export ROCKSDB_TESTS=CompactFiles
db_test
export ROCKSDB_TESTS=MetaData
db_test
Reviewers: ljin, igor, rven, sdong
Reviewed By: sdong
Subscribers: MarkCallaghan, dhruba, leveldb
Differential Revision: https://reviews.facebook.net/D24705
10 years ago
|
|
|
|
|
|
|
// File size in bytes.
|
|
|
|
uint64_t size;
|
|
|
|
// The name of the file.
|
|
|
|
std::string name;
|
|
|
|
// The full path where the file locates.
|
|
|
|
std::string db_path;
|
|
|
|
|
|
|
|
SequenceNumber smallest_seqno; // Smallest sequence number in file.
|
|
|
|
SequenceNumber largest_seqno; // Largest sequence number in file.
|
|
|
|
std::string smallestkey; // Smallest user defined key in the file.
|
|
|
|
std::string largestkey; // Largest user defined key in the file.
|
|
|
|
bool being_compacted; // true if the file is currently being compacted.
|
|
|
|
};
|
|
|
|
|
|
|
|
// The full set of metadata associated with each SST file.
|
|
|
|
struct LiveFileMetaData : SstFileMetaData {
|
|
|
|
std::string column_family_name; // Name of the column family
|
|
|
|
int level; // Level at which this file resides.
|
|
|
|
};
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
} // namespace rocksdb
|