Use deleters to label cache entries and collect stats (#8297)
Summary:
This change gathers and publishes statistics about the
kinds of items in block cache. This is especially important for
profiling relative usage of cache by index vs. filter vs. data blocks.
It works by iterating over the cache during periodic stats dump
(InternalStats, stats_dump_period_sec) or on demand when
DB::Get(Map)Property(kBlockCacheEntryStats), except that for
efficiency and sharing among column families, saved data from
the last scan is used when the data is not considered too old.
The new information can be seen in info LOG, for example:
Block cache LRUCache@0x7fca62229330 capacity: 95.37 MB collections: 8 last_copies: 0 last_secs: 0.00178 secs_since: 0
Block cache entry stats(count,size,portion): DataBlock(7092,28.24 MB,29.6136%) FilterBlock(215,867.90 KB,0.888728%) FilterMetaBlock(2,5.31 KB,0.00544%) IndexBlock(217,180.11 KB,0.184432%) WriteBuffer(1,256.00 KB,0.262144%) Misc(1,0.00 KB,0%)
And also through DB::GetProperty and GetMapProperty (here using
ldb just for demonstration):
$ ./ldb --db=/dev/shm/dbbench/ get_property rocksdb.block-cache-entry-stats
rocksdb.block-cache-entry-stats.bytes.data-block: 0
rocksdb.block-cache-entry-stats.bytes.deprecated-filter-block: 0
rocksdb.block-cache-entry-stats.bytes.filter-block: 0
rocksdb.block-cache-entry-stats.bytes.filter-meta-block: 0
rocksdb.block-cache-entry-stats.bytes.index-block: 178992
rocksdb.block-cache-entry-stats.bytes.misc: 0
rocksdb.block-cache-entry-stats.bytes.other-block: 0
rocksdb.block-cache-entry-stats.bytes.write-buffer: 0
rocksdb.block-cache-entry-stats.capacity: 8388608
rocksdb.block-cache-entry-stats.count.data-block: 0
rocksdb.block-cache-entry-stats.count.deprecated-filter-block: 0
rocksdb.block-cache-entry-stats.count.filter-block: 0
rocksdb.block-cache-entry-stats.count.filter-meta-block: 0
rocksdb.block-cache-entry-stats.count.index-block: 215
rocksdb.block-cache-entry-stats.count.misc: 1
rocksdb.block-cache-entry-stats.count.other-block: 0
rocksdb.block-cache-entry-stats.count.write-buffer: 0
rocksdb.block-cache-entry-stats.id: LRUCache@0x7f3636661290
rocksdb.block-cache-entry-stats.percent.data-block: 0.000000
rocksdb.block-cache-entry-stats.percent.deprecated-filter-block: 0.000000
rocksdb.block-cache-entry-stats.percent.filter-block: 0.000000
rocksdb.block-cache-entry-stats.percent.filter-meta-block: 0.000000
rocksdb.block-cache-entry-stats.percent.index-block: 2.133751
rocksdb.block-cache-entry-stats.percent.misc: 0.000000
rocksdb.block-cache-entry-stats.percent.other-block: 0.000000
rocksdb.block-cache-entry-stats.percent.write-buffer: 0.000000
rocksdb.block-cache-entry-stats.secs_for_last_collection: 0.000052
rocksdb.block-cache-entry-stats.secs_since_last_collection: 0
Solution detail - We need some way to flag what kind of blocks each
entry belongs to, preferably without changing the Cache API.
One of the complications is that Cache is a general interface that could
have other users that don't adhere to whichever convention we decide
on for keys and values. Or we would pay for an extra field in the Handle
that would only be used for this purpose.
This change uses a back-door approach, the deleter, to indicate the
"role" of a Cache entry (in addition to the value type, implicitly).
This has the added benefit of ensuring proper code origin whenever we
recognize a particular role for a cache entry; if the entry came from
some other part of the code, it will use an unrecognized deleter, which
we simply attribute to the "Misc" role.
An internal API makes for simple instantiation and automatic
registration of Cache deleters for a given value type and "role".
Another internal API, CacheEntryStatsCollector, solves the problem of
caching the results of a scan and sharing them, to ensure scans are
neither excessive nor redundant so as not to harm Cache performance.
Because code is added to BlocklikeTraits, it is pulled out of
block_based_table_reader.cc into its own file.
This is a reformulation of https://github.com/facebook/rocksdb/issues/8276, without the type checking option
(could still be added), and with actual stat gathering.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/8297
Test Plan: manual testing with db_bench, and a couple of basic unit tests
Reviewed By: ltamasi
Differential Revision: D28488721
Pulled By: pdillinger
fbshipit-source-id: 472f524a9691b5afb107934be2d41d84f2b129fb
4 years ago
|
|
|
// Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved.
|
|
|
|
// This source code is licensed under both the GPLv2 (found in the
|
|
|
|
// COPYING file in the root directory) and Apache 2.0 License
|
|
|
|
// (found in the LICENSE.Apache file in the root directory).
|
|
|
|
|
|
|
|
#pragma once
|
|
|
|
|
|
|
|
#include <array>
|
|
|
|
#include <cstdint>
|
|
|
|
#include <unordered_map>
|
|
|
|
|
|
|
|
#include "rocksdb/cache.h"
|
|
|
|
|
|
|
|
namespace ROCKSDB_NAMESPACE {
|
|
|
|
|
|
|
|
// Classifications of block cache entries, for reporting statistics
|
|
|
|
enum class CacheEntryRole {
|
|
|
|
// Block-based table data block
|
|
|
|
kDataBlock,
|
|
|
|
// Block-based table filter block (full or partitioned)
|
|
|
|
kFilterBlock,
|
|
|
|
// Block-based table metadata block for partitioned filter
|
|
|
|
kFilterMetaBlock,
|
|
|
|
// Block-based table deprecated filter block (old "block-based" filter)
|
|
|
|
kDeprecatedFilterBlock,
|
|
|
|
// Block-based table index block
|
|
|
|
kIndexBlock,
|
|
|
|
// Other kinds of block-based table block
|
|
|
|
kOtherBlock,
|
|
|
|
// WriteBufferManager reservations to account for memtable usage
|
Use deleters to label cache entries and collect stats (#8297)
Summary:
This change gathers and publishes statistics about the
kinds of items in block cache. This is especially important for
profiling relative usage of cache by index vs. filter vs. data blocks.
It works by iterating over the cache during periodic stats dump
(InternalStats, stats_dump_period_sec) or on demand when
DB::Get(Map)Property(kBlockCacheEntryStats), except that for
efficiency and sharing among column families, saved data from
the last scan is used when the data is not considered too old.
The new information can be seen in info LOG, for example:
Block cache LRUCache@0x7fca62229330 capacity: 95.37 MB collections: 8 last_copies: 0 last_secs: 0.00178 secs_since: 0
Block cache entry stats(count,size,portion): DataBlock(7092,28.24 MB,29.6136%) FilterBlock(215,867.90 KB,0.888728%) FilterMetaBlock(2,5.31 KB,0.00544%) IndexBlock(217,180.11 KB,0.184432%) WriteBuffer(1,256.00 KB,0.262144%) Misc(1,0.00 KB,0%)
And also through DB::GetProperty and GetMapProperty (here using
ldb just for demonstration):
$ ./ldb --db=/dev/shm/dbbench/ get_property rocksdb.block-cache-entry-stats
rocksdb.block-cache-entry-stats.bytes.data-block: 0
rocksdb.block-cache-entry-stats.bytes.deprecated-filter-block: 0
rocksdb.block-cache-entry-stats.bytes.filter-block: 0
rocksdb.block-cache-entry-stats.bytes.filter-meta-block: 0
rocksdb.block-cache-entry-stats.bytes.index-block: 178992
rocksdb.block-cache-entry-stats.bytes.misc: 0
rocksdb.block-cache-entry-stats.bytes.other-block: 0
rocksdb.block-cache-entry-stats.bytes.write-buffer: 0
rocksdb.block-cache-entry-stats.capacity: 8388608
rocksdb.block-cache-entry-stats.count.data-block: 0
rocksdb.block-cache-entry-stats.count.deprecated-filter-block: 0
rocksdb.block-cache-entry-stats.count.filter-block: 0
rocksdb.block-cache-entry-stats.count.filter-meta-block: 0
rocksdb.block-cache-entry-stats.count.index-block: 215
rocksdb.block-cache-entry-stats.count.misc: 1
rocksdb.block-cache-entry-stats.count.other-block: 0
rocksdb.block-cache-entry-stats.count.write-buffer: 0
rocksdb.block-cache-entry-stats.id: LRUCache@0x7f3636661290
rocksdb.block-cache-entry-stats.percent.data-block: 0.000000
rocksdb.block-cache-entry-stats.percent.deprecated-filter-block: 0.000000
rocksdb.block-cache-entry-stats.percent.filter-block: 0.000000
rocksdb.block-cache-entry-stats.percent.filter-meta-block: 0.000000
rocksdb.block-cache-entry-stats.percent.index-block: 2.133751
rocksdb.block-cache-entry-stats.percent.misc: 0.000000
rocksdb.block-cache-entry-stats.percent.other-block: 0.000000
rocksdb.block-cache-entry-stats.percent.write-buffer: 0.000000
rocksdb.block-cache-entry-stats.secs_for_last_collection: 0.000052
rocksdb.block-cache-entry-stats.secs_since_last_collection: 0
Solution detail - We need some way to flag what kind of blocks each
entry belongs to, preferably without changing the Cache API.
One of the complications is that Cache is a general interface that could
have other users that don't adhere to whichever convention we decide
on for keys and values. Or we would pay for an extra field in the Handle
that would only be used for this purpose.
This change uses a back-door approach, the deleter, to indicate the
"role" of a Cache entry (in addition to the value type, implicitly).
This has the added benefit of ensuring proper code origin whenever we
recognize a particular role for a cache entry; if the entry came from
some other part of the code, it will use an unrecognized deleter, which
we simply attribute to the "Misc" role.
An internal API makes for simple instantiation and automatic
registration of Cache deleters for a given value type and "role".
Another internal API, CacheEntryStatsCollector, solves the problem of
caching the results of a scan and sharing them, to ensure scans are
neither excessive nor redundant so as not to harm Cache performance.
Because code is added to BlocklikeTraits, it is pulled out of
block_based_table_reader.cc into its own file.
This is a reformulation of https://github.com/facebook/rocksdb/issues/8276, without the type checking option
(could still be added), and with actual stat gathering.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/8297
Test Plan: manual testing with db_bench, and a couple of basic unit tests
Reviewed By: ltamasi
Differential Revision: D28488721
Pulled By: pdillinger
fbshipit-source-id: 472f524a9691b5afb107934be2d41d84f2b129fb
4 years ago
|
|
|
kWriteBuffer,
|
|
|
|
// Default bucket, for miscellaneous cache entries. Do not use for
|
|
|
|
// entries that could potentially add up to large usage.
|
|
|
|
kMisc,
|
|
|
|
};
|
|
|
|
constexpr uint32_t kNumCacheEntryRoles =
|
|
|
|
static_cast<uint32_t>(CacheEntryRole::kMisc) + 1;
|
|
|
|
|
|
|
|
extern std::array<const char*, kNumCacheEntryRoles>
|
|
|
|
kCacheEntryRoleToCamelString;
|
|
|
|
extern std::array<const char*, kNumCacheEntryRoles>
|
|
|
|
kCacheEntryRoleToHyphenString;
|
|
|
|
|
|
|
|
// To associate cache entries with their role, we use a hack on the
|
|
|
|
// existing Cache interface. Because the deleter of an entry can authenticate
|
|
|
|
// the code origin of an entry, we can elaborate the choice of deleter to
|
|
|
|
// also encode role information, without inferring false role information
|
|
|
|
// from entries not choosing to encode a role.
|
|
|
|
//
|
|
|
|
// The rest of this file is for handling mappings between deleters and
|
|
|
|
// roles.
|
|
|
|
|
|
|
|
// To infer a role from a deleter, the deleter must be registered. This
|
|
|
|
// can be done "manually" with this function. This function is thread-safe,
|
|
|
|
// and the registration mappings go into private but static storage. (Note
|
|
|
|
// that DeleterFn is a function pointer, not std::function. Registrations
|
|
|
|
// should not be too many.)
|
|
|
|
void RegisterCacheDeleterRole(Cache::DeleterFn fn, CacheEntryRole role);
|
|
|
|
|
|
|
|
// Gets a copy of the registered deleter -> role mappings. This is the only
|
|
|
|
// function for reading the mappings made with RegisterCacheDeleterRole.
|
|
|
|
// Why only this interface for reading?
|
|
|
|
// * This function has to be thread safe, which could incur substantial
|
|
|
|
// overhead. We should not pay this overhead for every deleter look-up.
|
|
|
|
// * This is suitable for preparing for batch operations, like with
|
|
|
|
// CacheEntryStatsCollector.
|
|
|
|
// * The number of mappings should be sufficiently small (dozens).
|
|
|
|
std::unordered_map<Cache::DeleterFn, CacheEntryRole> CopyCacheDeleterRoleMap();
|
|
|
|
|
|
|
|
// ************************************************************** //
|
|
|
|
// An automatic registration infrastructure. This enables code
|
|
|
|
// to simply ask for a deleter associated with a particular type
|
|
|
|
// and role, and registration is automatic. In a sense, this is
|
|
|
|
// a small dependency injection infrastructure, because linking
|
|
|
|
// in new deleter instantiations is essentially sufficient for
|
|
|
|
// making stats collection (using CopyCacheDeleterRoleMap) aware
|
|
|
|
// of them.
|
|
|
|
|
|
|
|
namespace cache_entry_roles_detail {
|
|
|
|
|
|
|
|
template <typename T, CacheEntryRole R>
|
|
|
|
struct RegisteredDeleter {
|
|
|
|
RegisteredDeleter() { RegisterCacheDeleterRole(Delete, R); }
|
|
|
|
|
|
|
|
// These have global linkage to help ensure compiler optimizations do not
|
|
|
|
// break uniqueness for each <T,R>
|
|
|
|
static void Delete(const Slice& /* key */, void* value) {
|
|
|
|
delete static_cast<T*>(value);
|
|
|
|
}
|
|
|
|
};
|
|
|
|
|
|
|
|
template <CacheEntryRole R>
|
|
|
|
struct RegisteredNoopDeleter {
|
|
|
|
RegisteredNoopDeleter() { RegisterCacheDeleterRole(Delete, R); }
|
|
|
|
|
|
|
|
static void Delete(const Slice& /* key */, void* value) {
|
|
|
|
(void)value;
|
|
|
|
assert(value == nullptr);
|
|
|
|
}
|
|
|
|
};
|
|
|
|
|
|
|
|
} // namespace cache_entry_roles_detail
|
|
|
|
|
|
|
|
// Get an automatically registered deleter for value type T and role R.
|
|
|
|
// Based on C++ semantics, registration is invoked exactly once in a
|
|
|
|
// thread-safe way on first call to this function, for each <T, R>.
|
|
|
|
template <typename T, CacheEntryRole R>
|
|
|
|
Cache::DeleterFn GetCacheEntryDeleterForRole() {
|
|
|
|
static cache_entry_roles_detail::RegisteredDeleter<T, R> reg;
|
|
|
|
return reg.Delete;
|
|
|
|
}
|
|
|
|
|
|
|
|
// Get an automatically registered no-op deleter (value should be nullptr)
|
|
|
|
// and associated with role R. This is used for Cache "reservation" entries
|
|
|
|
// such as for WriteBufferManager.
|
|
|
|
template <CacheEntryRole R>
|
|
|
|
Cache::DeleterFn GetNoopDeleterForRole() {
|
|
|
|
static cache_entry_roles_detail::RegisteredNoopDeleter<R> reg;
|
|
|
|
return reg.Delete;
|
|
|
|
}
|
|
|
|
|
|
|
|
} // namespace ROCKSDB_NAMESPACE
|