NextGraph
/
rocksdb
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Use -Wno-invalid-offsetof instead of dangerous offset_of hack (#9563)

Browse Source 
Summary:
After https://github.com/facebook/rocksdb/issues/9515 added a unique_ptr to Status, we see some
warnings-as-error in some internal builds like this:

```
stderr: rocksdb/src/db/compaction/compaction_job.cc:2839:7: error:
offset of on non-standard-layout type 'struct CompactionServiceResult'
[-Werror,-Winvalid-offsetof]
     {offsetof(struct CompactionServiceResult, status),
      ^                                        ~~~~~~
```

I see three potential solutions to resolving this:

* Expand our use of an idiom that works around the warning (see offset_of
functions removed in this change, inspired by
https://gist.github.com/graphitemaster/494f21190bb2c63c5516)  However,
this construction is invoking undefined behavior that assumes consistent
layout with no compiler-introduced indirection. A compiler incompatible
with our assumptions will likely compile the code and exhibit undefined
behavior.
* Migrate to something in place of offset, like a function mapping
CompactionServiceResult* to Status* (for the `status` field). This might
be required in the long term.
* **Selected:** Use our new C++17 dependency to use offsetof in a well-defined way
when the compiler allows it. From a comment on
https://gist.github.com/graphitemaster/494f21190bb2c63c5516:

> A final note: in C++17, offsetof is conditionally supported, which
> means that you can use it on any type (not just standard layout
> types) and the compiler will error if it can't compile it correctly.
> That appears to be the best option if you can live with C++17 and
> don't need constexpr support.

The C++17 semantics are confirmed on
https://en.cppreference.com/w/cpp/types/offsetof, so we can suppress the
warning as long as we accept that we might run into a compiler that
rejects the code, and at that point we will find a solution, such as
the more intrusive "migrate" solution above.

Although this is currently only showing in our buck build, it will
surely show up also with make and cmake, so I have updated those
configurations as well.

Also in the buck build, -Wno-expansion-to-defined does not appear to be
needed anymore (both current compiler configurations) so I
removed it.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/9563

Test Plan: Tried out buck builds with both current compiler configurations

Reviewed By: riversand963

Differential Revision: D34220931

Pulled By: pdillinger

fbshipit-source-id: d39436008259bd1eaaa87c77be69fb2a5b559e1f
main
Peter Dillinger 3 years ago committed by Facebook GitHub Bot
parent 241b5aa15a
commit e24734f843
8 changed files with 104 additions and 132 deletions
Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL
Show Stats Download Patch File Download Diff File
  1. 2
      CMakeLists.txt
  2. 5
      Makefile
  3. 6
      TARGETS
  4. 6
      buckifier/targets_cfg.py
  5. 63
      db/compaction/compaction_job.cc
  6. 85
      options/cf_options.cc
  7. 17
      options/customizable_test.cc
  8. 52
      options/options_settable_test.cc

2
CMakeLists.txt
Unescape View File

@ -219,7 +219,7 @@ if(MSVC)
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} /FC /d2Zi+ /W4 /wd4127 /wd4800 /wd4996 /wd4351 /wd4100 /wd4204 /wd4324") set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} /FC /d2Zi+ /W4 /wd4127 /wd4800 /wd4996 /wd4351 /wd4100 /wd4204 /wd4324")
else() else()
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -W -Wextra -Wall -pthread") set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -W -Wextra -Wall -pthread")
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -Wsign-compare -Wshadow -Wno-unused-parameter -Wno-unused-variable -Woverloaded-virtual -Wnon-virtual-dtor -Wno-missing-field-initializers -Wno-strict-aliasing") set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -Wsign-compare -Wshadow -Wno-unused-parameter -Wno-unused-variable -Woverloaded-virtual -Wnon-virtual-dtor -Wno-missing-field-initializers -Wno-strict-aliasing -Wno-invalid-offsetof")
if(CMAKE_SYSTEM_PROCESSOR MATCHES "x86_64") if(CMAKE_SYSTEM_PROCESSOR MATCHES "x86_64")
set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -Wstrict-prototypes") set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -Wstrict-prototypes")
endif() endif()

5
Makefile
Unescape View File

@ -501,6 +501,11 @@ endif
CFLAGS += $(C_WARNING_FLAGS) $(WARNING_FLAGS) -I. -I./include $(PLATFORM_CCFLAGS) $(OPT) CFLAGS += $(C_WARNING_FLAGS) $(WARNING_FLAGS) -I. -I./include $(PLATFORM_CCFLAGS) $(OPT)
CXXFLAGS += $(WARNING_FLAGS) -I. -I./include $(PLATFORM_CXXFLAGS) $(OPT) -Woverloaded-virtual -Wnon-virtual-dtor -Wno-missing-field-initializers CXXFLAGS += $(WARNING_FLAGS) -I. -I./include $(PLATFORM_CXXFLAGS) $(OPT) -Woverloaded-virtual -Wnon-virtual-dtor -Wno-missing-field-initializers
# Allow offsetof to work on non-standard layout types. Some compiler could
# completely reject our usage of offsetof, but we will solve that when it
# happens.
CXXFLAGS += -Wno-invalid-offsetof
LDFLAGS += $(PLATFORM_LDFLAGS) LDFLAGS += $(PLATFORM_LDFLAGS)
LIB_OBJECTS = $(patsubst %.cc, $(OBJ_DIR)/%.o, $(LIB_SOURCES)) LIB_OBJECTS = $(patsubst %.cc, $(OBJ_DIR)/%.o, $(LIB_SOURCES))

6
TARGETS
Unescape View File

@ -14,8 +14,10 @@ REPO_PATH = package_name() + "/"
ROCKSDB_COMPILER_FLAGS_0 = [ ROCKSDB_COMPILER_FLAGS_0 = [
"-fno-builtin-memcmp", "-fno-builtin-memcmp",
# Needed to compile in fbcode # Allow offsetof to work on non-standard layout types. Some compiler could
"-Wno-expansion-to-defined", # completely reject our usage of offsetof, but we will solve that when it
# happens.
"-Wno-invalid-offsetof",
# Added missing flags from output of build_detect_platform # Added missing flags from output of build_detect_platform
"-Wnarrowing", "-Wnarrowing",
"-DROCKSDB_NO_DYNAMIC_EXTENSION", "-DROCKSDB_NO_DYNAMIC_EXTENSION",

6
buckifier/targets_cfg.py
Unescape View File

@ -21,8 +21,10 @@ REPO_PATH = package_name() + "/"
ROCKSDB_COMPILER_FLAGS_0 = [ ROCKSDB_COMPILER_FLAGS_0 = [
"-fno-builtin-memcmp", "-fno-builtin-memcmp",
# Needed to compile in fbcode # Allow offsetof to work on non-standard layout types. Some compiler could
"-Wno-expansion-to-defined", # completely reject our usage of offsetof, but we will solve that when it
# happens.
"-Wno-invalid-offsetof",
# Added missing flags from output of build_detect_platform # Added missing flags from output of build_detect_platform
"-Wnarrowing", "-Wnarrowing",
"-DROCKSDB_NO_DYNAMIC_EXTENSION", "-DROCKSDB_NO_DYNAMIC_EXTENSION",

63
db/compaction/compaction_job.cc
Unescape View File

@ -2506,34 +2506,14 @@ enum BinaryFormatVersion : uint32_t {
kOptionsString = 1, // Use string format similar to Option string format kOptionsString = 1, // Use string format similar to Option string format
}; };
// offset_of is used to get the offset of a class data member
// ex: offset_of(&ColumnFamilyDescriptor::options)
// This call will return the offset of options in ColumnFamilyDescriptor class
//
// This is the same as offsetof() but allow us to work with non standard-layout
// classes and structures
// refs:
// http://en.cppreference.com/w/cpp/concept/StandardLayoutType
// https://gist.github.com/graphitemaster/494f21190bb2c63c5516
static ColumnFamilyDescriptor dummy_cfd("", ColumnFamilyOptions());
template <typename T1>
int offset_of(T1 ColumnFamilyDescriptor::*member) {
return int(size_t(&(dummy_cfd.*member)) - size_t(&dummy_cfd));
}
static CompactionServiceInput dummy_cs_input;
template <typename T1>
int offset_of(T1 CompactionServiceInput::*member) {
return int(size_t(&(dummy_cs_input.*member)) - size_t(&dummy_cs_input));
}
static std::unordered_map<std::string, OptionTypeInfo> cfd_type_info = { static std::unordered_map<std::string, OptionTypeInfo> cfd_type_info = {
{"name", {"name",
{offset_of(&ColumnFamilyDescriptor::name), OptionType::kEncodedString, {offsetof(struct ColumnFamilyDescriptor, name), OptionType::kEncodedString,
OptionVerificationType::kNormal, OptionTypeFlags::kNone}}, OptionVerificationType::kNormal, OptionTypeFlags::kNone}},
{"options", {"options",
{offset_of(&ColumnFamilyDescriptor::options), OptionType::kConfigurable, {offsetof(struct ColumnFamilyDescriptor, options),
OptionVerificationType::kNormal, OptionTypeFlags::kNone, OptionType::kConfigurable, OptionVerificationType::kNormal,
OptionTypeFlags::kNone,
[](const ConfigOptions& opts, const std::string& /*name*/, [](const ConfigOptions& opts, const std::string& /*name*/,
const std::string& value, void* addr) { const std::string& value, void* addr) {
auto cf_options = static_cast<ColumnFamilyOptions*>(addr); auto cf_options = static_cast<ColumnFamilyOptions*>(addr);
@ -2567,13 +2547,14 @@ static std::unordered_map<std::string, OptionTypeInfo> cfd_type_info = {
static std::unordered_map<std::string, OptionTypeInfo> cs_input_type_info = { static std::unordered_map<std::string, OptionTypeInfo> cs_input_type_info = {
{"column_family", {"column_family",
OptionTypeInfo::Struct("column_family", &cfd_type_info, OptionTypeInfo::Struct(
offset_of(&CompactionServiceInput::column_family), "column_family", &cfd_type_info,
OptionVerificationType::kNormal, offsetof(struct CompactionServiceInput, column_family),
OptionTypeFlags::kNone)}, OptionVerificationType::kNormal, OptionTypeFlags::kNone)},
{"db_options", {"db_options",
{offset_of(&CompactionServiceInput::db_options), OptionType::kConfigurable, {offsetof(struct CompactionServiceInput, db_options),
OptionVerificationType::kNormal, OptionTypeFlags::kNone, OptionType::kConfigurable, OptionVerificationType::kNormal,
OptionTypeFlags::kNone,
[](const ConfigOptions& opts, const std::string& /*name*/, [](const ConfigOptions& opts, const std::string& /*name*/,
const std::string& value, void* addr) { const std::string& value, void* addr) {
auto options = static_cast<DBOptions*>(addr); auto options = static_cast<DBOptions*>(addr);
@ -2602,31 +2583,33 @@ static std::unordered_map<std::string, OptionTypeInfo> cs_input_type_info = {
return result; return result;
}}}, }}},
{"snapshots", OptionTypeInfo::Vector<uint64_t>( {"snapshots", OptionTypeInfo::Vector<uint64_t>(
offset_of(&CompactionServiceInput::snapshots), offsetof(struct CompactionServiceInput, snapshots),
OptionVerificationType::kNormal, OptionTypeFlags::kNone, OptionVerificationType::kNormal, OptionTypeFlags::kNone,
{0, OptionType::kUInt64T})}, {0, OptionType::kUInt64T})},
{"input_files", OptionTypeInfo::Vector<std::string>( {"input_files", OptionTypeInfo::Vector<std::string>(
offset_of(&CompactionServiceInput::input_files), offsetof(struct CompactionServiceInput, input_files),
OptionVerificationType::kNormal, OptionTypeFlags::kNone, OptionVerificationType::kNormal, OptionTypeFlags::kNone,
{0, OptionType::kEncodedString})}, {0, OptionType::kEncodedString})},
{"output_level", {"output_level",
{offset_of(&CompactionServiceInput::output_level), OptionType::kInt, {offsetof(struct CompactionServiceInput, output_level), OptionType::kInt,
OptionVerificationType::kNormal, OptionTypeFlags::kNone}}, OptionVerificationType::kNormal, OptionTypeFlags::kNone}},
{"has_begin", {"has_begin",
{offset_of(&CompactionServiceInput::has_begin), OptionType::kBoolean, {offsetof(struct CompactionServiceInput, has_begin), OptionType::kBoolean,
OptionVerificationType::kNormal, OptionTypeFlags::kNone}}, OptionVerificationType::kNormal, OptionTypeFlags::kNone}},
{"begin", {"begin",
{offset_of(&CompactionServiceInput::begin), OptionType::kEncodedString, {offsetof(struct CompactionServiceInput, begin),
OptionVerificationType::kNormal, OptionTypeFlags::kNone}}, OptionType::kEncodedString, OptionVerificationType::kNormal,
OptionTypeFlags::kNone}},
{"has_end", {"has_end",
{offset_of(&CompactionServiceInput::has_end), OptionType::kBoolean, {offsetof(struct CompactionServiceInput, has_end), OptionType::kBoolean,
OptionVerificationType::kNormal, OptionTypeFlags::kNone}}, OptionVerificationType::kNormal, OptionTypeFlags::kNone}},
{"end", {"end",
{offset_of(&CompactionServiceInput::end), OptionType::kEncodedString, {offsetof(struct CompactionServiceInput, end), OptionType::kEncodedString,
OptionVerificationType::kNormal, OptionTypeFlags::kNone}}, OptionVerificationType::kNormal, OptionTypeFlags::kNone}},
{"approx_size", {"approx_size",
{offset_of(&CompactionServiceInput::approx_size), OptionType::kUInt64T, {offsetof(struct CompactionServiceInput, approx_size),
OptionVerificationType::kNormal, OptionTypeFlags::kNone}}, OptionType::kUInt64T, OptionVerificationType::kNormal,
OptionTypeFlags::kNone}},
}; };
static std::unordered_map<std::string, OptionTypeInfo> static std::unordered_map<std::string, OptionTypeInfo>

85
options/cf_options.cc
Unescape View File

@ -35,22 +35,8 @@
// OPTIONS files. // OPTIONS files.
namespace ROCKSDB_NAMESPACE { namespace ROCKSDB_NAMESPACE {
// offset_of is used to get the offset of a class data member
// ex: offset_of(&ColumnFamilyOptions::num_levels)
// This call will return the offset of num_levels in ColumnFamilyOptions class
//
// This is the same as offsetof() but allow us to work with non standard-layout
// classes and structures
// refs:
// http://en.cppreference.com/w/cpp/concept/StandardLayoutType
// https://gist.github.com/graphitemaster/494f21190bb2c63c5516
#ifndef ROCKSDB_LITE
static ImmutableCFOptions dummy_cf_options;
template <typename T1>
int offset_of(T1 ImmutableCFOptions::*member) {
return int(size_t(&(dummy_cf_options.*member)) - size_t(&dummy_cf_options));
}
#ifndef ROCKSDB_LITE
static Status ParseCompressionOptions(const std::string& value, static Status ParseCompressionOptions(const std::string& value,
const std::string& name, const std::string& name,
CompressionOptions& compression_opts) { CompressionOptions& compression_opts) {
@ -518,19 +504,20 @@ static std::unordered_map<std::string, OptionTypeInfo>
{0, OptionType::kBoolean, OptionVerificationType::kDeprecated, {0, OptionType::kBoolean, OptionVerificationType::kDeprecated,
OptionTypeFlags::kNone}}, OptionTypeFlags::kNone}},
{"inplace_update_support", {"inplace_update_support",
{offset_of(&ImmutableCFOptions::inplace_update_support), {offsetof(struct ImmutableCFOptions, inplace_update_support),
OptionType::kBoolean, OptionVerificationType::kNormal, OptionType::kBoolean, OptionVerificationType::kNormal,
OptionTypeFlags::kNone}}, OptionTypeFlags::kNone}},
{"level_compaction_dynamic_level_bytes", {"level_compaction_dynamic_level_bytes",
{offset_of(&ImmutableCFOptions::level_compaction_dynamic_level_bytes), {offsetof(struct ImmutableCFOptions,
level_compaction_dynamic_level_bytes),
OptionType::kBoolean, OptionVerificationType::kNormal, OptionType::kBoolean, OptionVerificationType::kNormal,
OptionTypeFlags::kNone}}, OptionTypeFlags::kNone}},
{"optimize_filters_for_hits", {"optimize_filters_for_hits",
{offset_of(&ImmutableCFOptions::optimize_filters_for_hits), {offsetof(struct ImmutableCFOptions, optimize_filters_for_hits),
OptionType::kBoolean, OptionVerificationType::kNormal, OptionType::kBoolean, OptionVerificationType::kNormal,
OptionTypeFlags::kNone}}, OptionTypeFlags::kNone}},
{"force_consistency_checks", {"force_consistency_checks",
{offset_of(&ImmutableCFOptions::force_consistency_checks), {offsetof(struct ImmutableCFOptions, force_consistency_checks),
OptionType::kBoolean, OptionVerificationType::kNormal, OptionType::kBoolean, OptionVerificationType::kNormal,
OptionTypeFlags::kNone}}, OptionTypeFlags::kNone}},
// Need to keep this around to be able to read old OPTIONS files. // Need to keep this around to be able to read old OPTIONS files.
@ -538,34 +525,37 @@ static std::unordered_map<std::string, OptionTypeInfo>
{0, OptionType::kInt, OptionVerificationType::kDeprecated, {0, OptionType::kInt, OptionVerificationType::kDeprecated,
OptionTypeFlags::kNone}}, OptionTypeFlags::kNone}},
{"max_write_buffer_number_to_maintain", {"max_write_buffer_number_to_maintain",
{offset_of(&ImmutableCFOptions::max_write_buffer_number_to_maintain), {offsetof(struct ImmutableCFOptions,
max_write_buffer_number_to_maintain),
OptionType::kInt, OptionVerificationType::kNormal, OptionType::kInt, OptionVerificationType::kNormal,
OptionTypeFlags::kNone, 0}}, OptionTypeFlags::kNone, 0}},
{"max_write_buffer_size_to_maintain", {"max_write_buffer_size_to_maintain",
{offset_of(&ImmutableCFOptions::max_write_buffer_size_to_maintain), {offsetof(struct ImmutableCFOptions,
max_write_buffer_size_to_maintain),
OptionType::kInt64T, OptionVerificationType::kNormal, OptionType::kInt64T, OptionVerificationType::kNormal,
OptionTypeFlags::kNone}}, OptionTypeFlags::kNone}},
{"min_write_buffer_number_to_merge", {"min_write_buffer_number_to_merge",
{offset_of(&ImmutableCFOptions::min_write_buffer_number_to_merge), {offsetof(struct ImmutableCFOptions, min_write_buffer_number_to_merge),
OptionType::kInt, OptionVerificationType::kNormal, OptionType::kInt, OptionVerificationType::kNormal,
OptionTypeFlags::kNone, 0}}, OptionTypeFlags::kNone, 0}},
{"num_levels", {"num_levels",
{offset_of(&ImmutableCFOptions::num_levels), OptionType::kInt, {offsetof(struct ImmutableCFOptions, num_levels), OptionType::kInt,
OptionVerificationType::kNormal, OptionTypeFlags::kNone}}, OptionVerificationType::kNormal, OptionTypeFlags::kNone}},
{"bloom_locality", {"bloom_locality",
{offset_of(&ImmutableCFOptions::bloom_locality), OptionType::kUInt32T, {offsetof(struct ImmutableCFOptions, bloom_locality),
OptionVerificationType::kNormal, OptionTypeFlags::kNone}}, OptionType::kUInt32T, OptionVerificationType::kNormal,
OptionTypeFlags::kNone}},
{"rate_limit_delay_max_milliseconds", {"rate_limit_delay_max_milliseconds",
{0, OptionType::kUInt, OptionVerificationType::kDeprecated, {0, OptionType::kUInt, OptionVerificationType::kDeprecated,
OptionTypeFlags::kNone}}, OptionTypeFlags::kNone}},
{"compression_per_level", {"compression_per_level",
OptionTypeInfo::Vector<CompressionType>( OptionTypeInfo::Vector<CompressionType>(
offset_of(&ImmutableCFOptions::compression_per_level), offsetof(struct ImmutableCFOptions, compression_per_level),
OptionVerificationType::kNormal, OptionTypeFlags::kNone, OptionVerificationType::kNormal, OptionTypeFlags::kNone,
{0, OptionType::kCompressionType})}, {0, OptionType::kCompressionType})},
{"comparator", {"comparator",
OptionTypeInfo::AsCustomRawPtr<const Comparator>( OptionTypeInfo::AsCustomRawPtr<const Comparator>(
offset_of(&ImmutableCFOptions::user_comparator), offsetof(struct ImmutableCFOptions, user_comparator),
OptionVerificationType::kByName, OptionTypeFlags::kCompareLoose, OptionVerificationType::kByName, OptionTypeFlags::kCompareLoose,
// Serializes a Comparator // Serializes a Comparator
[](const ConfigOptions& opts, const std::string&, const void* addr, [](const ConfigOptions& opts, const std::string&, const void* addr,
@ -592,11 +582,11 @@ static std::unordered_map<std::string, OptionTypeInfo>
/* Use the default match function*/ nullptr)}, /* Use the default match function*/ nullptr)},
{"memtable_insert_with_hint_prefix_extractor", {"memtable_insert_with_hint_prefix_extractor",
OptionTypeInfo::AsCustomSharedPtr<const SliceTransform>( OptionTypeInfo::AsCustomSharedPtr<const SliceTransform>(
offset_of(&ImmutableCFOptions:: offsetof(struct ImmutableCFOptions,
memtable_insert_with_hint_prefix_extractor), memtable_insert_with_hint_prefix_extractor),
OptionVerificationType::kByNameAllowNull, OptionTypeFlags::kNone)}, OptionVerificationType::kByNameAllowNull, OptionTypeFlags::kNone)},
{"memtable_factory", {"memtable_factory",
{offset_of(&ImmutableCFOptions::memtable_factory), {offsetof(struct ImmutableCFOptions, memtable_factory),
OptionType::kCustomizable, OptionVerificationType::kByName, OptionType::kCustomizable, OptionVerificationType::kByName,
OptionTypeFlags::kShared, OptionTypeFlags::kShared,
[](const ConfigOptions& opts, const std::string&, [](const ConfigOptions& opts, const std::string&,
@ -612,7 +602,7 @@ static std::unordered_map<std::string, OptionTypeInfo>
return s; return s;
}}}, }}},
{"memtable", {"memtable",
{offset_of(&ImmutableCFOptions::memtable_factory), {offsetof(struct ImmutableCFOptions, memtable_factory),
OptionType::kCustomizable, OptionVerificationType::kAlias, OptionType::kCustomizable, OptionVerificationType::kAlias,
OptionTypeFlags::kShared, OptionTypeFlags::kShared,
[](const ConfigOptions& opts, const std::string&, [](const ConfigOptions& opts, const std::string&,
@ -627,14 +617,15 @@ static std::unordered_map<std::string, OptionTypeInfo>
} }
return s; return s;
}}}, }}},
{"table_factory", OptionTypeInfo::AsCustomSharedPtr<TableFactory>( {"table_factory",
offset_of(&ImmutableCFOptions::table_factory), OptionTypeInfo::AsCustomSharedPtr<TableFactory>(
OptionVerificationType::kByName, offsetof(struct ImmutableCFOptions, table_factory),
(OptionTypeFlags::kCompareLoose | OptionVerificationType::kByName,
OptionTypeFlags::kStringNameOnly | (OptionTypeFlags::kCompareLoose |
OptionTypeFlags::kDontPrepare))}, OptionTypeFlags::kStringNameOnly |
OptionTypeFlags::kDontPrepare))},
{"block_based_table_factory", {"block_based_table_factory",
{offset_of(&ImmutableCFOptions::table_factory), {offsetof(struct ImmutableCFOptions, table_factory),
OptionType::kCustomizable, OptionVerificationType::kAlias, OptionType::kCustomizable, OptionVerificationType::kAlias,
OptionTypeFlags::kShared | OptionTypeFlags::kCompareLoose, OptionTypeFlags::kShared | OptionTypeFlags::kCompareLoose,
// Parses the input value and creates a BlockBasedTableFactory // Parses the input value and creates a BlockBasedTableFactory
@ -666,7 +657,7 @@ static std::unordered_map<std::string, OptionTypeInfo>
} }
}}}, }}},
{"plain_table_factory", {"plain_table_factory",
{offset_of(&ImmutableCFOptions::table_factory), {offsetof(struct ImmutableCFOptions, table_factory),
OptionType::kCustomizable, OptionVerificationType::kAlias, OptionType::kCustomizable, OptionVerificationType::kAlias,
OptionTypeFlags::kShared | OptionTypeFlags::kCompareLoose, OptionTypeFlags::kShared | OptionTypeFlags::kCompareLoose,
// Parses the input value and creates a PlainTableFactory // Parses the input value and creates a PlainTableFactory
@ -699,35 +690,35 @@ static std::unordered_map<std::string, OptionTypeInfo>
{"table_properties_collectors", {"table_properties_collectors",
OptionTypeInfo::Vector< OptionTypeInfo::Vector<
std::shared_ptr<TablePropertiesCollectorFactory>>( std::shared_ptr<TablePropertiesCollectorFactory>>(
offset_of( offsetof(struct ImmutableCFOptions,
&ImmutableCFOptions::table_properties_collector_factories), table_properties_collector_factories),
OptionVerificationType::kByName, OptionTypeFlags::kNone, OptionVerificationType::kByName, OptionTypeFlags::kNone,
OptionTypeInfo::AsCustomSharedPtr<TablePropertiesCollectorFactory>( OptionTypeInfo::AsCustomSharedPtr<TablePropertiesCollectorFactory>(
0, OptionVerificationType::kByName, OptionTypeFlags::kNone))}, 0, OptionVerificationType::kByName, OptionTypeFlags::kNone))},
{"compaction_filter", {"compaction_filter",
OptionTypeInfo::AsCustomRawPtr<const CompactionFilter>( OptionTypeInfo::AsCustomRawPtr<const CompactionFilter>(
offset_of(&ImmutableCFOptions::compaction_filter), offsetof(struct ImmutableCFOptions, compaction_filter),
OptionVerificationType::kByName, OptionTypeFlags::kAllowNull)}, OptionVerificationType::kByName, OptionTypeFlags::kAllowNull)},
{"compaction_filter_factory", {"compaction_filter_factory",
OptionTypeInfo::AsCustomSharedPtr<CompactionFilterFactory>( OptionTypeInfo::AsCustomSharedPtr<CompactionFilterFactory>(
offset_of(&ImmutableCFOptions::compaction_filter_factory), offsetof(struct ImmutableCFOptions, compaction_filter_factory),
OptionVerificationType::kByName, OptionTypeFlags::kAllowNull)}, OptionVerificationType::kByName, OptionTypeFlags::kAllowNull)},
{"merge_operator", {"merge_operator",
OptionTypeInfo::AsCustomSharedPtr<MergeOperator>( OptionTypeInfo::AsCustomSharedPtr<MergeOperator>(
offset_of(&ImmutableCFOptions::merge_operator), offsetof(struct ImmutableCFOptions, merge_operator),
OptionVerificationType::kByNameAllowFromNull, OptionVerificationType::kByNameAllowFromNull,
OptionTypeFlags::kCompareLoose | OptionTypeFlags::kAllowNull)}, OptionTypeFlags::kCompareLoose | OptionTypeFlags::kAllowNull)},
{"compaction_style", {"compaction_style",
{offset_of(&ImmutableCFOptions::compaction_style), {offsetof(struct ImmutableCFOptions, compaction_style),
OptionType::kCompactionStyle, OptionVerificationType::kNormal, OptionType::kCompactionStyle, OptionVerificationType::kNormal,
OptionTypeFlags::kNone}}, OptionTypeFlags::kNone}},
{"compaction_pri", {"compaction_pri",
{offset_of(&ImmutableCFOptions::compaction_pri), {offsetof(struct ImmutableCFOptions, compaction_pri),
OptionType::kCompactionPri, OptionVerificationType::kNormal, OptionType::kCompactionPri, OptionVerificationType::kNormal,
OptionTypeFlags::kNone}}, OptionTypeFlags::kNone}},
{"sst_partitioner_factory", {"sst_partitioner_factory",
OptionTypeInfo::AsCustomSharedPtr<SstPartitionerFactory>( OptionTypeInfo::AsCustomSharedPtr<SstPartitionerFactory>(
offset_of(&ImmutableCFOptions::sst_partitioner_factory), offsetof(struct ImmutableCFOptions, sst_partitioner_factory),
OptionVerificationType::kByName, OptionTypeFlags::kAllowNull)}, OptionVerificationType::kByName, OptionTypeFlags::kAllowNull)},
}; };

17
options/customizable_test.cc
Unescape View File

@ -206,31 +206,22 @@ struct SimpleOptions {
TestCustomizable* cp = nullptr; TestCustomizable* cp = nullptr;
}; };
static SimpleOptions dummy_simple_options;
template <typename T1>
int offset_of(T1 SimpleOptions::*member) {
return static_cast<int>(
reinterpret_cast<uintptr_t>(
std::addressof(dummy_simple_options.*member)) -
reinterpret_cast<uintptr_t>(std::addressof(dummy_simple_options)));
}
static std::unordered_map<std::string, OptionTypeInfo> simple_option_info = { static std::unordered_map<std::string, OptionTypeInfo> simple_option_info = {
#ifndef ROCKSDB_LITE #ifndef ROCKSDB_LITE
{"bool", {"bool",
{offset_of(&SimpleOptions::b), OptionType::kBoolean, {offsetof(struct SimpleOptions, b), OptionType::kBoolean,
OptionVerificationType::kNormal, OptionTypeFlags::kNone}}, OptionVerificationType::kNormal, OptionTypeFlags::kNone}},
{"unique", {"unique",
OptionTypeInfo::AsCustomUniquePtr<TestCustomizable>( OptionTypeInfo::AsCustomUniquePtr<TestCustomizable>(
offset_of(&SimpleOptions::cu), OptionVerificationType::kNormal, offsetof(struct SimpleOptions, cu), OptionVerificationType::kNormal,
OptionTypeFlags::kAllowNull)}, OptionTypeFlags::kAllowNull)},
{"shared", {"shared",
OptionTypeInfo::AsCustomSharedPtr<TestCustomizable>( OptionTypeInfo::AsCustomSharedPtr<TestCustomizable>(
offset_of(&SimpleOptions::cs), OptionVerificationType::kNormal, offsetof(struct SimpleOptions, cs), OptionVerificationType::kNormal,
OptionTypeFlags::kAllowNull)}, OptionTypeFlags::kAllowNull)},
{"pointer", {"pointer",
OptionTypeInfo::AsCustomRawPtr<TestCustomizable>( OptionTypeInfo::AsCustomRawPtr<TestCustomizable>(
offset_of(&SimpleOptions::cp), OptionVerificationType::kNormal, offsetof(struct SimpleOptions, cp), OptionVerificationType::kNormal,
OptionTypeFlags::kAllowNull)}, OptionTypeFlags::kAllowNull)},
#endif // ROCKSDB_LITE #endif // ROCKSDB_LITE
}; };

52
options/options_settable_test.cc
Unescape View File

@ -367,12 +367,6 @@ TEST_F(OptionsSettableTest, DBOptionsAllFieldsSettable) {
delete[] new_options_ptr; delete[] new_options_ptr;
} }
template <typename T1, typename T2>
inline int offset_of(T1 T2::*member) {
static T2 obj;
return int(size_t(&(obj.*member)) - size_t(&obj));
}
// If the test fails, likely a new option is added to ColumnFamilyOptions // If the test fails, likely a new option is added to ColumnFamilyOptions
// but it cannot be set through GetColumnFamilyOptionsFromString(), or the // but it cannot be set through GetColumnFamilyOptionsFromString(), or the
// test is not updated accordingly. // test is not updated accordingly.
@ -386,36 +380,39 @@ TEST_F(OptionsSettableTest, ColumnFamilyOptionsAllFieldsSettable) {
// options in the excluded set need to appear in the same order as in // options in the excluded set need to appear in the same order as in
// ColumnFamilyOptions. // ColumnFamilyOptions.
const OffsetGap kColumnFamilyOptionsExcluded = { const OffsetGap kColumnFamilyOptionsExcluded = {
{offset_of(&ColumnFamilyOptions::inplace_callback), {offsetof(struct ColumnFamilyOptions, inplace_callback),
sizeof(UpdateStatus(*)(char*, uint32_t*, Slice, std::string*))}, sizeof(UpdateStatus(*)(char*, uint32_t*, Slice, std::string*))},
{offset_of( {offsetof(struct ColumnFamilyOptions,
&ColumnFamilyOptions::memtable_insert_with_hint_prefix_extractor), memtable_insert_with_hint_prefix_extractor),
sizeof(std::shared_ptr<const SliceTransform>)}, sizeof(std::shared_ptr<const SliceTransform>)},
{offset_of(&ColumnFamilyOptions::compression_per_level), {offsetof(struct ColumnFamilyOptions, compression_per_level),
sizeof(std::vector<CompressionType>)}, sizeof(std::vector<CompressionType>)},
{offset_of( {offsetof(struct ColumnFamilyOptions,
&ColumnFamilyOptions::max_bytes_for_level_multiplier_additional), max_bytes_for_level_multiplier_additional),
sizeof(std::vector<int>)}, sizeof(std::vector<int>)},
{offset_of(&ColumnFamilyOptions::memtable_factory), {offsetof(struct ColumnFamilyOptions, memtable_factory),
sizeof(std::shared_ptr<MemTableRepFactory>)}, sizeof(std::shared_ptr<MemTableRepFactory>)},
{offset_of(&ColumnFamilyOptions::table_properties_collector_factories), {offsetof(struct ColumnFamilyOptions,
table_properties_collector_factories),
sizeof(ColumnFamilyOptions::TablePropertiesCollectorFactories)}, sizeof(ColumnFamilyOptions::TablePropertiesCollectorFactories)},
{offset_of(&ColumnFamilyOptions::comparator), sizeof(Comparator*)}, {offsetof(struct ColumnFamilyOptions, comparator), sizeof(Comparator*)},
{offset_of(&ColumnFamilyOptions::merge_operator), {offsetof(struct ColumnFamilyOptions, merge_operator),
sizeof(std::shared_ptr<MergeOperator>)}, sizeof(std::shared_ptr<MergeOperator>)},
{offset_of(&ColumnFamilyOptions::compaction_filter), {offsetof(struct ColumnFamilyOptions, compaction_filter),
sizeof(const CompactionFilter*)}, sizeof(const CompactionFilter*)},
{offset_of(&ColumnFamilyOptions::compaction_filter_factory), {offsetof(struct ColumnFamilyOptions, compaction_filter_factory),
sizeof(std::shared_ptr<CompactionFilterFactory>)}, sizeof(std::shared_ptr<CompactionFilterFactory>)},
{offset_of(&ColumnFamilyOptions::prefix_extractor), {offsetof(struct ColumnFamilyOptions, prefix_extractor),
sizeof(std::shared_ptr<const SliceTransform>)}, sizeof(std::shared_ptr<const SliceTransform>)},
{offset_of(&ColumnFamilyOptions::snap_refresh_nanos), sizeof(uint64_t)}, {offsetof(struct ColumnFamilyOptions, snap_refresh_nanos),
{offset_of(&ColumnFamilyOptions::table_factory), sizeof(uint64_t)},
{offsetof(struct ColumnFamilyOptions, table_factory),
sizeof(std::shared_ptr<TableFactory>)}, sizeof(std::shared_ptr<TableFactory>)},
{offset_of(&ColumnFamilyOptions::cf_paths), sizeof(std::vector<DbPath>)}, {offsetof(struct ColumnFamilyOptions, cf_paths),
{offset_of(&ColumnFamilyOptions::compaction_thread_limiter), sizeof(std::vector<DbPath>)},
{offsetof(struct ColumnFamilyOptions, compaction_thread_limiter),
sizeof(std::shared_ptr<ConcurrentTaskLimiter>)}, sizeof(std::shared_ptr<ConcurrentTaskLimiter>)},
{offset_of(&ColumnFamilyOptions::sst_partitioner_factory), {offsetof(struct ColumnFamilyOptions, sst_partitioner_factory),
sizeof(std::shared_ptr<SstPartitionerFactory>)}, sizeof(std::shared_ptr<SstPartitionerFactory>)},
}; };
@ -538,11 +535,12 @@ TEST_F(OptionsSettableTest, ColumnFamilyOptionsAllFieldsSettable) {
// Test copying to mutabable and immutable options and copy back the mutable // Test copying to mutabable and immutable options and copy back the mutable
// part. // part.
const OffsetGap kMutableCFOptionsExcluded = { const OffsetGap kMutableCFOptionsExcluded = {
{offset_of(&MutableCFOptions::prefix_extractor), {offsetof(struct MutableCFOptions, prefix_extractor),
sizeof(std::shared_ptr<const SliceTransform>)}, sizeof(std::shared_ptr<const SliceTransform>)},
{offset_of(&MutableCFOptions::max_bytes_for_level_multiplier_additional), {offsetof(struct MutableCFOptions,
max_bytes_for_level_multiplier_additional),
sizeof(std::vector<int>)}, sizeof(std::vector<int>)},
{offset_of(&MutableCFOptions::max_file_size), {offsetof(struct MutableCFOptions, max_file_size),
sizeof(std::vector<uint64_t>)}, sizeof(std::vector<uint64_t>)},
}; };

Write Preview
Loading…
Cancel
Save