fork of https://github.com/oxigraph/rocksdb and https://github.com/facebook/rocksdb for nextgraph and oxigraph

You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

Andrew Kryczka c6abe30ee3 Fix concurrent full purge and WAL recycling (#5900) ... Summary: We were removing the file from `log_recycle_files_` before renaming it with `ReuseWritableFile()`. Since `ReuseWritableFile()` occurs outside the DB mutex, it was possible for a concurrent full purge to sneak in and delete the file before it could be renamed. Consequently, `SwitchMemtable()` would fail and the DB would enter read-only mode. The fix is to hold the old file number in `log_recycle_files_` until after the file has been renamed. Full purge uses that list to decide which files to keep, so it can no longer delete a file pending recycling. Pull Request resolved: https://github.com/facebook/rocksdb/pull/5900 Test Plan: new unit test Differential Revision: D19771719 Pulled By: ajkr fbshipit-source-id: 094346349ca3fb499712e62de03905acc30b5ce8		4 years ago
..
compaction	Checksum for each SST file and stores in MANIFEST (#6216)	4 years ago
db_impl	Fix concurrent full purge and WAL recycling (#5900)	4 years ago
arena_wrapped_db_iter.cc	delete superversions in BackgroundCallPurge (#6146)	5 years ago
arena_wrapped_db_iter.h	Refactor ArenaWrappedDBIter into separate files (#5801)	5 years ago
blob_index.h	Mark BlobIndex::DebugString const	5 years ago
builder.cc	Checksum for each SST file and stores in MANIFEST (#6216)	4 years ago
builder.h	Introduce a new storage specific Env API (#5761)	5 years ago
c.cc	Add an option to prevent DB::Open() from querying sizes of all sst files (#6353)	4 years ago
c_test.c	Return NotSupported from WriteBatchWithIndex::DeleteRange (#5393)	4 years ago
column_family.cc	Avoid create directory for every column families (#6358)	4 years ago
column_family.h	Avoid create directory for every column families (#6358)	4 years ago
column_family_test.cc	Add unit tests for concurrent CF iteration and drop (#6180)	5 years ago
compact_files_test.cc	Use aggregate initialization for FlushJobInfo/CompactionJobInfo (#5997)	5 years ago
compacted_db_impl.cc	New API to get all merge operands for a Key (#5604)	5 years ago
compacted_db_impl.h	Use delete to disable automatic generated methods. (#5009)	5 years ago
comparator_db_test.cc	Checksum for each SST file and stores in MANIFEST (#6216)	4 years ago
convenience.cc	Introduce a new storage specific Env API (#5761)	5 years ago
corruption_test.cc	Fail fast in paranoid mode when LoadTableHandlers fail during recovering (#6368)	4 years ago
cuckoo_table_db_test.cc	Fail fast in paranoid mode when LoadTableHandlers fail during recovering (#6368)	4 years ago
db_basic_test.cc	Ensure all MultiGet IO errors are propagated to user (#6403)	4 years ago
db_blob_index_test.cc	Disable blob iterator test with max_sequential_skip_in_iterations==0 in LITE mode (#6052)	5 years ago
db_block_cache_test.cc	Small tidy and speed up of the travis build (#6181)	5 years ago
db_bloom_filter_test.cc	Fix db_bloom_filter_test clang LITE build (#6340)	4 years ago
db_compaction_filter_test.cc	Shorten certain test names to avoid infra failure (#6352)	4 years ago
db_compaction_test.cc	Make DBCompactionTest.SkipStatsUpdateTest more robust (#6306)	4 years ago
db_dynamic_level_test.cc	Fix flaky DBDynamicLevelTest.DynamicLevelMaxBytesBase2 (#4668)	6 years ago
db_encryption_test.cc	Fix EncryptedEnv assert (#5735)	5 years ago
db_filesnapshot.cc	Support concurrent CF iteration and drop (#6147)	5 years ago
db_flush_test.cc	Atomic flush rollback once on failure (#6385)	4 years ago
db_info_dumper.cc	Apply formatter to recent 200+ commits. (#5830)	5 years ago
db_info_dumper.h	Change RocksDB License	7 years ago
db_inplace_update_test.cc	Change RocksDB License	7 years ago
db_io_failure_test.cc	Disable DBIOFailureTest.NoSpaceCompactRange in LITE (#4596)	6 years ago
db_iter.cc	Add ReadOptions.auto_prefix_mode (#6314)	4 years ago
db_iter.h	Add ReadOptions.auto_prefix_mode (#6314)	4 years ago
db_iter_stress_test.cc	Move some memory related files from util/ to memory/ (#5382)	5 years ago
db_iter_test.cc	Move some memory related files from util/ to memory/ (#5382)	5 years ago
db_iterator_test.cc	Add unit tests for concurrent CF iteration and drop (#6180)	5 years ago
db_log_iter_test.cc	Apply modernize-use-override (2nd iteration)	5 years ago
db_memtable_test.cc	upgrade gtest 1.7.0 => 1.8.1 for json result writing	5 years ago
db_merge_operand_test.cc	New API to get all merge operands for a Key (#5604)	5 years ago
db_merge_operator_test.cc	unordered_write incompatible with max_successive_merges (#6284)	5 years ago
db_options_test.cc	Adjust thread pool sizes when setting max_background_jobs dynamically (#6300)	5 years ago
db_properties_test.cc	Refactor/consolidate legacy Bloom implementation details (#5784)	5 years ago
db_range_del_test.cc	Return NotSupported from WriteBatchWithIndex::DeleteRange (#5393)	4 years ago
db_sst_test.cc	Be able to read compatible leveldb sst files (#6370)	4 years ago
db_statistics_test.cc	Make statistics's stats_level change thread-safe (#5030)	5 years ago
db_table_properties_test.cc	upgrade gtest 1.7.0 => 1.8.1 for json result writing	5 years ago
db_tailing_iter_test.cc	Remove managed iterator	6 years ago
db_test.cc	Avoid to get manifest file size when recovering from it. (#6369)	4 years ago
db_test2.cc	Fix DBTest2.ChangePrefixExtractor LITE build (#6356)	4 years ago
db_test_util.cc	Add oldest snapshot sequence property (#6228)	5 years ago
db_test_util.h	Add oldest snapshot sequence property (#6228)	5 years ago
db_universal_compaction_test.cc	Shorten certain test names to avoid infra failure (#6352)	4 years ago
db_wal_test.cc	Fix concurrent full purge and WAL recycling (#5900)	4 years ago
db_write_test.cc	Fix queue manipulation in WriteThread::BeginWriteStall() (#6322)	4 years ago
dbformat.cc	Apply formatter to recent 200+ commits. (#5830)	5 years ago
dbformat.h	Use delete to disable automatic generated methods. (#5009)	5 years ago
dbformat_test.cc	Move some logging related files to logging/ (#5387)	5 years ago
deletefile_test.cc	Refactor deletefile_test.cc (#5822)	5 years ago
error_handler.cc	Force a new manifest file if append to current one fails (#6331)	4 years ago
error_handler.h	Fix typos in comments (#4456)	6 years ago
error_handler_test.cc	Fix a test failure in error_handler_test (#6367)	4 years ago
event_helpers.cc	BlobDB GC: add SST <-> oldest blob file referenced mapping (#5903)	5 years ago
event_helpers.h	BlobDB GC: add SST <-> oldest blob file referenced mapping (#5903)	5 years ago
experimental.cc	Organizing rocksdb/db directory	5 years ago
external_sst_file_basic_test.cc	Allow ingesting overlapping files (#5539)	5 years ago
external_sst_file_ingestion_job.cc	Checksum for each SST file and stores in MANIFEST (#6216)	4 years ago
external_sst_file_ingestion_job.h	Introduce a new storage specific Env API (#5761)	5 years ago
external_sst_file_test.cc	wait pending memtable writes on file ingestion or compact range (#6113)	5 years ago
fault_injection_test.cc	Move some logging related files to logging/ (#5387)	5 years ago
file_indexer.cc	Bug when multiple files at one level contains the same smallest key (#6285)	5 years ago
file_indexer.h	Move some memory related files from util/ to memory/ (#5382)	5 years ago
file_indexer_test.cc	Move some memory related files from util/ to memory/ (#5382)	5 years ago
filename_test.cc	Move some logging related files to logging/ (#5387)	5 years ago
flush_job.cc	Checksum for each SST file and stores in MANIFEST (#6216)	4 years ago
flush_job.h	Introduce a new storage specific Env API (#5761)	5 years ago
flush_job_test.cc	Introduce a new storage specific Env API (#5761)	5 years ago
flush_scheduler.cc	Support concurrent CF iteration and drop (#6147)	5 years ago
flush_scheduler.h	Refactor trimming logic for immutable memtables (#5022)	5 years ago
forward_iterator.cc	delete superversions in BackgroundCallPurge (#6146)	5 years ago
forward_iterator.h	Move some memory related files from util/ to memory/ (#5382)	5 years ago
forward_iterator_bench.cc	simplify include directive involving inttypes (#5402)	5 years ago
import_column_family_job.cc	Checksum for each SST file and stores in MANIFEST (#6216)	4 years ago
import_column_family_job.h	Introduce a new storage specific Env API (#5761)	5 years ago
import_column_family_test.cc	Apply formatter to recent 200+ commits. (#5830)	5 years ago
internal_stats.cc	Remove inaccurate code comment (#6274)	5 years ago
internal_stats.h	Add oldest snapshot sequence property (#6228)	5 years ago
job_context.h	WritePrepared: Fix visible key compacted out by compaction (#4883)	6 years ago
listener_test.cc	Propagate SST and blob file numbers through the EventListener interface (#5962)	5 years ago
log_format.h	Fix an inaccurate comment (#4315)	6 years ago
log_reader.cc	Divide file_reader_writer.h and .cc (#5803)	5 years ago
log_reader.h	Avoid to get manifest file size when recovering from it. (#6369)	4 years ago
log_test.cc	Introduce a new storage specific Env API (#5761)	5 years ago
log_writer.cc	Divide file_reader_writer.h and .cc (#5803)	5 years ago
log_writer.h	Use delete to disable automatic generated methods. (#5009)	5 years ago
logs_with_prep_tracker.cc	Skip deleted WALs during recovery	6 years ago
logs_with_prep_tracker.h	Skip deleted WALs during recovery	6 years ago
lookup_key.h	Avoid user key copying for Get/Put/Write with user-timestamp (#5502)	5 years ago
malloc_stats.cc	Support jemalloc compiled with `--with-jemalloc-prefix` (#5521)	5 years ago
malloc_stats.h	Change RocksDB License	7 years ago
manual_compaction_test.cc	Move some memory related files from util/ to memory/ (#5382)	5 years ago
memtable.cc	Add ReadOptions.auto_prefix_mode (#6314)	4 years ago
memtable.h	Do not schedule memtable trimming if there is no history (#6177)	5 years ago
memtable_list.cc	Fix a data race related to memtable trimming (#6187)	5 years ago
memtable_list.h	Fix a data race related to memtable trimming (#6187)	5 years ago
memtable_list_test.cc	Introduce a new storage specific Env API (#5761)	5 years ago
merge_context.h	Introduce a new MultiGet batching implementation (#5011)	5 years ago
merge_helper.cc	Fix merging range tombstone covering put during flush/compaction (#5406)	5 years ago
merge_helper.h	Remove v1 RangeDelAggregator (#4778)	6 years ago
merge_helper_test.cc	Move some memory related files from util/ to memory/ (#5382)	5 years ago
merge_operator.cc	Change RocksDB License	7 years ago
merge_test.cc	unordered_write incompatible with max_successive_merges (#6284)	5 years ago
obsolete_files_test.cc	Refactor ObsoleteFilesTest to inherit from DBTestBase (#5820)	5 years ago
options_file_test.cc	Organizing rocksdb/db directory	5 years ago
perf_context_test.cc	Move some memory related files from util/ to memory/ (#5382)	5 years ago
pinned_iterators_manager.h	Change RocksDB License	7 years ago
plain_table_db_test.cc	Fail fast in paranoid mode when LoadTableHandlers fail during recovering (#6368)	4 years ago
pre_release_callback.h	WritePrepared: reduce prepared_mutex_ overhead (#5420)	5 years ago
prefix_test.cc	Organizing rocksdb/db directory	5 years ago
range_del_aggregator.cc	Correct pragma once problem with Bazel on Windows (#6321)	4 years ago
range_del_aggregator.h	Correct pragma once problem with Bazel on Windows (#6321)	4 years ago
range_del_aggregator_bench.cc	Move some memory related files from util/ to memory/ (#5382)	5 years ago
range_del_aggregator_test.cc	Move test related files under util/ to test_util/ (#5377)	5 years ago
range_tombstone_fragmenter.cc	Apply formatter to recent 200+ commits. (#5830)	5 years ago
range_tombstone_fragmenter.h	Initialized pinned_pos_ and pinned_seq_pos_ in FragmentedRangeTombstoneIterator (#5720)	5 years ago
range_tombstone_fragmenter_test.cc	Move test related files under util/ to test_util/ (#5377)	5 years ago
read_callback.h	WriteUnPrepared: improve read your own write functionality (#5573)	5 years ago
repair.cc	Checksum for each SST file and stores in MANIFEST (#6216)	4 years ago
repair_test.cc	Introduce a new storage specific Env API (#5761)	5 years ago
snapshot_checker.h	WritePrepared: fix issue with snapshot released during compaction (#4858)	6 years ago
snapshot_impl.cc	Change RocksDB License	7 years ago
snapshot_impl.h	Add oldest snapshot sequence property (#6228)	5 years ago
table_cache.cc	Ensure all MultiGet IO errors are propagated to user (#6403)	4 years ago
table_cache.h	Introduce a new storage specific Env API (#5761)	5 years ago
table_properties_collector.cc	Feature for sampling and reporting compressibility (#4842)	5 years ago
table_properties_collector.h	Feature for sampling and reporting compressibility (#4842)	5 years ago
table_properties_collector_test.cc	Introduce a new storage specific Env API (#5761)	5 years ago
transaction_log_impl.cc	Introduce a new storage specific Env API (#5761)	5 years ago
transaction_log_impl.h	reuse scratch buffer in transaction_log_reader (#5702)	5 years ago
trim_history_scheduler.cc	Support concurrent CF iteration and drop (#6147)	5 years ago
trim_history_scheduler.h	Refactor trimming logic for immutable memtables (#5022)	5 years ago
version_builder.cc	Clean up VersionEdit a bit (#6383)	4 years ago
version_builder.h	Introduce a new storage specific Env API (#5761)	5 years ago
version_builder_test.cc	Checksum for each SST file and stores in MANIFEST (#6216)	4 years ago
version_edit.cc	Checksum for each SST file and stores in MANIFEST (#6216)	4 years ago
version_edit.h	Checksum for each SST file and stores in MANIFEST (#6216)	4 years ago
version_edit_test.cc	Checksum for each SST file and stores in MANIFEST (#6216)	4 years ago
version_set.cc	Fail fast in paranoid mode when LoadTableHandlers fail during recovering (#6368)	4 years ago
version_set.h	Checksum for each SST file and stores in MANIFEST (#6216)	4 years ago
version_set_test.cc	Checksum for each SST file and stores in MANIFEST (#6216)	4 years ago
wal_manager.cc	Introduce a new storage specific Env API (#5761)	5 years ago
wal_manager.h	Introduce a new storage specific Env API (#5761)	5 years ago
wal_manager_test.cc	Introduce a new storage specific Env API (#5761)	5 years ago
write_batch.cc	WriteUnPrepared: Fix assertion during recovery (#6419)	4 years ago
write_batch_base.cc	Change RocksDB License	7 years ago
write_batch_internal.h	Add insert hints for each writebatch (#5728)	5 years ago
write_batch_test.cc	Return NotSupported from WriteBatchWithIndex::DeleteRange (#5393)	4 years ago
write_callback.h	Change RocksDB License	7 years ago
write_callback_test.cc	WritePrepared: reduce prepared_mutex_ overhead (#5420)	5 years ago
write_controller.cc	Change RocksDB License	7 years ago
write_controller.h	Change RocksDB License	7 years ago
write_controller_test.cc	Move test related files under util/ to test_util/ (#5377)	5 years ago
write_thread.cc	db/write_thread.cc: Initialize state (#6275)	4 years ago
write_thread.h	Option to make write group size configurable (#5759)	5 years ago