fork of https://github.com/oxigraph/rocksdb and https://github.com/facebook/rocksdb for nextgraph and oxigraph

You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

Dhruba Borthakur 95dda37858 Move filesize-based-sorting to outside the Mutex ... Summary: When a new version is created, we sort all the files at every level based on their size. This is necessary because we want to compact the largest file first. The sorting takes quite a bit of CPU. Moved the sorting code to be outside the mutex. Also, the earlier code was sorting files at all levels but we do not need to sort the highest-number level because those files are never the cause of any compaction. To reduce sorting costs, we sort only the first few files in each level because it is likely that those are the only files in that level that will be picked for compaction. At steady state, I have seen that this patch increase throughout from 1500 writes/sec to 1700 writes/sec at the end of a 72 hour run. The cpu saving by not sorting the last level was not distinctive in this test run because there were only 100K files in the highest numbered level. I expect the cpu saving to be significant when the number of files is much higher. This is mostly an early preview and not ready for rigorous review. With this patch, the writs/sec is now bottlenecked not by the sorting code but by GetOverlappingInputs. I am working on a patch to optimize GetOverlappingInputs. Test Plan: make check Reviewers: MarkCallaghan, heyongqiang Reviewed By: heyongqiang Differential Revision: https://reviews.facebook.net/D6411		12 years ago
..
builder.cc	Allow having different compression algorithms on different levels.	12 years ago
builder.h	A number of fixes:	13 years ago
c.cc	Make compression options configurable. These include window-bits, level and strategy for ZlibCompression	12 years ago
c_test.c	Make compression options configurable. These include window-bits, level and strategy for ZlibCompression	12 years ago
corruption_test.cc	Fix all warnings generated by -Wall option to the compiler.	12 years ago
db_bench.cc	Merge branch 'master' into performance	12 years ago
db_filesnapshot.cc	The BackupAPI should also list the length of the manifest file.	12 years ago
db_impl.cc	Move filesize-based-sorting to outside the Mutex	12 years ago
db_impl.h	Merge branch 'master' into performance	12 years ago
db_iter.cc	A number of fixes:	13 years ago
db_iter.h	A number of fixes:	13 years ago
db_statistics.h	Fix table-cache size bug, gather table-cache statistics and prevent readahead done by fs. Summary:	12 years ago
db_stats_logger.cc	remove boost	12 years ago
db_test.cc	Flush Data at object destruction if disableWal is used.	12 years ago
dbformat.cc	Added bloom filter support.	12 years ago
dbformat.h	Make some variables configurable for each db instance	12 years ago
dbformat_test.cc	Fix all warnings generated by -Wall option to the compiler.	12 years ago
filename.cc	put log in a seperate dir	12 years ago
filename.h	put log in a seperate dir	12 years ago
filename_test.cc	Fix all warnings generated by -Wall option to the compiler.	12 years ago
log_format.h	A number of fixes:	13 years ago
log_reader.cc	Fix all warnings generated by -Wall option to the compiler.	12 years ago
log_reader.h	A number of fixes:	13 years ago
log_test.cc	Fix all warnings generated by -Wall option to the compiler.	12 years ago
log_writer.cc	A number of fixes:	13 years ago
log_writer.h	A number of fixes:	13 years ago
memtable.cc	This is the mega-patch multi-threaded compaction	12 years ago
memtable.h	This is the mega-patch multi-threaded compaction	12 years ago
memtablelist.cc	This is the mega-patch multi-threaded compaction	12 years ago
memtablelist.h	This is the mega-patch multi-threaded compaction	12 years ago
repair.cc	This is the mega-patch multi-threaded compaction	12 years ago
skiplist.h	This is the mega-patch multi-threaded compaction	12 years ago
skiplist_test.cc	Fix all warnings generated by -Wall option to the compiler.	12 years ago
snapshot.h	A number of fixes:	13 years ago
table_cache.cc	Add two more options: disable block cache and make table cache shard number configuable	12 years ago
table_cache.h	Trigger read compaction only if seeks to storage are incurred.	12 years ago
version_edit.cc	Clean up compiler warnings generated by -Wall option.	12 years ago
version_edit.h	This is the mega-patch multi-threaded compaction	12 years ago
version_edit_test.cc	Make some variables configurable for each db instance	12 years ago
version_set.cc	Move filesize-based-sorting to outside the Mutex	12 years ago
version_set.h	Move filesize-based-sorting to outside the Mutex	12 years ago
version_set_reduce_num_levels.cc	Add a tool to change number of levels	12 years ago
version_set_test.cc	Fix all warnings generated by -Wall option to the compiler.	12 years ago
write_batch.cc	added group commit; drastically speeds up mult-threaded synchronous write workloads	12 years ago
write_batch_internal.h	added group commit; drastically speeds up mult-threaded synchronous write workloads	12 years ago
write_batch_test.cc	Fix all warnings generated by -Wall option to the compiler.	12 years ago