Summary:
This diff merges pull request #208. Contributor: ankgup87
[Java] Optimize statistics collector
* Optimize statistics collector by collecting statistics of multiple DBs in a single thread rather than starting up a new thread for each DB.
* Also, fix packaging of jnilib file on OS_X platform.
* Diff review: https://reviews.facebook.net/D20265
[Java] Add documentation on interdependency of dispose call of RocksObjects
* Remove transferCppRawPointersOwnershipFrom function.
- This function was setting opt.filter_ and thus filter_ to be null. This way there is no
one holding reference for filter object and can thus be GC'd which is not the intention.
Replaced it with storeOptionsInstace which stores options instance. Options class
internally holds Filter instance. Thus when Options is GC'd, filter reference
will be GC'd automatically.
* Added documentation explaining interdependency of Filter, Options and DB.
* Diff review: https://reviews.facebook.net/D20379
Test Plan:
described in their diff reviews
Reviewers: haobo sdong swapnilghike zzbennett rsumbaly yhchiang
Reviewed by: yhchiang
Summary:
Add setCacheNumShardBits() and cacheNumShardBits() to Options. This allows
developers to control the number of shards for the block cache.
Test Plan:
make rocksdbjava
cd java
make db_bench
./jdb_bench.sh --cache_size=1048576 --cache_numshardbits=6
Reviewers: sdong, ljin, ankgup87
Reviewed By: ankgup87
Subscribers: leveldb
Differential Revision: https://reviews.facebook.net/D19347
Summary:
1. Move disOwnNativeHandle() function from RocksDB to RocksObject
to allow other RocksObject to use disOwnNativeHandle() when its
ownership of native handle has been transferred.
2. RocksObject now has an abstract implementation of dispose(),
which does the following two things. First, it checks whether
both isOwningNativeHandle() and isInitialized() return true.
If so, it will call the protected abstract function dispose0(),
which all the subclasses of RocksObject should implement. Second,
it sets nativeHandle_ = 0. This redesign ensure all subclasses
of RocksObject have the same dispose behavior.
3. All subclasses of RocksObject now should implement dispose0()
instead of dispose(), and dispose0() will be called only when
isInitialized() returns true.
Test Plan:
make rocksdbjava
make jtest
Reviewers: dhruba, sdong, ankgup87, rsumbaly, swapnilghike, zzbennett, haobo
Reviewed By: haobo
Subscribers: leveldb
Differential Revision: https://reviews.facebook.net/D18801
Summary:
Add RocksObject, the base class of all java objects which has a c++ pointer.
While the finalizer of a RocksObject will release its c++ resource, it is
suggested to call its RocksObject.dispose() to manually release its
c++ resource.
Existing RocksDB java classes are now extending RocksObject.
Test Plan:
make rocksdbjava
make jtest
make jdb_bench
Reviewers: haobo, dhruba, sdong, ankgup87, rsumbaly, swapnilghike, zzbennett
Reviewed By: haobo
CC: leveldb
Differential Revision: https://reviews.facebook.net/D18411
Summary:
* Add filluniquerandom
* Add readseq, implemented using iterator.
* Realize most command-line-arguments from db_bench.cc (70+).
* Some code are commented out as some of the options in Options
not yet have Java bindings.
* Add default option to DbBenchmark.
* RocksDB will now take the ownership of all c++ raw-pointers from Options, which includes a c++ raw-pointer for Filter.
Test Plan: ./jdb_bench.sh --db=/tmp/rocksjava-bench/db --num_levels=6 --key_size=20 --prefix_size=20 --keys_per_prefix=0 --value_size=100 --block_size=4096 --cache_size=17179869184 --cache_numshardbits=6 --compression_ratio=1 --min_level_to_compress=-1 --disable_seek_compaction=1 --hard_rate_limit=2 --write_buffer_size=134217728 --max_write_buffer_number=2 --level0_file_num_compaction_trigger=8 --target_file_size_base=134217728 --max_bytes_for_level_base=1073741824 --disable_wal=0 --wal_dir=/tmp/rocksjava-bench/wal --sync=0 --disable_data_sync=1 --verify_checksum=1 --delete_obsolete_files_period_micros=314572800 --max_grandparent_overlap_factor=10 --max_background_compactions=4 --max_background_flushes=0 --level0_slowdown_writes_trigger=16 --level0_stop_writes_trigger=24 --statistics=0 --stats_per_interval=0 --stats_interval=1048576 --histogram=0 --use_plain_table=1 --open_files=-1 --mmap_read=1 --mmap_write=0 --memtablerep=prefix_hash --bloom_bits=10 --bloom_locality=1 --perf_level=0 --benchmarks=filluniquerandom,readseq,readrandom --use_existing_db=0 --threads=4
Reviewers: haobo, dhruba, sdong, ankgup87, rsumbaly, swapnilghike, zzbennett
Reviewed By: haobo
CC: leveldb
Differential Revision: https://reviews.facebook.net/D18267
Summary:
Add Java bindings and Java tests for 30+ rocksdb::Options. Codes are
machine-genearted based on include/rocksdb/options.h with manual-polishment.
Test Plan:
make rocksdbjava
make jtest
Reviewers: haobo, dhruba, sdong, ankgup87, rsumbaly, swapnilghike, zzbennett
Reviewed By: haobo
CC: leveldb
Differential Revision: https://reviews.facebook.net/D18273
Summary:
Add Java bindings for memtables and sst format. Specifically,
add two abstract Java classses --- MemTableConfig and SstFormatConfig.
Each MemTable / SST implementation should has its own config class
extends MemTableConfig / SstFormatConfig respectively and pass it
to Options via setMemTableConfig / setSstConfig.
Test Plan:
make rocksdbjava
make jdb_test
make jdb_bench
java/jdb_bench.sh \
--benchmarks=fillseq,readrandom,readwhilewriting \
--memtablerep=hash_skiplist \
--use_plain_table=1 \
--key_size=20 \
--prefix_size=12 \
--value_size=100 \
--cache_size=17179869184 \
--disable_wal=0 \
--sync=0 \
Reviewers: haobo, ankgup87, sdong
Reviewed By: haobo
CC: leveldb, dhruba
Differential Revision: https://reviews.facebook.net/D17997
Summary:
Add a skeleton binding and test for BackupableDB which shows that BackupableDB
and RocksDB can share the same JNI calls.
Test Plan:
make rocksdbjava
make jtest
Reviewers: haobo, ankgup87, sdong, dhruba
Reviewed By: haobo
CC: leveldb
Differential Revision: https://reviews.facebook.net/D17793
Summary:
* Add a class SizeUnit to store frequently used consts. Currently
it has KB, MB, GB, TB, and PB.
* Change the parameter type of Options.writeBufferSize and Options.blockSize
from int to long.
Test Plan:
make rocksdbjava
make jtest
Reviewers: haobo, ankgup87, sdong, dhruba
Reviewed By: haobo
CC: leveldb
Differential Revision: https://reviews.facebook.net/D17703
Summary:
Add java bindings for Options.block_cache and allow DbBenchmark to
set cache_size.
Test Plan:
make rocksdbjava
make jtest
make jdb_Bench
Reviewers: haobo, sdong, ankgup87
Reviewed By: ankgup87
CC: leveldb, dhruba
Differential Revision: https://reviews.facebook.net/D17481
Summary:
Make each set function of Options / WriteOptions return its option instance.
Java developers can now easier specify each option like the following:
options.setCreateIfMissing(true)
.setWriteBufferSize(8 * 1024)
.setMaxWriteBufferNumber(3)
.setDisableSeekCompaction(true)
.setBlockSize(64 * 1024)
.setMaxBackgroundCompactions(10);
Test Plan:
make rocksdbjava
make jtest
Reviewers: haobo, ankgup87, sdong, dhruba
Reviewed By: haobo
CC: leveldb
Differential Revision: https://reviews.facebook.net/D17661
Summary:
* [java] Add a java api for rocksdb::Options, currently only supports create_if_missing.
* [java] Add a test for RocksDBException in RocksDBSample.
Test Plan: make jtest
Reviewers: haobo, sdong
Reviewed By: haobo
CC: leveldb, dhruba
Differential Revision: https://reviews.facebook.net/D17385
Yueh-Hsuan Chiang