Summary:
This diff merges pull request #208. Contributor: ankgup87
[Java] Optimize statistics collector
* Optimize statistics collector by collecting statistics of multiple DBs in a single thread rather than starting up a new thread for each DB.
* Also, fix packaging of jnilib file on OS_X platform.
* Diff review: https://reviews.facebook.net/D20265
[Java] Add documentation on interdependency of dispose call of RocksObjects
* Remove transferCppRawPointersOwnershipFrom function.
- This function was setting opt.filter_ and thus filter_ to be null. This way there is no
one holding reference for filter object and can thus be GC'd which is not the intention.
Replaced it with storeOptionsInstace which stores options instance. Options class
internally holds Filter instance. Thus when Options is GC'd, filter reference
will be GC'd automatically.
* Added documentation explaining interdependency of Filter, Options and DB.
* Diff review: https://reviews.facebook.net/D20379
Test Plan:
described in their diff reviews
Reviewers: haobo sdong swapnilghike zzbennett rsumbaly yhchiang
Reviewed by: yhchiang
Summary:
Add setCacheNumShardBits() and cacheNumShardBits() to Options. This allows
developers to control the number of shards for the block cache.
Test Plan:
make rocksdbjava
cd java
make db_bench
./jdb_bench.sh --cache_size=1048576 --cache_numshardbits=6
Reviewers: sdong, ljin, ankgup87
Reviewed By: ankgup87
Subscribers: leveldb
Differential Revision: https://reviews.facebook.net/D19347
Summary:
Correct the library loading for zlib in RocksJava: zlib should
be loaded by loadLibrary("z") instead of loadLibrary("zlib").
Test Plan:
make rocksdbjava
cd java
make db_bench
./jdb_bench.sh --compression_type=zlib
Reviewers: sdong, ljin, ankgup87
Reviewed By: ankgup87
Subscribers: leveldb
Differential Revision: https://reviews.facebook.net/D19341
Summary:
Enable the random values in Java DB Bench to be generated based
on the compression_ratio specified in the command-line arguments.
Test Plan:
make rocksdbjava
java/jdb_bench.sh
Reviewers: sdong, ankgup87, haobo
Reviewed By: haobo
Subscribers: leveldb
Differential Revision: https://reviews.facebook.net/D19101
Summary:
Improve the Java Doc of RocksObject, which explains more about the life-cycle
of a native handle.
Test Plan: no code change
Reviewers: haobo, sdong, ankgup87
Reviewed By: ankgup87
Subscribers: leveldb
Differential Revision: https://reviews.facebook.net/D19179
Summary:
Improve documentation for RocksEnv and its C++ resource. Specifically,
the result of RocksEnv::Default() is a singleton, and the ownership
of its c++ resource belongs to rocksdb c++. As a result, calling
dispose() of the return value of RocksDB::Default() will be no-op.
Test Plan: no code change.
Reviewers: haobo, ankgup87
Reviewed By: ankgup87
Subscribers: leveldb
Differential Revision: https://reviews.facebook.net/D18945
Summary:
1. Move disOwnNativeHandle() function from RocksDB to RocksObject
to allow other RocksObject to use disOwnNativeHandle() when its
ownership of native handle has been transferred.
2. RocksObject now has an abstract implementation of dispose(),
which does the following two things. First, it checks whether
both isOwningNativeHandle() and isInitialized() return true.
If so, it will call the protected abstract function dispose0(),
which all the subclasses of RocksObject should implement. Second,
it sets nativeHandle_ = 0. This redesign ensure all subclasses
of RocksObject have the same dispose behavior.
3. All subclasses of RocksObject now should implement dispose0()
instead of dispose(), and dispose0() will be called only when
isInitialized() returns true.
Test Plan:
make rocksdbjava
make jtest
Reviewers: dhruba, sdong, ankgup87, rsumbaly, swapnilghike, zzbennett, haobo
Reviewed By: haobo
Subscribers: leveldb
Differential Revision: https://reviews.facebook.net/D18801
Summary:
Originally, the boolean arguments in Java DB benchmark only takes
true and false as valid input. This diff allows boolean arguments
to be set using 0 or 1.
Test Plan:
make rocksdbjava
make jdb_bench
java/jdb_bench.sh --db=/tmp/path-not-exist --use_existing_db=1
java/jdb_bench.sh --db=/tmp/path-not-exist --use_existing_db=0
java/jdb_bench.sh --db=/tmp/path-not-exist --use_existing_db=1
Reviewers: haobo, dhruba, sdong, ankgup87, rsumbaly, swapnilghike, zzbennett
Reviewed By: haobo
Subscribers: leveldb
Differential Revision: https://reviews.facebook.net/D18687
Summary: Fixed a bug in Java DB Benchmark where random reads does not consider full key range.
Test Plan:
make rocksdbjava
make jdb_bench
cd java
jdb_bench.sh --db=/tmp/rocksdb-test --benchmarks=fillseq --use_existing_db=false --num=100000
jdb_bench.sh --db=/tmp/rocksdb-test --benchmarks=readrandom --use_existing_db=true --num=100000 --reads=1000000
Reviewers: haobo, sdong
Reviewed By: haobo
CC: leveldb
Differential Revision: https://reviews.facebook.net/D18693
Summary: Make read benchmarks print out found / not-found information.
Test Plan:
make rocksdbjava
make jdb_bench
cd java
./jdb_bench.sh
Reviewers: haobo, sdong
CC: leveldb
Differential Revision: https://reviews.facebook.net/D18699
Summary:
Add RocksObject, the base class of all java objects which has a c++ pointer.
While the finalizer of a RocksObject will release its c++ resource, it is
suggested to call its RocksObject.dispose() to manually release its
c++ resource.
Existing RocksDB java classes are now extending RocksObject.
Test Plan:
make rocksdbjava
make jtest
make jdb_bench
Reviewers: haobo, dhruba, sdong, ankgup87, rsumbaly, swapnilghike, zzbennett
Reviewed By: haobo
CC: leveldb
Differential Revision: https://reviews.facebook.net/D18411
Summary:
* Add filluniquerandom
* Add readseq, implemented using iterator.
* Realize most command-line-arguments from db_bench.cc (70+).
* Some code are commented out as some of the options in Options
not yet have Java bindings.
* Add default option to DbBenchmark.
* RocksDB will now take the ownership of all c++ raw-pointers from Options, which includes a c++ raw-pointer for Filter.
Test Plan: ./jdb_bench.sh --db=/tmp/rocksjava-bench/db --num_levels=6 --key_size=20 --prefix_size=20 --keys_per_prefix=0 --value_size=100 --block_size=4096 --cache_size=17179869184 --cache_numshardbits=6 --compression_ratio=1 --min_level_to_compress=-1 --disable_seek_compaction=1 --hard_rate_limit=2 --write_buffer_size=134217728 --max_write_buffer_number=2 --level0_file_num_compaction_trigger=8 --target_file_size_base=134217728 --max_bytes_for_level_base=1073741824 --disable_wal=0 --wal_dir=/tmp/rocksjava-bench/wal --sync=0 --disable_data_sync=1 --verify_checksum=1 --delete_obsolete_files_period_micros=314572800 --max_grandparent_overlap_factor=10 --max_background_compactions=4 --max_background_flushes=0 --level0_slowdown_writes_trigger=16 --level0_stop_writes_trigger=24 --statistics=0 --stats_per_interval=0 --stats_interval=1048576 --histogram=0 --use_plain_table=1 --open_files=-1 --mmap_read=1 --mmap_write=0 --memtablerep=prefix_hash --bloom_bits=10 --bloom_locality=1 --perf_level=0 --benchmarks=filluniquerandom,readseq,readrandom --use_existing_db=0 --threads=4
Reviewers: haobo, dhruba, sdong, ankgup87, rsumbaly, swapnilghike, zzbennett
Reviewed By: haobo
CC: leveldb
Differential Revision: https://reviews.facebook.net/D18267
Summary:
Add Java bindings and Java tests for 30+ rocksdb::Options. Codes are
machine-genearted based on include/rocksdb/options.h with manual-polishment.
Test Plan:
make rocksdbjava
make jtest
Reviewers: haobo, dhruba, sdong, ankgup87, rsumbaly, swapnilghike, zzbennett
Reviewed By: haobo
CC: leveldb
Differential Revision: https://reviews.facebook.net/D18273
Yueh-Hsuan Chiang