NextGraph
/
rocksdb
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Improve Java API get() performance by reducing copies (#10970)

Browse Source 
Summary:
Performance improvements for `get()` paths in the RocksJava API (JNI).
Document describing the performance results.

Replace uses of the legacy `DB::Get()` method wrapper returning data in a `std::string` with direct calls to `DB::Get()` passing a pinnable slice to receive this data. Copying from a pinned slice direct to the destination java byte array, without going via an intervening std::string, is a major performance gain for this code path.

Note that this gain only comes where `DB::Get()` is able to return a pinned buffer; where it has to copy into the buffer owned by the slice, there is still the intervening copy and no performance gain. It may be possible to address this case too, but it is not trivial.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/10970

Reviewed By: pdillinger

Differential Revision: D42125567

Pulled By: ajkr

fbshipit-source-id: b7a4df7523b0420cadb1e9b6c7da3ec030a8da34
main
Alan Paxton 2 years ago committed by Facebook GitHub Bot
parent dbf37c290a
commit f8969ad7d4
6 changed files with 286 additions and 66 deletions
Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL
Show Stats Download Patch File Download Diff File
  1. 161
      java/GetBenchmarks.md
  2. 2
      java/jmh/pom.xml
  3. 112
      java/jmh/src/main/java/org/rocksdb/jmh/GetBenchmarks.java
  4. 18
      java/jmh/src/main/java/org/rocksdb/jmh/MultiGetBenchmarks.java
  5. 4
      java/pom.xml.template
  6. 53
      java/rocksjni/rocksjni.cc

161
java/GetBenchmarks.md
Unescape View File

@ -0,0 +1,161 @@
# RocksDB Get Performance Benchmarks
Results associated with [Improve Java API `get()` performance by reducing copies](https://github.com/facebook/rocksdb/pull/10970)
## Build/Run
Mac
```
make clean jclean
DEBUG_LEVEL=0 make -j12 rocksdbjava
(cd java/target; cp rocksdbjni-7.9.0-osx.jar rocksdbjni-7.9.0-SNAPSHOT-osx.jar)
mvn install:install-file -Dfile=./java/target/rocksdbjni-7.9.0-SNAPSHOT-osx.jar -DgroupId=org.rocksdb -DartifactId=rocksdbjni -Dversion=7.9.0-SNAPSHOT -Dpackaging=jar
```
Linux
```
make clean jclean
DEBUG_LEVEL=0 make -j12 rocksdbjava
(cd java/target; cp rocksdbjni-7.9.0-linux64.jar rocksdbjni-7.9.0-SNAPSHOT-linux64.jar)
mvn install:install-file -Dfile=./java/target/rocksdbjni-7.9.0-SNAPSHOT-linux64.jar -DgroupId=org.rocksdb -DartifactId=rocksdbjni -Dversion=7.9.0-SNAPSHOT -Dpackaging=jar
```
Build jmh test package, on either platform
```
pushd java/jmh
mvn clean package
```
A quick test run, just as a sanity check, using a small number of keys, would be
```
java -jar target/rocksdbjni-jmh-1.0-SNAPSHOT-benchmarks.jar -p keyCount=1000 -p keySize=128 -p valueSize=32768 -p columnFamilyTestType="no_column_family" GetBenchmarks
```
The long performance run (as big as we can make it on our Ubuntu box without filling the disk)
```
java -jar target/rocksdbjni-jmh-1.0-SNAPSHOT-benchmarks.jar -p keyCount=1000,50000 -p keySize=128 -p valueSize=1024,16384 -p columnFamilyTestType="1_column_family","20_column_families" GetBenchmarks.get GetBenchmarks.preallocatedByteBufferGet GetBenchmarks.preallocatedGet
```
## Results (small runs, Mac)
These are run on a 10-core M1 with 64GB of memory and 2TB of SSD.
They probably reflect the absolute best case for this optimization, hitting in-memory buffers and completely eliminating a buffer copy.
### Before
Benchmark (columnFamilyTestType) (keyCount) (keySize) (multiGetSize) (valueSize) Mode Cnt Score Error Units
GetBenchmarks.get no_column_family 1000 128 N/A 32768 thrpt 25 43496.578 ± 5743.090 ops/s
GetBenchmarks.preallocatedByteBufferGet no_column_family 1000 128 N/A 32768 thrpt 25 70765.578 ± 697.548 ops/s
GetBenchmarks.preallocatedGet no_column_family 1000 128 N/A 32768 thrpt 25 69883.554 ± 944.184 ops/s
### After fixing byte[] (.get and .preallocatedGet)
Benchmark (columnFamilyTestType) (keyCount) (keySize) (multiGetSize) (valueSize) Mode Cnt Score Error Units
GetBenchmarks.get no_column_family 1000 128 N/A 32768 thrpt 25 149207.681 ± 2261.671 ops/s
GetBenchmarks.preallocatedByteBufferGet no_column_family 1000 128 N/A 32768 thrpt 25 68920.489 ± 1574.664 ops/s
GetBenchmarks.preallocatedGet no_column_family 1000 128 N/A 32768 thrpt 25 177399.022 ± 2107.375 ops/s
### After fixing ByteBuffer (.preallocatedByteBufferGet)
Benchmark (columnFamilyTestType) (keyCount) (keySize) (multiGetSize) (valueSize) Mode Cnt Score Error Units
GetBenchmarks.get no_column_family 1000 128 N/A 32768 thrpt 25 150389.259 ± 1371.473 ops/s
GetBenchmarks.preallocatedByteBufferGet no_column_family 1000 128 N/A 32768 thrpt 25 179919.468 ± 1670.714 ops/s
GetBenchmarks.preallocatedGet no_column_family 1000 128 N/A 32768 thrpt 25 178261.938 ± 2630.571 ops/s
## Results (Ubuntu, big runs)
These take 3-4 hours
```
java -jar target/rocksdbjni-jmh-1.0-SNAPSHOT-benchmarks.jar -p keyCount=1000,50000 -p keySize=128 -p valueSize=1024,16384 -p columnFamilyTestType="1_column_family","20_column_families" GetBenchmarks.get GetBenchmarks.preallocatedByteBufferGet GetBenchmarks.preallocatedGet
```
It's clear that all `get()` variants have noticeably improved performance, though not the spectacular gains of the M1.
### With fixes for all of the `get()` instances
Benchmark (columnFamilyTestType) (keyCount) (keySize) (valueSize) Mode Cnt Score Error Units
GetBenchmarks.get 1_column_family 1000 128 1024 thrpt 25 935648.793 ± 22879.910 ops/s
GetBenchmarks.get 1_column_family 1000 128 16384 thrpt 25 204366.301 ± 1326.570 ops/s
GetBenchmarks.get 1_column_family 50000 128 1024 thrpt 25 693451.990 ± 19822.720 ops/s
GetBenchmarks.get 1_column_family 50000 128 16384 thrpt 25 50473.768 ± 497.335 ops/s
GetBenchmarks.get 20_column_families 1000 128 1024 thrpt 25 550118.874 ± 14289.009 ops/s
GetBenchmarks.get 20_column_families 1000 128 16384 thrpt 25 120545.549 ± 648.280 ops/s
GetBenchmarks.get 20_column_families 50000 128 1024 thrpt 25 235671.353 ± 2231.195 ops/s
GetBenchmarks.get 20_column_families 50000 128 16384 thrpt 25 12463.887 ± 1950.746 ops/s
GetBenchmarks.preallocatedByteBufferGet 1_column_family 1000 128 1024 thrpt 25 1196026.040 ± 35435.729 ops/s
GetBenchmarks.preallocatedByteBufferGet 1_column_family 1000 128 16384 thrpt 25 403252.655 ± 3287.054 ops/s
GetBenchmarks.preallocatedByteBufferGet 1_column_family 50000 128 1024 thrpt 25 829965.448 ± 16945.452 ops/s
GetBenchmarks.preallocatedByteBufferGet 1_column_family 50000 128 16384 thrpt 25 63798.042 ± 1292.858 ops/s
GetBenchmarks.preallocatedByteBufferGet 20_column_families 1000 128 1024 thrpt 25 724557.253 ± 12710.828 ops/s
GetBenchmarks.preallocatedByteBufferGet 20_column_families 1000 128 16384 thrpt 25 176846.615 ± 1121.644 ops/s
GetBenchmarks.preallocatedByteBufferGet 20_column_families 50000 128 1024 thrpt 25 263553.764 ± 1304.243 ops/s
GetBenchmarks.preallocatedByteBufferGet 20_column_families 50000 128 16384 thrpt 25 14721.693 ± 2574.240 ops/s
GetBenchmarks.preallocatedGet 1_column_family 1000 128 1024 thrpt 25 1093947.765 ± 42846.276 ops/s
GetBenchmarks.preallocatedGet 1_column_family 1000 128 16384 thrpt 25 391629.913 ± 4039.965 ops/s
GetBenchmarks.preallocatedGet 1_column_family 50000 128 1024 thrpt 25 769332.958 ± 24180.749 ops/s
GetBenchmarks.preallocatedGet 1_column_family 50000 128 16384 thrpt 25 61712.038 ± 423.494 ops/s
GetBenchmarks.preallocatedGet 20_column_families 1000 128 1024 thrpt 25 694684.465 ± 5484.205 ops/s
GetBenchmarks.preallocatedGet 20_column_families 1000 128 16384 thrpt 25 172383.593 ± 841.679 ops/s
GetBenchmarks.preallocatedGet 20_column_families 50000 128 1024 thrpt 25 257447.351 ± 1388.667 ops/s
GetBenchmarks.preallocatedGet 20_column_families 50000 128 16384 thrpt 25 13418.522 ± 2418.619 ops/s
### Baseline (no fixes)
Benchmark (columnFamilyTestType) (keyCount) (keySize) (valueSize) Mode Cnt Score Error Units
GetBenchmarks.get 1_column_family 1000 128 1024 thrpt 25 866745.224 ± 8834.629 ops/s
GetBenchmarks.get 1_column_family 1000 128 16384 thrpt 25 184332.195 ± 2304.217 ops/s
GetBenchmarks.get 1_column_family 50000 128 1024 thrpt 25 666794.288 ± 16150.684 ops/s
GetBenchmarks.get 1_column_family 50000 128 16384 thrpt 25 47221.788 ± 433.165 ops/s
GetBenchmarks.get 20_column_families 1000 128 1024 thrpt 25 551513.636 ± 7763.681 ops/s
GetBenchmarks.get 20_column_families 1000 128 16384 thrpt 25 113117.720 ± 580.738 ops/s
GetBenchmarks.get 20_column_families 50000 128 1024 thrpt 25 238675.555 ± 1758.978 ops/s
GetBenchmarks.get 20_column_families 50000 128 16384 thrpt 25 11639.390 ± 1459.765 ops/s
GetBenchmarks.preallocatedByteBufferGet 1_column_family 1000 128 1024 thrpt 25 1153617.917 ± 26350.028 ops/s
GetBenchmarks.preallocatedByteBufferGet 1_column_family 1000 128 16384 thrpt 25 401710.334 ± 4324.539 ops/s
GetBenchmarks.preallocatedByteBufferGet 1_column_family 50000 128 1024 thrpt 25 809384.073 ± 13833.871 ops/s
GetBenchmarks.preallocatedByteBufferGet 1_column_family 50000 128 16384 thrpt 25 59279.005 ± 443.207 ops/s
GetBenchmarks.preallocatedByteBufferGet 20_column_families 1000 128 1024 thrpt 25 715466.403 ± 6591.375 ops/s
GetBenchmarks.preallocatedByteBufferGet 20_column_families 1000 128 16384 thrpt 25 175279.163 ± 910.923 ops/s
GetBenchmarks.preallocatedByteBufferGet 20_column_families 50000 128 1024 thrpt 25 263295.180 ± 856.456 ops/s
GetBenchmarks.preallocatedByteBufferGet 20_column_families 50000 128 16384 thrpt 25 14001.928 ± 2462.067 ops/s
GetBenchmarks.preallocatedGet 1_column_family 1000 128 1024 thrpt 25 1072866.854 ± 27030.592 ops/s
GetBenchmarks.preallocatedGet 1_column_family 1000 128 16384 thrpt 25 383950.853 ± 4510.654 ops/s
GetBenchmarks.preallocatedGet 1_column_family 50000 128 1024 thrpt 25 764395.469 ± 10097.417 ops/s
GetBenchmarks.preallocatedGet 1_column_family 50000 128 16384 thrpt 25 56851.330 ± 388.029 ops/s
GetBenchmarks.preallocatedGet 20_column_families 1000 128 1024 thrpt 25 668518.593 ± 9764.117 ops/s
GetBenchmarks.preallocatedGet 20_column_families 1000 128 16384 thrpt 25 171309.695 ± 875.895 ops/s
GetBenchmarks.preallocatedGet 20_column_families 50000 128 1024 thrpt 25 256057.801 ± 954.621 ops/s
GetBenchmarks.preallocatedGet 20_column_families 50000 128 16384 thrpt 25 13319.380 ± 2126.654 ops/s
### Comparison
It does at least look best when the data is cached. That is to say, smallest number of column families, and least keys.
GetBenchmarks.get 1_column_family 1000 128 16384 thrpt 25 204366.301 ± 1326.570 ops/s
GetBenchmarks.get 1_column_family 1000 128 16384 thrpt 25 184332.195 ± 2304.217 ops/s
GetBenchmarks.get 1_column_family 50000 128 16384 thrpt 25 50473.768 ± 497.335 ops/s
GetBenchmarks.get 1_column_family 50000 128 16384 thrpt 25 47221.788 ± 433.165 ops/s
GetBenchmarks.get 20_column_families 1000 128 16384 thrpt 25 120545.549 ± 648.280 ops/s
GetBenchmarks.get 20_column_families 1000 128 16384 thrpt 25 113117.720 ± 580.738 ops/s
GetBenchmarks.get 20_column_families 50000 128 16384 thrpt 25 12463.887 ± 1950.746 ops/s
GetBenchmarks.get 20_column_families 50000 128 16384 thrpt 25 11639.390 ± 1459.765 ops/s
### Baseline
25 minute run, small number of keys
```
java -jar target/rocksdbjni-jmh-1.0-SNAPSHOT-benchmarks.jar -p keyCount=1000 -p keySize=128 -p valueSize=32768 -p columnFamilyTestType="no_column_families" GetBenchmarks.get GetBenchmarks.preallocatedByteBufferGet GetBenchmarks.preallocatedGet
```
Benchmark (columnFamilyTestType) (keyCount) (keySize) (valueSize) Mode Cnt Score Error Units
GetBenchmarks.get no_column_families 1000 128 32768 thrpt 25 32344.908 ± 296.651 ops/s
GetBenchmarks.preallocatedByteBufferGet no_column_families 1000 128 32768 thrpt 25 45266.968 ± 424.514 ops/s
GetBenchmarks.preallocatedGet no_column_families 1000 128 32768 thrpt 25 43531.088 ± 291.785 ops/s
### Optimized
Benchmark (columnFamilyTestType) (keyCount) (keySize) (valueSize) Mode Cnt Score Error Units
GetBenchmarks.get no_column_families 1000 128 32768 thrpt 25 37463.716 ± 235.744 ops/s
GetBenchmarks.preallocatedByteBufferGet no_column_families 1000 128 32768 thrpt 25 48946.105 ± 466.463 ops/s
GetBenchmarks.preallocatedGet no_column_families 1000 128 32768 thrpt 25 47143.624 ± 576.763 ops/s
## Conclusion
The performance improvement is real.

2
java/jmh/pom.xml
Unescape View File

@ -50,7 +50,7 @@
<dependency> <dependency>
<groupId>org.rocksdb</groupId> <groupId>org.rocksdb</groupId>
<artifactId>rocksdbjni</artifactId> <artifactId>rocksdbjni</artifactId>
<version>6.27.0-SNAPSHOT</version> <version>7.9.0-SNAPSHOT</version>
</dependency> </dependency>
<dependency> <dependency>

112
java/jmh/src/main/java/org/rocksdb/jmh/GetBenchmarks.java
Unescape View File

@ -1,23 +1,24 @@
/** /**
* Copyright (c) 2011-present, Facebook, Inc. All rights reserved. * Copyright (c) 2011-present, Facebook, Inc. All rights reserved.
* This source code is licensed under both the GPLv2 (found in the * This source code is licensed under both the GPLv2 (found in the
* COPYING file in the root directory) and Apache 2.0 License * COPYING file in the root directory) and Apache 2.0 License
* (found in the LICENSE.Apache file in the root directory). * (found in the LICENSE.Apache file in the root directory).
*/ */
package org.rocksdb.jmh; package org.rocksdb.jmh;
import org.openjdk.jmh.annotations.*; import static org.rocksdb.util.KVUtils.ba;
import org.rocksdb.*;
import org.rocksdb.util.FileUtils;
import java.io.IOException; import java.io.IOException;
import java.nio.ByteBuffer;
import java.nio.file.Files; import java.nio.file.Files;
import java.nio.file.Path; import java.nio.file.Path;
import java.util.ArrayList; import java.util.ArrayList;
import java.util.Arrays;
import java.util.List; import java.util.List;
import java.util.concurrent.atomic.AtomicInteger; import java.util.concurrent.atomic.AtomicInteger;
import org.openjdk.jmh.annotations.*;
import static org.rocksdb.util.KVUtils.ba; import org.rocksdb.*;
import org.rocksdb.util.FileUtils;
@State(Scope.Benchmark) @State(Scope.Benchmark)
public class GetBenchmarks { public class GetBenchmarks {
@ -30,16 +31,24 @@ public class GetBenchmarks {
}) })
String columnFamilyTestType; String columnFamilyTestType;
@Param("100000") @Param({"1000", "100000"}) int keyCount;
int keyCount;
@Param({"12", "64", "128"}) int keySize;
@Param({"64", "1024", "65536"}) int valueSize;
Path dbDir; Path dbDir;
DBOptions options; DBOptions options;
ReadOptions readOptions;
int cfs = 0; // number of column families int cfs = 0; // number of column families
private AtomicInteger cfHandlesIdx; private AtomicInteger cfHandlesIdx;
ColumnFamilyHandle[] cfHandles; ColumnFamilyHandle[] cfHandles;
RocksDB db; RocksDB db;
private final AtomicInteger keyIndex = new AtomicInteger(); private final AtomicInteger keyIndex = new AtomicInteger();
private ByteBuffer keyBuf;
private ByteBuffer valueBuf;
private byte[] keyArr;
private byte[] valueArr;
@Setup(Level.Trial) @Setup(Level.Trial)
public void setup() throws IOException, RocksDBException { public void setup() throws IOException, RocksDBException {
@ -50,6 +59,7 @@ public class GetBenchmarks {
options = new DBOptions() options = new DBOptions()
.setCreateIfMissing(true) .setCreateIfMissing(true)
.setCreateMissingColumnFamilies(true); .setCreateMissingColumnFamilies(true);
readOptions = new ReadOptions();
final List<ColumnFamilyDescriptor> cfDescriptors = new ArrayList<>(); final List<ColumnFamilyDescriptor> cfDescriptors = new ArrayList<>();
cfDescriptors.add(new ColumnFamilyDescriptor(RocksDB.DEFAULT_COLUMN_FAMILY)); cfDescriptors.add(new ColumnFamilyDescriptor(RocksDB.DEFAULT_COLUMN_FAMILY));
@ -74,16 +84,32 @@ public class GetBenchmarks {
cfHandles = cfHandlesList.toArray(new ColumnFamilyHandle[0]); cfHandles = cfHandlesList.toArray(new ColumnFamilyHandle[0]);
// store initial data for retrieving via get // store initial data for retrieving via get
for (int i = 0; i < cfs; i++) { keyArr = new byte[keySize];
valueArr = new byte[valueSize];
Arrays.fill(keyArr, (byte) 0x30);
Arrays.fill(valueArr, (byte) 0x30);
for (int i = 0; i <= cfs; i++) {
for (int j = 0; j < keyCount; j++) { for (int j = 0; j < keyCount; j++) {
db.put(cfHandles[i], ba("key" + j), ba("value" + j)); final byte[] keyPrefix = ba("key" + j);
final byte[] valuePrefix = ba("value" + j);
System.arraycopy(keyPrefix, 0, keyArr, 0, keyPrefix.length);
System.arraycopy(valuePrefix, 0, valueArr, 0, valuePrefix.length);
db.put(cfHandles[i], keyArr, valueArr);
} }
} }
try (final FlushOptions flushOptions = new FlushOptions() try (final FlushOptions flushOptions = new FlushOptions().setWaitForFlush(true)) {
.setWaitForFlush(true)) {
db.flush(flushOptions); db.flush(flushOptions);
} }
keyBuf = ByteBuffer.allocateDirect(keySize);
valueBuf = ByteBuffer.allocateDirect(valueSize);
Arrays.fill(keyArr, (byte) 0x30);
Arrays.fill(valueArr, (byte) 0x30);
keyBuf.put(keyArr);
keyBuf.flip();
valueBuf.put(valueArr);
valueBuf.flip();
} }
@TearDown(Level.Trial) @TearDown(Level.Trial)
@ -93,13 +119,14 @@ public class GetBenchmarks {
} }
db.close(); db.close();
options.close(); options.close();
readOptions.close();
FileUtils.delete(dbDir); FileUtils.delete(dbDir);
} }
private ColumnFamilyHandle getColumnFamily() { private ColumnFamilyHandle getColumnFamily() {
if (cfs == 0) { if (cfs == 0) {
return cfHandles[0]; return cfHandles[0];
} else if (cfs == 1) { } else if (cfs == 1) {
return cfHandles[1]; return cfHandles[1];
} else { } else {
int idx = cfHandlesIdx.getAndIncrement(); int idx = cfHandlesIdx.getAndIncrement();
@ -131,9 +158,58 @@ public class GetBenchmarks {
return idx; return idx;
} }
@Benchmark // String -> byte[]
public byte[] get() throws RocksDBException { private byte[] getKeyArr() {
final int MAX_LEN = 9; // key100000
final int keyIdx = next();
final byte[] keyPrefix = ba("key" + keyIdx);
System.arraycopy(keyPrefix, 0, keyArr, 0, keyPrefix.length);
Arrays.fill(keyArr, keyPrefix.length, MAX_LEN, (byte) 0x30);
return keyArr;
}
// String -> ByteBuffer
private ByteBuffer getKeyBuf() {
final int MAX_LEN = 9; // key100000
final int keyIdx = next(); final int keyIdx = next();
return db.get(getColumnFamily(), ba("key" + keyIdx)); final String keyStr = "key" + keyIdx;
for (int i = 0; i < keyStr.length(); ++i) {
keyBuf.put(i, (byte) keyStr.charAt(i));
}
for (int i = keyStr.length(); i < MAX_LEN; ++i) {
keyBuf.put(i, (byte) 0x30);
}
// Reset position for future reading
keyBuf.position(0);
return keyBuf;
}
private byte[] getValueArr() {
return valueArr;
}
private ByteBuffer getValueBuf() {
return valueBuf;
}
@Benchmark
public void get() throws RocksDBException {
db.get(getColumnFamily(), getKeyArr());
}
@Benchmark
public void preallocatedGet() throws RocksDBException {
db.get(getColumnFamily(), getKeyArr(), getValueArr());
}
@Benchmark
public void preallocatedByteBufferGet() throws RocksDBException {
int res = db.get(getColumnFamily(), readOptions, getKeyBuf(), getValueBuf());
// For testing correctness:
// assert res > 0;
// final byte[] ret = new byte[valueSize];
// valueBuf.get(ret);
// System.out.println(str(ret));
// valueBuf.flip();
} }
} }

18
java/jmh/src/main/java/org/rocksdb/jmh/MultiGetBenchmarks.java
Unescape View File

@ -194,24 +194,6 @@ public class MultiGetBenchmarks {
return new ArrayList<>(); return new ArrayList<>();
} }
@Benchmark
public List<RocksDB.MultiGetInstance> multiGetDirect10() throws RocksDBException {
final int fromKeyIdx = next(multiGetSize, keyCount);
if (fromKeyIdx >= 0) {
final List<ByteBuffer> keys = keys(keyBuffersList, fromKeyIdx, fromKeyIdx + multiGetSize);
final List<RocksDB.MultiGetInstance> results = db.multiGetByteBuffers(
keys, valueBuffersList.subList(fromKeyIdx, fromKeyIdx + multiGetSize));
for (final RocksDB.MultiGetInstance result : results) {
if (result.status.getCode() != Status.Code.Ok)
throw new RuntimeException("Test status assumption wrong");
if (result.valueSize != valueSize)
throw new RuntimeException("Test valueSize assumption wrong");
}
return results;
}
return new ArrayList<>();
}
public static void main(final String[] args) throws RunnerException { public static void main(final String[] args) throws RunnerException {
final org.openjdk.jmh.runner.options.Options opt = final org.openjdk.jmh.runner.options.Options opt =
new OptionsBuilder() new OptionsBuilder()

4
java/pom.xml.template
Unescape View File

@ -59,8 +59,8 @@
</mailingLists> </mailingLists>
<properties> <properties>
<project.build.source>1.7</project.build.source> <project.build.source>1.8</project.build.source>
<project.build.target>1.7</project.build.target> <project.build.target>1.8</project.build.target>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding> <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
</properties> </properties>

53
java/rocksjni/rocksjni.cc
Unescape View File

@ -1060,15 +1060,14 @@ jint rocksdb_get_helper_direct(
ROCKSDB_NAMESPACE::Slice key_slice(key, jkey_len); ROCKSDB_NAMESPACE::Slice key_slice(key, jkey_len);
// TODO(yhchiang): we might save one memory allocation here by adding ROCKSDB_NAMESPACE::PinnableSlice pinnable_value;
// a DB::Get() function which takes preallocated jbyte* as input.
std::string cvalue;
ROCKSDB_NAMESPACE::Status s; ROCKSDB_NAMESPACE::Status s;
if (column_family_handle != nullptr) { if (column_family_handle != nullptr) {
s = db->Get(read_options, column_family_handle, key_slice, &cvalue); s = db->Get(read_options, column_family_handle, key_slice, &pinnable_value);
} else { } else {
// backwards compatibility // backwards compatibility
s = db->Get(read_options, key_slice, &cvalue); s = db->Get(read_options, db->DefaultColumnFamily(), key_slice,
&pinnable_value);
} }
if (s.IsNotFound()) { if (s.IsNotFound()) {
@ -1088,13 +1087,14 @@ jint rocksdb_get_helper_direct(
return kStatusError; return kStatusError;
} }
const jint cvalue_len = static_cast<jint>(cvalue.size()); const jint pinnable_value_len = static_cast<jint>(pinnable_value.size());
const jint length = std::min(jval_len, cvalue_len); const jint length = std::min(jval_len, pinnable_value_len);
memcpy(value, cvalue.c_str(), length); memcpy(value, pinnable_value.data(), length);
pinnable_value.Reset();
*has_exception = false; *has_exception = false;
return cvalue_len; return pinnable_value_len;
} }
/* /*
@ -1425,13 +1425,13 @@ jbyteArray rocksdb_get_helper(
ROCKSDB_NAMESPACE::Slice key_slice(reinterpret_cast<char*>(key), jkey_len); ROCKSDB_NAMESPACE::Slice key_slice(reinterpret_cast<char*>(key), jkey_len);
std::string value; ROCKSDB_NAMESPACE::PinnableSlice pinnable_value;
ROCKSDB_NAMESPACE::Status s; ROCKSDB_NAMESPACE::Status s;
if (column_family_handle != nullptr) { if (column_family_handle != nullptr) {
s = db->Get(read_opt, column_family_handle, key_slice, &value); s = db->Get(read_opt, column_family_handle, key_slice, &pinnable_value);
} else { } else {
// backwards compatibility s = db->Get(read_opt, db->DefaultColumnFamily(), key_slice,
s = db->Get(read_opt, key_slice, &value); &pinnable_value);
} }
// cleanup // cleanup
@ -1442,7 +1442,9 @@ jbyteArray rocksdb_get_helper(
} }
if (s.ok()) { if (s.ok()) {
jbyteArray jret_value = ROCKSDB_NAMESPACE::JniUtil::copyBytes(env, value); jbyteArray jret_value =
ROCKSDB_NAMESPACE::JniUtil::copyBytes(env, pinnable_value);
pinnable_value.Reset();
if (jret_value == nullptr) { if (jret_value == nullptr) {
// exception occurred // exception occurred
return nullptr; return nullptr;
@ -1551,15 +1553,13 @@ jint rocksdb_get_helper(
} }
ROCKSDB_NAMESPACE::Slice key_slice(reinterpret_cast<char*>(key), jkey_len); ROCKSDB_NAMESPACE::Slice key_slice(reinterpret_cast<char*>(key), jkey_len);
// TODO(yhchiang): we might save one memory allocation here by adding ROCKSDB_NAMESPACE::PinnableSlice pinnable_value;
// a DB::Get() function which takes preallocated jbyte* as input.
std::string cvalue;
ROCKSDB_NAMESPACE::Status s; ROCKSDB_NAMESPACE::Status s;
if (column_family_handle != nullptr) { if (column_family_handle != nullptr) {
s = db->Get(read_options, column_family_handle, key_slice, &cvalue); s = db->Get(read_options, column_family_handle, key_slice, &pinnable_value);
} else { } else {
// backwards compatibility s = db->Get(read_options, db->DefaultColumnFamily(), key_slice,
s = db->Get(read_options, key_slice, &cvalue); &pinnable_value);
} }
// cleanup // cleanup
@ -1582,12 +1582,13 @@ jint rocksdb_get_helper(
return kStatusError; return kStatusError;
} }
const jint cvalue_len = static_cast<jint>(cvalue.size()); const jint pinnable_value_len = static_cast<jint>(pinnable_value.size());
const jint length = std::min(jval_len, cvalue_len); const jint length = std::min(jval_len, pinnable_value_len);
env->SetByteArrayRegion( env->SetByteArrayRegion(jval, jval_off, length,
jval, jval_off, length, const_cast<jbyte*>(reinterpret_cast<const jbyte*>(
const_cast<jbyte*>(reinterpret_cast<const jbyte*>(cvalue.c_str()))); pinnable_value.data())));
pinnable_value.Reset();
if (env->ExceptionCheck()) { if (env->ExceptionCheck()) {
// exception thrown: OutOfMemoryError // exception thrown: OutOfMemoryError
*has_exception = true; *has_exception = true;
@ -1595,7 +1596,7 @@ jint rocksdb_get_helper(
} }
*has_exception = false; *has_exception = false;
return cvalue_len; return pinnable_value_len;
} }
/* /*

Write Preview
Loading…
Cancel
Save