Java RocksDB安装与应用

时间:2021-05-20

rocksDB 是一个可嵌入的,持久性的 key-value存储。

以下介绍来自rocksDB 中文官网

https://rocksdb.org.cn/

它有以下四个特点

1 高性能:RocksDB使用一套日志结构的数据库引擎,为了更好的性能,这套引擎是用C++编写的。 Key和value是任意大小的字节流。

2为快速存储而优化:RocksDB为快速而又低延迟的存储设备(例如闪存或者高速硬盘)而特殊优化处理。 RocksDB将最大限度的发挥闪存和RAM的高度率读写性能。

3可适配性 :RocksDB适合于多种不同工作量类型。 从像MyRocks这样的数据存储引擎, 到应用数据缓存, 甚至是一些嵌入式工作量,RocksDB都可以从容面对这些不同的数据工作量需求。

4基础和高级的数据库操作RocksDB提供了一些基础的操作,例如打开和关闭数据库。 对于合并和压缩过滤等高级操作,也提供了读写支持。

​​​​​​RockDB 安装与使用

rocksDB 安装有多种方式。由于官方没有提供对应平台的二进制库,所以需要自己编译使用。

rocksDB 的安装很简单,但是需要转变一下对于rocksDB 的看法。它不是一个重量级别的数据库,是一个嵌入式的key-value 存储。这意味着你只要在你的Maven项目中添加 rocksDB的依赖,就可以在开发环境中自我尝试了。如果你没有理解这点,你就可能会走入下面这两种不推荐的安装方式。

方式一去查看rocksDB 的官网 发现要写 一个C++ 程序(不推荐)

#include <assert>#include "rocksdb/db.h"rocksdb::DB* db;rocksdb::Options options;options.create_if_missing = true;rocksdb::Status status = rocksdb::DB::Open(options, "/tmp/testdb", &db);assert(status.ok());

创建一个数据库???? 怎么和之前用的mysql 或者mongo 不一样,为啥没有一个start.sh 或者start.bat 之类的脚本。难道要我写。写完了编译发现还不知道怎么和rocksDB 库进行关联,怎么办,我C++都忘完了。

方式二 使用pyrocksDB (不推荐)

http://pyrocksdb.readthedocs.io/en/latest/installation.html

详细的安装文档见pyrocksDB 的官网安装文档。

以上两种方式对于熟悉C++ 或者python 的开发者来说都比较友好,但对于java 开发者来说不是太友好。

接下来就介绍第三种方式。

方式三 使用maven (推荐)

新建maven 项目,修改pom.xml 依赖里面添加

<dependency> <groupId>org.rocksdb</groupId> <artifactId>rocksdbjni</artifactId> <version>5.8.6</version></dependency>

可以选择你喜欢的版本。

然后更高maven 的语言级别,我这里全局设置为了1.8

<profiles> <profile> <id>jdk18</id> <activation> <activeByDefault>true</activeByDefault> <jdk>1.8</jdk> </activation> <properties> <maven.compiler.source>1.8</maven.compiler.source> <maven.compiler.target>1.8</maven.compiler.target> <maven.compiler.compilerVersion>1.8</maven.compiler.compilerVersion> </properties> </profile></profiles>

到这里,环境就装好了,是不是又回到了熟悉的java 世界。

然后copy 源码包下的一个类,在IDE中修改一下运行配置,加一个程序运行中数据库存储路径,就可以运行测试了 。我会在文章最后给出这个类。

运行控制台会有日志输出,同时也文件中也会出现一下新的文件。

后面会更新更多关于rockDB 开发API 的介绍,以及在生产中的应用,希望大家关注。

// Copyright (c) 2011-present, Facebook, Inc. All rights reserved.// This source code is licensed under both the GPLv2 (found in the// COPYING file in the root directory) and Apache 2.0 License// (found in the LICENSE.Apache file in the root directory).import org.rocksdb.*;import org.rocksdb.util.SizeUnit;import java.util.ArrayList;import java.util.Arrays;import java.util.List;import java.util.Map;public class RocksDBSample { static { RocksDB.loadLibrary(); } public static void main(final String[] args) { if (args.length < 1) { System.out.println("usage: RocksDBSample db_path"); System.exit(-1); } final String db_path = args[0]; final String db_path_not_found = db_path + "_not_found"; System.out.println("RocksDBSample"); try (final Options options = new Options(); final Filter bloomFilter = new BloomFilter(10); final ReadOptions readOptions = new ReadOptions() .setFillCache(false); final Statistics stats = new Statistics(); final RateLimiter rateLimiter = new RateLimiter(10000000,10000, 10)) { try (final RocksDB db = RocksDB.open(options, db_path_not_found)) { assert (false); } catch (final RocksDBException e) { System.out.format("Caught the expected exception -- %s\n", e); } try { options.setCreateIfMissing(true) .setStatistics(stats) .setWriteBufferSize(8 * SizeUnit.KB) .setMaxWriteBufferNumber(3) .setMaxBackgroundCompactions(10) .setCompressionType(CompressionType.SNAPPY_COMPRESSION) .setCompactionStyle(CompactionStyle.UNIVERSAL); } catch (final IllegalArgumentException e) { assert (false); } assert (options.createIfMissing() == true); assert (options.writeBufferSize() == 8 * SizeUnit.KB); assert (options.maxWriteBufferNumber() == 3); assert (options.maxBackgroundCompactions() == 10); assert (options.compressionType() == CompressionType.SNAPPY_COMPRESSION); assert (options.compactionStyle() == CompactionStyle.UNIVERSAL); assert (options.memTableFactoryName().equals("SkipListFactory")); options.setMemTableConfig( new HashSkipListMemTableConfig() .setHeight(4) .setBranchingFactor(4) .setBucketCount(2000000)); assert (options.memTableFactoryName().equals("HashSkipListRepFactory")); options.setMemTableConfig( new HashLinkedListMemTableConfig() .setBucketCount(100000)); assert (options.memTableFactoryName().equals("HashLinkedListRepFactory")); options.setMemTableConfig( new VectorMemTableConfig().setReservedSize(10000)); assert (options.memTableFactoryName().equals("VectorRepFactory")); options.setMemTableConfig(new SkipListMemTableConfig()); assert (options.memTableFactoryName().equals("SkipListFactory")); options.setTableFormatConfig(new PlainTableConfig()); // Plain-Table requires mmap read options.setAllowMmapReads(true); assert (options.tableFactoryName().equals("PlainTable")); options.setRateLimiter(rateLimiter); final BlockBasedTableConfig table_options = new BlockBasedTableConfig(); table_options.setBlockCacheSize(64 * SizeUnit.KB) .setFilter(bloomFilter) .setCacheNumShardBits(6) .setBlockSizeDeviation(5) .setBlockRestartInterval(10) .setCacheIndexAndFilterBlocks(true) .setHashIndexAllowCollision(false) .setBlockCacheCompressedSize(64 * SizeUnit.KB) .setBlockCacheCompressedNumShardBits(10); assert (table_options.blockCacheSize() == 64 * SizeUnit.KB); assert (table_options.cacheNumShardBits() == 6); assert (table_options.blockSizeDeviation() == 5); assert (table_options.blockRestartInterval() == 10); assert (table_options.cacheIndexAndFilterBlocks() == true); assert (table_options.hashIndexAllowCollision() == false); assert (table_options.blockCacheCompressedSize() == 64 * SizeUnit.KB); assert (table_options.blockCacheCompressedNumShardBits() == 10); options.setTableFormatConfig(table_options); assert (options.tableFactoryName().equals("BlockBasedTable")); try (final RocksDB db = RocksDB.open(options, db_path)) { db.put("hello".getBytes(), "world".getBytes()); final byte[] value = db.get("hello".getBytes()); assert ("world".equals(new String(value))); final String str = db.getProperty("rocksdb.stats"); assert (str != null && !str.equals("")); } catch (final RocksDBException e) { System.out.format("[ERROR] caught the unexpected exception -- %s\n", e); assert (false); } try (final RocksDB db = RocksDB.open(options, db_path)) { db.put("hello".getBytes(), "world".getBytes()); byte[] value = db.get("hello".getBytes()); System.out.format("Get('hello') = %s\n", new String(value)); for (int i = 1; i <= 9; ++i) { for (int j = 1; j <= 9; ++j) { db.put(String.format("%dx%d", i, j).getBytes(), String.format("%d", i * j).getBytes()); } } for (int i = 1; i <= 9; ++i) { for (int j = 1; j <= 9; ++j) { System.out.format("%s ", new String(db.get( String.format("%dx%d", i, j).getBytes()))); } System.out.println(""); } // write batch test try (final WriteOptions writeOpt = new WriteOptions()) { for (int i = 10; i <= 19; ++i) { try (final WriteBatch batch = new WriteBatch()) { for (int j = 10; j <= 19; ++j) { batch.put(String.format("%dx%d", i, j).getBytes(), String.format("%d", i * j).getBytes()); } db.write(writeOpt, batch); } } } for (int i = 10; i <= 19; ++i) { for (int j = 10; j <= 19; ++j) { assert (new String( db.get(String.format("%dx%d", i, j).getBytes())).equals( String.format("%d", i * j))); System.out.format("%s ", new String(db.get( String.format("%dx%d", i, j).getBytes()))); } System.out.println(""); } value = db.get("1x1".getBytes()); assert (value != null); value = db.get("world".getBytes()); assert (value == null); value = db.get(readOptions, "world".getBytes()); assert (value == null); final byte[] testKey = "asdf".getBytes(); final byte[] testValue = "asdfghjkl;'?><MNBVCXZQWERTYUIOP{+_)(*&^%$#@".getBytes(); db.put(testKey, testValue); byte[] testResult = db.get(testKey); assert (testResult != null); assert (Arrays.equals(testValue, testResult)); assert (new String(testValue).equals(new String(testResult))); testResult = db.get(readOptions, testKey); assert (testResult != null); assert (Arrays.equals(testValue, testResult)); assert (new String(testValue).equals(new String(testResult))); final byte[] insufficientArray = new byte[10]; final byte[] enoughArray = new byte[50]; int len; len = db.get(testKey, insufficientArray); assert (len > insufficientArray.length); len = db.get("asdfjkl;".getBytes(), enoughArray); assert (len == RocksDB.NOT_FOUND); len = db.get(testKey, enoughArray); assert (len == testValue.length); len = db.get(readOptions, testKey, insufficientArray); assert (len > insufficientArray.length); len = db.get(readOptions, "asdfjkl;".getBytes(), enoughArray); assert (len == RocksDB.NOT_FOUND); len = db.get(readOptions, testKey, enoughArray); assert (len == testValue.length); db.remove(testKey); len = db.get(testKey, enoughArray); assert (len == RocksDB.NOT_FOUND); // repeat the test with WriteOptions try (final WriteOptions writeOpts = new WriteOptions()) { writeOpts.setSync(true); writeOpts.setDisableWAL(true); db.put(writeOpts, testKey, testValue); len = db.get(testKey, enoughArray); assert (len == testValue.length); assert (new String(testValue).equals( new String(enoughArray, 0, len))); } try { for (final TickerType statsType : TickerType.values()) { if (statsType != TickerType.TICKER_ENUM_MAX) { stats.getTickerCount(statsType); } } System.out.println("getTickerCount() passed."); } catch (final Exception e) { System.out.println("Failed in call to getTickerCount()"); assert (false); //Should never reach here. } try { for (final HistogramType histogramType : HistogramType.values()) { if (histogramType != HistogramType.HISTOGRAM_ENUM_MAX) { HistogramData data = stats.getHistogramData(histogramType); } } System.out.println("getHistogramData() passed."); } catch (final Exception e) { System.out.println("Failed in call to getHistogramData()"); assert (false); //Should never reach here. } try (final RocksIterator iterator = db.newIterator()) { boolean seekToFirstPassed = false; for (iterator.seekToFirst(); iterator.isValid(); iterator.next()) { iterator.status(); assert (iterator.key() != null); assert (iterator.value() != null); seekToFirstPassed = true; } if (seekToFirstPassed) { System.out.println("iterator seekToFirst tests passed."); } boolean seekToLastPassed = false; for (iterator.seekToLast(); iterator.isValid(); iterator.prev()) { iterator.status(); assert (iterator.key() != null); assert (iterator.value() != null); seekToLastPassed = true; } if (seekToLastPassed) { System.out.println("iterator seekToLastPassed tests passed."); } iterator.seekToFirst(); iterator.seek(iterator.key()); assert (iterator.key() != null); assert (iterator.value() != null); System.out.println("iterator seek test passed."); } System.out.println("iterator tests passed."); final List<byte[]> keys = new ArrayList<>(); try (final RocksIterator iterator = db.newIterator()) { for (iterator.seekToLast(); iterator.isValid(); iterator.prev()) { keys.add(iterator.key()); } } Map<byte[], byte[]> values = db.multiGet(keys); assert (values.size() == keys.size()); for (final byte[] value1 : values.values()) { assert (value1 != null); } values = db.multiGet(new ReadOptions(), keys); assert (values.size() == keys.size()); for (final byte[] value1 : values.values()) { assert (value1 != null); } } catch (final RocksDBException e) { System.err.println(e); } } }}

以上就是本次给大家介绍的Java中RocksDB安装与应用的全部内容,如果大家在学习后还有任何不明白的可以在下方的留言区域讨论,感谢对的支持。

声明:本页内容来源网络,仅供用户参考;我单位不保证亦不表示资料全面及准确无误,也不保证亦不表示这些资料为最新信息,如因任何原因,本网内容或者用户因倚赖本网内容造成任何损失或损害,我单位将不会负任何法律责任。如涉及版权问题,请提交至online#300.cn邮箱联系删除。

相关文章