Skip to content

Commit

Permalink
Merge branch 'unstable' into formart_code
Browse files Browse the repository at this point in the history
  • Loading branch information
AlexStocks authored Apr 3, 2024
2 parents 866b2a4 + fe91578 commit 464c16c
Show file tree
Hide file tree
Showing 20 changed files with 673 additions and 88 deletions.
14 changes: 12 additions & 2 deletions CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -19,11 +19,21 @@ IF (CMAKE_CXX_COMPILER_ID STREQUAL "Clang")
ENDIF ()
ELSEIF (CMAKE_CXX_COMPILER_ID STREQUAL "GNU")
# using GCC
IF (CMAKE_CXX_COMPILER_VERSION VERSION_LESS "10.0")
MESSAGE(FATAL_ERROR "GCC G++ version must be greater than 10.0")
IF (CMAKE_CXX_COMPILER_VERSION VERSION_LESS "11.0")
MESSAGE(FATAL_ERROR "GCC G++ version must be greater than 11.0")
ENDIF ()
ENDIF ()

# get current date and time
EXECUTE_PROCESS(COMMAND date "+%Y-%m-%d_%H:%M:%S" OUTPUT_VARIABLE BUILD_TIMESTAMP OUTPUT_STRIP_TRAILING_WHITESPACE)
ADD_DEFINITIONS(-DKPIKIWIDB_BUILD_DATE="${BUILD_TIMESTAMP}")

message(STATUS "Build timestamp: ${BUILD_TIMESTAMP}")

# get git commit id
EXECUTE_PROCESS(COMMAND git rev-parse HEAD OUTPUT_VARIABLE GIT_COMMIT_ID OUTPUT_STRIP_TRAILING_WHITESPACE)
ADD_DEFINITIONS(-DKPIKIWIDB_GIT_COMMIT_ID="${GIT_COMMIT_ID}")
MESSAGE(STATUS "Git commit id: ${GIT_COMMIT_ID}")

############# You should enable sanitizer if you are developing pika #############
# Uncomment the following two lines to enable AddressSanitizer to detect memory leaks and other memory-related bugs.
Expand Down
34 changes: 32 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,12 +2,42 @@
![](docs/images/pikiwidb-logo.png)
[中文](README_CN.md)

A C++11 implementation of Redis Server, use RocksDB for persist storage.(not including cluster yet)
A C++20 implementation of Redis Server, use RocksDB for persist storage.(not including cluster yet)

## Requirements
* C++11

* C++20
* Linux or OS X

## compile

**It is recommended to use the latest version of Ubuntu or Debian for Linux systems**

Execute compilation

If the machine's GCC version is less than 11, especially on CentOS6 or CentOS7, you need to upgrade the gcc version firstly.

Execute the following commands on CentOS:

```bash
sudo yum -y install centos-release-scl
sudo yum -y install devtoolset-11-gcc devtoolset-11-gcc-c++
scl enable devtoolset-11 bash
```

Execute this command to start compiling Pikiwidb:

```bash
./build.sh
```

Pikiwidb is compiled by default in release mode, which does not support debugging. If debugging is needed, compile in debug mode.

```bash
./clean.sh
./build.sh --debug
```

## Support module for write your own extensions
PikiwiDB supports module now, still in progress, much work to do.
I added three commands(ldel, skeys, hgets) for demonstration.
Expand Down
83 changes: 63 additions & 20 deletions README_CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,79 +2,122 @@
![](docs/images/pikiwidb-logo.png)
[Click me switch to English](README.en.md)

C++11实现的增强版Redis服务器,使用RocksDB作为持久化存储引擎。(集群支持尚正在计划中)
C++20 实现的增强版 Redis 服务器,使用 RocksDB 作为持久化存储引擎。(集群支持尚正在计划中)

## 环境需求
* C++11、CMake

* C++20、CMake
* Linux 或 MAC OS

## 与Redis完全兼容
你可以用redis的各种工具来测试PikiwiDB,比如官方的redis-cli, redis-benchmark。
## 编译

**建议使用最新版本的 Ubuntu 或 Debian Linux 系统**

执行编译:

如果机器的 GCC 版本低于 11,特别是在 CentOS 6.x 或 CentOS 7.x 上,你需要先升级 GCC 版本。

在 CentOS 上执行以下命令:

```bash
sudo yum -y install centos-release-scl
sudo yum -y install devtoolset-11-gcc devtoolset-11-gcc-c++
scl enable devtoolset-11 bash
```

执行以下命令开始编译 PikiwiDB:

```bash
./build.sh
```

PikiwiDB 默认以 release 模式编译,不支持调试。如果需要调试,请以 debug 模式编译。

```bash
./clean.sh
./build.sh --debug
```

## 与 Redis 完全兼容

PikiwiDB可以和redis之间进行复制,可以读取redis的rdb文件或aof文件。当然,PikiwiDB生成的aof或rdb文件也可以被redis读取
你可以用 Redis 的各种工具来测试 PikiwiDB,比如官方的 redis-cli, redis-benchmark

你还可以用redis-sentinel来实现PikiwiDB的高可用!
PikiwiDB 可以和 Redis 之间进行复制,可以读取 Redis 的 rdb 文件或 aof 文件。当然,PikiwiDB 生成的 aof 或 rdb 文件也可以被 Redis 读取。

总之,PikiwiDB与Redis完全兼容。
你还可以用 redis-sentinel 来实现 PikiwiDB 的高可用!

总之,PikiwiDB 与 Redis 完全兼容。

## 高性能
- PikiwiDB性能大约比Redis3.2高出20%(使用redis-benchmark测试pipeline请求,比如设置-P=50或更高)
- PikiwiDB的高性能有一部分得益于独立的网络线程处理IO,因此和redis比占了便宜。但PikiwiDB逻辑仍然是单线程的。
- 另一部分得益于C++ STL的高效率(CLANG的表现比GCC更好)。
- 在测试前,你要确保std::list的size()是O(1)复杂度,这才遵循C++11的标准。否则list相关命令不可测。

运行下面这个命令,试试和redis比一比~
- PikiwiDB 性能大约比 Redis 3.2 高出 20% (使用 redis-benchmark 测试 pipeline 请求,比如设置 -P=50 或更高)
- PikiwiDB 的高性能有一部分得益于独立的网络线程处理 IO,因此和 redis 比占了便宜。但 PikiwiDB 逻辑仍然是单线程的。
- 另一部分得益于 C++ STL 的高效率(CLANG 的表现比 GCC 更好)。
- 在测试前,你要确保 std::list 的 size() 是 O(1) 复杂度,这才遵循 C++11 的标准。否则 list 相关命令不可测。

运行下面这个命令,试试和 redis 比一比~
```bash
./redis-benchmark -q -n 1000000 -P 50 -c 50
```

## 编写扩展模块
PikiwiDB支持动态库模块,可以在运行时添加新命令。
我添加了三个命令(ldel, skeys, hgets)作为演示。

## 支持冷数据淘汰
是的,在内存受限的情况下,你可以让PikiwiDB根据简单的LRU算法淘汰一些key以释放内存。

是的,在内存受限的情况下,你可以让 PikiwiDB 根据简单的 LRU 算法淘汰一些 key 以释放内存。

## 主从复制,事务,RDB/AOF持久化,慢日志,发布订阅
这些特性PikiwiDB都有:-)

这些特性 PikiwiDB 都有:-)

## 持久化:内存不再是上限
RocksDB可以配置为PikiwiDB的持久化存储引擎,可以存储更多的数据。

RocksDB 可以配置为 PikiwiDB 的持久化存储引擎,可以存储更多的数据。

## 命令列表
#### 展示PikiwiDB支持的所有命令

#### 展示 PikiwiDB 支持的所有命令

- cmdlist

#### key commands

- type exists del expire pexpire expireat pexpireat ttl pttl persist move keys randomkey rename renamenx scan sort

#### server commands

- select dbsize bgsave save lastsave flushdb flushall client debug shutdown bgrewriteaof ping echo info monitor auth

#### string commands

- set get getrange setrange getset append bitcount bitop getbit setbit incr incrby incrbyfloat decr decrby mget mset msetnx setnx setex psetex strlen

#### list commands

- lpush rpush lpushx rpushx lpop rpop lindex llen lset ltrim lrange linsert lrem rpoplpush blpop brpop brpoplpush

#### hash commands

- hget hmget hgetall hset hsetnx hmset hlen hexists hkeys hvals hdel hincrby hincrbyfloat hscan hstrlen

#### set commands

- sadd scard srem sismember smembers sdiff sdiffstore sinter sinterstore sunion sunionstore smove spop srandmember sscan

#### sorted set commands

- zadd zcard zrank zrevrank zrem zincrby zscore zrange zrevrange zrangebyscore zrevrangebyscore zremrangebyrank zremrangebyscore

#### pubsub commands

- subscribe unsubscribe publish psubscribe punsubscribe pubsub

#### multi commands

- watch unwatch multi exec discard

#### replication commands

- sync slaveof


## Contact Us

Expand Down
16 changes: 1 addition & 15 deletions build.sh
Original file line number Diff line number Diff line change
Expand Up @@ -6,12 +6,6 @@ C_GREEN="\033[32m"

C_END="\033[0m"

BUILD_TIME=$(git log -1 --format=%ai)
BUILD_TIME=${BUILD_TIME: 0: 10}

COMMIT_ID=$(git rev-parse HEAD)
SHORT_COMMIT_ID=${COMMIT_ID: 0: 8}

BUILD_TYPE=release
VERBOSE=0
CMAKE_FLAGS=""
Expand Down Expand Up @@ -66,19 +60,11 @@ fi

echo "cpu core ${CPU_CORE}"

if [ -z "$SHORT_COMMIT_ID" ]; then
echo "no git commit id"
SHORT_COMMIT_ID="pikiwidb"
fi

echo "BUILD_TIME:" $BUILD_TIME
echo "COMMIT_ID:" $SHORT_COMMIT_ID

echo "BUILD_TYPE:" $BUILD_TYPE
echo "CMAKE_FLAGS:" $CMAKE_FLAGS
echo "MAKE_FLAGS:" $MAKE_FLAGS

cmake -DCMAKE_BUILD_TYPE=${BUILD_TYPE} -DBUILD_TIME=$BUILD_TIME -DGIT_COMMIT_ID=$SHORT_COMMIT_ID ${CMAKE_FLAGS} -S . -B ${PREFIX}
cmake -DCMAKE_BUILD_TYPE=${BUILD_TYPE} ${CMAKE_FLAGS} -S . -B ${PREFIX}
cmake --build ${PREFIX} -- ${MAKE_FLAGS} -j ${CPU_CORE}

if [ $? -eq 0 ]; then
Expand Down
10 changes: 5 additions & 5 deletions src/base_cmd.cc
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,8 @@ BaseCmd::BaseCmd(std::string name, int16_t arity, uint32_t flag, uint32_t aclCat
name_ = std::move(name);
arity_ = arity;
flag_ = flag;
aclCategory_ = aclCategory;
cmdId_ = g_pikiwidb->GetCmdTableManager().GetCmdId();
acl_category_ = aclCategory;
cmd_id_ = g_pikiwidb->GetCmdID();
}

bool BaseCmd::CheckArg(size_t num) const {
Expand Down Expand Up @@ -55,13 +55,13 @@ void BaseCmd::SetFlag(uint32_t flag) { flag_ |= flag; }
void BaseCmd::ResetFlag(uint32_t flag) { flag_ &= ~flag; }
bool BaseCmd::HasSubCommand() const { return false; }
BaseCmd* BaseCmd::GetSubCmd(const std::string& cmdName) { return nullptr; }
uint32_t BaseCmd::AclCategory() const { return aclCategory_; }
void BaseCmd::AddAclCategory(uint32_t aclCategory) { aclCategory_ |= aclCategory; }
uint32_t BaseCmd::AclCategory() const { return acl_category_; }
void BaseCmd::AddAclCategory(uint32_t aclCategory) { acl_category_ |= aclCategory; }
std::string BaseCmd::Name() const { return name_; }
// CmdRes& BaseCommand::Res() { return res_; }
// void BaseCommand::SetResp(const std::shared_ptr<std::string>& resp) { resp_ = resp; }
// std::shared_ptr<std::string> BaseCommand::GetResp() { return resp_.lock(); }
uint32_t BaseCmd::GetCmdId() const { return cmdId_; }
uint32_t BaseCmd::GetCmdID() const { return cmd_id_; }

// BaseCmdGroup
BaseCmdGroup::BaseCmdGroup(const std::string& name, uint32_t flag) : BaseCmdGroup(name, -2, flag) {}
Expand Down
6 changes: 3 additions & 3 deletions src/base_cmd.h
Original file line number Diff line number Diff line change
Expand Up @@ -285,7 +285,7 @@ class BaseCmd : public std::enable_shared_from_this<BaseCmd> {
// void SetResp(const std::shared_ptr<std::string>& resp);
// std::shared_ptr<std::string> GetResp();

uint32_t GetCmdId() const;
uint32_t GetCmdID() const;

bool isExclusive() { return static_cast<bool>(flag_ & kCmdFlagsExclusive); }

Expand All @@ -303,8 +303,8 @@ class BaseCmd : public std::enable_shared_from_this<BaseCmd> {
// std::weak_ptr<std::string> resp_;
// uint64_t doDuration_ = 0;

uint32_t cmdId_ = 0;
uint32_t aclCategory_ = 0;
uint32_t cmd_id_ = 0;
uint32_t acl_category_ = 0;

private:
// The function to be executed first before executing `DoCmd`
Expand Down
Loading

0 comments on commit 464c16c

Please sign in to comment.