Skip to content

Latest commit

 

History

History
122 lines (79 loc) · 7.62 KB

k8s-storage.md

File metadata and controls

122 lines (79 loc) · 7.62 KB

Kubernetes Storage Options

Use this data to assist in your CSI decision. However, when using a cloud provider we suggest you use their Kubernetes CSI.

Feature Matrix

Product BB Compatible License Type In Ironbank RWX/RWM Support Airgap Compatible Cloud Agnostic
Amazon EBS CSI X Apache License 2.0 X AWS Dependent No
Azure Disk CSI Not Tested Apache License 2.0 X Azure Dependent No
Longhorn v1.1.0 X Apache License 2.0 X X - Docs Yes, uses host storage
OpenEBS (jiva) X Apache License 2.0 X Alpha Manual Work Required Yes, uses host storage
Rook-Ceph X Rook - Apache License 2.0. Ceph - dual licensed under the LGPL version 2.1 or 3.0 X Manual Work Required Yes, uses host storage
Portworx X Tiered License - See website X X - Docs Yes, uses host storage

Benchmark Results

Benchmarks were tested on AWS with GP2 ebs volumes using using FIO, see example

Product Random Read/Write IOPS Average Latency (usec) Sequential Read/Write Mixed Random Read/Write IOPS
Amazon EBS CSI 2997/2996. BW: 128MiB/s / 128MiB/s 1331.61 129MiB/s / 131MiB/s 7203/2390
Azure Disk CSI
Longhorn v1.1.0 6155/1551 BW: 230MiB/s / 96.3MiB/s 1042.53 319MiB/s / 130MiB/s 3804/1267
OpenEBS (jiva) 2183/770. BW: 76.8MiB/s / 45.8MiB/s 2059.55 132MiB/s / 98.2MiB/s 1590/533
Rook-Ceph 10.7k/3205. BW: 503MiB/s / 148MiB/s 548.36/s 496MiB/s / 154MiB/s 6664/2228
Portworx 2.6 3016/19.3k. BW: 74.5MiB/s / 85.1MiB/s 1337.31 113MiB/s / 124MiB/s 35.1k/11.1k

Amazon EBS CSI

Website/Docs

Requirements

  • Must be using AWS

Notes

  • Apache License 2.0
  • Very easy to install and use, apply CSI spec and you are ready.

Azure Disk CSI

Website/Docs

Requirements

  • Must be using Azure

Notes

  • Apache License 2.0
  • Very easy to install and use, apply CSI spec and you are ready.

Longhorn

Website/Docs

Requirements

Notes

  • Apache License 2.0
  • Easiest to install
  • Built-in backup tool.
  • Documented airgap install process. Docs
  • GUI provides data and observability; replica status, cluster health status, backup status, and backup initiation/recovery.
  • Native backup to S3 or NFS

OpenEBS

Website/Docs

Requirements

  • Blank and un-partitioned attached disk(s)
  • RWX is in Alpha and requires work. OpenEBS RWX Docs

Notes

  • Very flexible, supports multiple storage designs.
Application requirements Storage Type OpenEBS Volumes
Low Latency, High Availability, Synchronous replication, Snapshots, Clones, Thin provisioning SSDs/Cloud Volumes OpenEBS Mayastor
High Availability, Synchronous replication, Snapshots, Clones, Thin provisioning Disks/SSDs/Cloud Volumes OpenEBS cStor
High Availability, Synchronous replication, Thin provisioning hostpath or external mounted storage OpenEBS Jiva
Low latency, Local PV hostpath or external mounted storage Dynamic Local PV - Hostpath
Low latency, Local PV Disks/SSDs/Cloud Volumes Dynamic Local PV - Device
Low latency, Local PV, Snapshots, Clones Disks/SSDs/Cloud Volumes OpenEBS Dynamic Local PV - ZFS

Rook-Ceph

Website/Docs

Requirements

  • Blank and un-partitioned attached disk(s)

Notes

  • Rook - Apache License 2.0.
  • Ceph - dual licensed under the LGPL version 2.1 or 3.0
  • Very Fast

Portworx

Website/Docs

Requirements

  • Blank and un-partitioned attached disk(s)

Notes

  • Portworx Essentials is free up to 5nodes, 5TB Storage, 500 volumes
  • Portworx Enterprise and PX-Backup require paid licenses
  • Best Mixed IOPS, average read/write performance
  • Install is very picky about the container runtime hostpath
  • Tested on Konvoy 1.6.1 due to Portworx issues when using RKE2