From 122561e45605d62dd2b62d853e220d80360503b2 Mon Sep 17 00:00:00 2001
From: Wei Fu <weifu@microsoft.com>
Date: Wed, 31 Jan 2024 09:11:47 +0000
Subject: [PATCH] virtualcluster: add README.md for nodepool command

Signed-off-by: Wei Fu <weifu@microsoft.com>
---
 cmd/kperf/commands/virtualcluster/README.md | 77 +++++++++++++++++++++
 1 file changed, 77 insertions(+)
 create mode 100644 cmd/kperf/commands/virtualcluster/README.md

diff --git a/cmd/kperf/commands/virtualcluster/README.md b/cmd/kperf/commands/virtualcluster/README.md
new file mode 100644
index 0000000..5967ab7
--- /dev/null
+++ b/cmd/kperf/commands/virtualcluster/README.md
@@ -0,0 +1,77 @@
+# virtualcluster subcommand
+
+## nodepool subcommand
+
+The `nodepool` subcmd is using [kwok](https://github.com/kubernetes-sigs/kwok) to
+deploy virtual nodepool. The user can use few physical resources to simulate
+more than 1,000 nodes scenario.
+
+The kperf uses `virtualnodes-kperf-io` namespace to host resources related to
+nodepool.
+
+If the user want to schedule pods to virtual nodes, the user needs to change
+node affinity and tolerations.
+
+```YAML
+affinity:
+  nodeAffinity:
+    requiredDuringSchedulingIgnoredDuringExecution:
+      nodeSelectorTerms:
+      - matchExpressions:
+        - key: type
+          operator: In
+          values:
+          - kperf-virtualnodes
+
+tolerations:
+- key: "kperf.io/nodepool"
+  operator: "Exists"
+  effect: "NoSchedule"
+```
+
+Be default, the pod created by job controller will be completed after 5 seconds.
+Other pods will be long running until receiving delete event.
+
+### add - add a set of nodes with the same setting
+
+We can use the following command to add nodepool named by `example` with 100 nodes.
+
+```bash
+$ # cd kperf repo
+$ # please build binary by make build
+$
+$ bin/kperf vc nodepool add example \
+  --nodes=100 --cpu=32 --memory=96 --max-pods=50 \
+  --affinity="node.kubernetes.io/instance-type=Standard_D16s_v3"
+```
+
+The `--affinity` is used to deploy node controller (kwok) to nodes with the
+specific labels.
+
+### list - list all the existing nodepools created by kperf
+
+```bash
+$ # cd kperf repo
+$ # please build binary by make build
+$
+$ bin/kperf vc nodepool list
+NAME         NODES     CPU   MEMORY (GiB)   MAX PODS   STATUS
+example      ? / 100   32    96             50         deployed
+example-v2   ? / 10    8     16             130        deployed
+```
+
+> NOTE: There is TODO item to show the number of ready nodes. Before that, we
+use `?` as read nodes.
+
+### delete - delete the target nodepool
+
+```bash
+$ # cd kperf repo
+$ # please build binary by make build
+$
+$ bin/kperf vc nodepool delete example
+$
+$ bin/kperf vc nodepool list
+NAME         NODES    CPU   MEMORY (GiB)   MAX PODS   STATUS
+example-v2   ? / 10   8     16             130        deployed
+```