Skip to content

Latest commit

 

History

History
 
 

multi

README.md

Akash: Multi-Node Local Setup

This testing harness sets up a minikube environment running multiple Akash nodes and providers with kubernetes.

The kubernetes configuration is managed by two helm charts: akash-node and akash-provider.

Running through the entire suite requires two terminals. Each command is marked t1-t2 to indicate a suggested terminal number.

Logging of nodes and providers can easily be obtained by using kail

Example snippets for working with this environment can be found below.

Dependencies

Ensure that you have installed the base dependencies and have set GOPATH as described here. Then install these additional dependencies before continuing:

Setup

t1: Start minikube

$ cd $GOPATH/src/github.com/ovrclk/akash/_run/multi
$ minikube start --cpus 4 --memory 4096
$ minikube addons enable ingress
$ minikube addons enable metrics-server
$ kubectl create -f rbac.yml

t1: Initialize helm

$ helm init

t1: Build, push docker image into minikube

Creates 6 providers: us-west-1, us-west-2, us-east-1, us-east-2, ap-southeast-1, ap-southeast-2

$ make image-minikube

t1: Generate genesis and config

run.sh wraps various shell commands to make the prototype eaiser to run. The init command makes two wallets (named master and other), a genesis configuration giving master all the tokens, and configuration for four nodes located in data/node/

$ make # make sure to compile latest akash binaries
$ ./run.sh init

Start Network

t1: Deploy Akash nodes

$ make helm-install-nodes

t1: Wait for blocks to be created

Repeat akash status as needed until first block is created

$ source env.sh
$ akash status

Transfer Tokens from Master Account to Other Account

t1: Query master account

Checks master's token balance

$ ./run.sh query master

t1: Send tokens to other account

Sends 100 tokens to other

$ ./run.sh send

t1: Query master account

Checks master's token balance to verify send (for demo purposes)

$ ./run.sh query master

t1: Query other account

Checks other's token balance to verify receipt (for demo purposes)

$ ./run.sh query other

Marketplace

t2: Start marketplace monitor in terminal 2

Starts marketplace and prints marketplace transaction log for visibility into the goings-on

$ ./run.sh marketplace

t1: Run providers

Creates providers (datacenters). Check the markeplace monitor to see them come online

$ make helm-install-providers

t1: Create Deployment

Creates a deployment for the master acct using the sample deployment.yml file. Then:

  • orders are then created from deployments,
  • providers bid on them using fulfillments, which are printed in the format deployment-address/group-id/order-id/provider-address, along with bid price,
  • a lease is awarded to the lowest bid provider and printed
  • the manifest file is then automatically sent to the winning provider
$ akash deployment create deployment.yml -k master

t1: Check/View deployed app

Quick check to see that the sample app has been automatically deployed, then take a look at the sample app

$ curl -I hello.$(minikube ip).nip.io
$ open http://hello.$(minikube ip).nip.io

t1: Create, deploy, view a second app in a different region

Copies deployment.yml to world.yml, replacing the "hello" subdomains with "world" subdomains and the us-west region with ap-southeast. Then sends the deployment and checks the sample app as before

$ source env.sh
$ sed -e 's/hello/world/g' -e 's/us-west/ap-southeast/g' -e 's/westcoast/singapore/g'< deployment.yml > world.yml
$ akash deployment create world.yml -k master
$ curl -I world.$(minikube ip).nip.io
$ open http://world.$(minikube ip).nip.io

Shutdown

t1: Delete minikube

Deletes minikube to conserve resources on your local machine

$ minikube delete

t2: Shut down marketplace

Please do not judge our lack of elegance. It's a prototype

$ ^c

Tinkering

Kubernetes/Helm

Get logs from all deployments

$ kail -l akash.network=true

Get logs from all nodes

$ kail -l akash.network/component=akashd

Get logs from all providers

$ kail -l akash.network/component=provider

Check status of node node-0:

$ make helm-check-node-node-0

Check status of all nodes:

$ make helm-check-nodes

Create/upgrade/delete/reset a single node (node-0)

$ make helm-install-node-node-0
$ make helm-upgrade-node-node-0
$ make helm-delete-node-node-0
$ make helm-reset-node-node-0

Check status of provider us-west-1:

$ make helm-check-provider-us-west-1
$ curl us-west-1.$(minikube ip).nip.io/status

Check status of all providers:

$ make helm-check-providers

Create/upgrade/delete/reset a provider in region us-central (name: us-central-1)

$ make helm-install-provider-us-central-1
$ make helm-upgrade-provider-us-central-1
$ make helm-delete-provider-us-central-1
$ make helm-reset-provider-us-central-1

Akash API

Note that you must load shell helpers as shown below into any terminal in which you wish to run akash commands

Load shell helpers

$ source env.sh
# Needed to run akash commands

List deployments

$ akash query deployment

Other list options

$ akash query -h

Close deployment

$ akash deployment close <deployment-id> -k master

Close all active deployments

$ akash query deployment | \
  jq -r '.items[]|select(has("state")|not)|.address' | \
  while read id; do
    echo "closing deployment $id..."
    akash deployment close "$id" -k master
  done