Skip to content

Commit

Permalink
feat: update mirroring contents
Browse files Browse the repository at this point in the history
  • Loading branch information
@fraidev
fraidev committed Jun 3, 2024
1 parent ad004ef commit 9d64f51
Show file tree
Hide file tree
Showing 2 changed files with 72 additions and 64 deletions.
62 changes: 32 additions & 30 deletions docs/tutorials/iot-mirroring-cloud.mdx
View file
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ Download and install mirroring binary.
Use `curl` to download and install:

```bash
curl -fsS https://hub.infinyon.cloud/install/install.sh | VERSION='0.10.15-dev-2+mirroring-9961bdb' bash
curl -fsS https://hub.infinyon.cloud/install/install.sh | bash
```

Make sure to add `.fluvio/bin` to the `$PATH`as specified in the installation script.
Expand All @@ -87,10 +87,10 @@ Let's provision a new cluster in AWS `eu-central` using the experimental fluvio


```bash
fluvio cloud cluster create --region aws-eu-central-1 --version 0.10.15-dev-2+mirroring-b4f07fc
fluvio cloud cluster create
```

Check cluster status:
Check the result with:


```bash
Expand All @@ -99,6 +99,14 @@ fluvio cluster status

Next, we'll configure the cluster to receive traffic from the edge clusters.

### Register Edge cluster

Let's register the edge cluster `edge1` to inform our Cloud cluster to accept connection requests from the remote device:


```bash
fluvio remote register edge1
```

### Create the mirror topic

Expand All @@ -109,15 +117,15 @@ Create a partition assignment file with an array of edge mirros we expect to con

```bash
echo '[
"edge1", "edge2"
"edge1"
]' > assignment_file.json
```

Apply the configuration file to create the topic:


```bash
fluvio topic create edge-topic --mirror-assignment assignment_file.json
fluvio topic create edge-topic --mirror-apply assignment_file.json
```

List partitions to check the assignment:
Expand All @@ -131,34 +139,21 @@ It should display all partitions:

```bash
TOPIC PARTITION LEADER MIRROR REPLICAS RESOLUTION SIZE HW LEO LRS FOLLOWER OFFSETS
edge-topic 0 5001 edge1 [] Online 0 B 0 0 0 0 []
edge-topic 1 5001 edge2 [] Online 0 B 0 0 0 0 []
```

We created 2 partitions, but we'll only use one in this tutorial.


### Register Edge cluster

Let's register the edge cluster `edge1` to inform our Cloud cluster to accept connection requests from the remote device:


```bash
fluvio cluster remote-cluster register --type mirror-edge edge1
edge-topic 0 5001 edge1 [] Online 0 B 0 0 0 0 []
```

List remote clusters to check their status:


```bash
fluvio cluster remote-cluster list
fluvio remote list
```

It should show the following:

```bash
RemoteCluster RemoteType Paired Status Last Seen
edge1 mirror-edge - - -
RemoteCluster Status Last Seen
edge1 Waiting -
```


Expand All @@ -178,10 +173,10 @@ Each edge cluster requires a unique metadata file that informs the edge cluster


```bash
fluvio cluster remote-cluster metadata export --topic edge-topic --mirror edge1 --file edge1.json
fluvio cloud remote export --remote edge1 --file edge1.json
```

The Cloud cluster configuration is now complete. Next, we'll create an edge cluster and configure a mirror topic that synchronizes data to the Cloud.
The Cloud cluster configuration is now complete. Next, we'll create an edge cluster that synchronizes data to the Cloud.


## Install Edge Cluster on Local VM
Expand Down Expand Up @@ -214,7 +209,7 @@ Download binaries:


```bash
curl -fsS https://hub.infinyon.cloud/install/install.sh | VERSION='0.10.15-dev-2+mirroring-9961bdb' bash
curl -fsS https://hub.infinyon.cloud/install/install.sh | bash
```

Add to path:
Expand All @@ -235,11 +230,18 @@ fluvio version

### Start Edge Cluster

We'll use the metadata `edge1` to start the edge cluster:
First we will start the cluster:


```bash
fluvio cluster start
```


Then, we'll connect to the cloud with the metadata `edge1`:

```bash
fluvio cluster start --read-only edge1.json
fluvio home connect --file edge1.json
```

Let's check the partitions:
Expand All @@ -253,7 +255,7 @@ The edge device should show the following partition:

```bash
TOPIC PARTITION LEADER MIRROR REPLICAS RESOLUTION SIZE HW LEO LRS FOLLOWER OFFSETS
edge-topic 0 5001 upstream:0 [] Online 0 B 0 0 0 0 []
edge-topic 0 5001 0:public_endpoint [] Online 0 B 0 0 0 0 []
```

---
Expand Down Expand Up @@ -361,7 +363,7 @@ Restart the cluster:


```bash
fluvio cluster upgrade --read-only edge1.json
fluvio cluster upgrade
```

### Consume from edge cluster
Expand Down Expand Up @@ -394,4 +396,4 @@ Join us on [Discord] if you have questions, or would like to suggest new improve
* ["IoT Mirroring - Raspberry Pi to a Local Cluster"]


["IoT Mirroring - Raspberry Pi to a Local Cluster"]: /tutorials/iot-mirroring-local
["IoT Mirroring - Raspberry Pi to a Local Cluster"]: /docs/tutorials/iot-mirroring-local.mdx
74 changes: 40 additions & 34 deletions docs/tutorials/iot-mirroring-local.mdx
View file
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ title: "IoT Mirroring - Local"
description: "Reliable IoT monitoring from movable or static Edges sensors."
---

This advanced tutorial reuqires a Raspberry Pi and a local installation of your collector cluster running on Kubernetes.
This advanced tutorial requires a Raspberry Pi and a local installation of your collector cluster running on Kubernetes.
Checkout the basic version at: ["IoT Mirroring - Cloud"][iot-mirroring-cloud].

## Raspberry Pi to Local Cluster
Expand All @@ -13,7 +13,7 @@ This section will use Raspberry Pi v3 running Ubuntu 32-bit as the edge device a
Let's start with installing and configuring the target cluster.


## Install Target Cluster on Local Machine
## Install Target Cluster on Local Machine

Installing the target cluster on Linux or Mac requires Kubernetes. Use the following instructions to set up Kubernetes on your local machine.

Expand Down Expand Up @@ -47,7 +47,7 @@ Use `curl` to download and install:


```bash
curl -fsS https://hub.infinyon.cloud/install/install.sh | VERSION='0.10.15-dev-2+mirroring-9961bdb' bash
curl -fsS https://hub.infinyon.cloud/install/install.sh | bash
```

Make sure to add `.fluvio/bin` to the `$PATH`as specified in the installation script.
Expand All @@ -69,6 +69,23 @@ Check the result with:
fluvio cluster status
```

### Register Edge clusters

Use the `remote-cluster` CLI to register the edge clusters (edge1 and edge2) with the upstream cluster:

**Edge 1**:


```bash
fluvio cluster remote-cluster register edge1
```

**Edge 2**:


```bash
fluvio cluster remote-cluster register edge2
```

### Create the mirror topic

Expand All @@ -87,7 +104,7 @@ Apply the configuration file to create the topic:


```bash
fluvio topic create edge-topic --mirror-assignment assignment_file.json
fluvio topic create edge-topic --mirror-apply assignment_file.json
```

List partitions to check the assignment:
Expand All @@ -105,37 +122,19 @@ It should display all partitions:
edge-topic 1 5001 edge2 [] Online 0 B 0 0 0 0 []
```

### Register Edge clusters

Use the `remote-cluster` CLI to register the edge clusters (edge1 and edge2) with the upstream cluster:

**Edge 1**:


```bash
fluvio cluster remote-cluster register --type mirror-edge edge1
```

**Edge 2**:


```bash
fluvio cluster remote-cluster register --type mirror-edge edge2
```

List remote clusters to check their status:


```bash
fluvio cluster remote-cluster list
fluvio remote list
```

It should show the following:

```bash
RemoteCluster RemoteType Paired Status Last Seen
edge1 mirror-edge - - -
edge2 mirror-edge - - -
RemoteCluster Status Last Seen
edge1 Waiting -
edge2 Waiting -
```

### Generate Metadata for Edge Clusters
Expand All @@ -150,7 +149,7 @@ The target edge device is a Virtual Machine emulating an IoT device:


```bash
fluvio cluster remote-cluster metadata export --topic edge-topic --mirror edge1 --upstream host.orb.internal --file edge1.json
fluvio remote export edge1 --public-endpoint host.orb.internal --file edge1.json
```

**Edge 2**:
Expand All @@ -163,7 +162,7 @@ The target edge device is a Raspberry Pi device. You may skip this if you don't


```bash
fluvio cluster remote-cluster metadata export --topic edge-topic --mirror edge2 --upstream 192.168.79.252 --file edge2.json
fluvio remote export edge1 --public-endpoint 192.168.79.252 --file edge2.json
```

We'll transfer these files to edge devices in the following sections.
Expand Down Expand Up @@ -203,18 +202,25 @@ On the raspberry pi, run the following command:


```bash
curl -fsS https://hub.infinyon.cloud/install/install.sh | VERSION='0.10.15-dev-2+mirroring-9961bdb' bash
curl -fsS https://hub.infinyon.cloud/install/install.sh | bash
```

Run `fluvio version` to double check.

### Start cluster

We'll use the metadata file to start the edge cluster on the Raspberry Pi:
First we will start the cluster:


```bash
fluvio cluster start
```


Then, we'll use the metadata file on the Raspberry Pi to connect:

```bash
fluvio cluster start --read-only edge2.json
fluvio home connect --file edge2.json
```

Let's check the partitions:
Expand All @@ -227,8 +233,8 @@ fluvio partition list
The edge device should show the following partition::

```bash
TOPIC PARTITION LEADER MIRROR REPLICAS RESOLUTION SIZE HW LEO LRS FOLLOWER OFFSETS
edge-topic 0 5001 Source:upstream:5001 [] Online 0 B 11 11 11 0 []
TOPIC PARTITION LEADER MIRROR REPLICAS RESOLUTION SIZE HW LEO LRS FOLLOWER OFFSETS
edge-topic 0 5001 home_name:0:public_endpoint [] Online 0 B 11 11 11 0 []
```


Expand Down Expand Up @@ -357,7 +363,7 @@ Restart the cluster:


```bash
fluvio cluster upgrade --read-only edge2.json
fluvio cluster upgrade
```

### Consume from edge cluster
Expand Down

0 comments on commit 9d64f51

Please sign in to comment.