Skip to content

Commit

Permalink
Merge pull request #4 from padok-team/fix/spring-update
Browse files Browse the repository at this point in the history
Fix/spring update
  • Loading branch information
OscarPoels authored May 2, 2023
2 parents 9ec025e + 6157916 commit dccba46
Show file tree
Hide file tree
Showing 27 changed files with 101 additions and 42 deletions.
Binary file added .assets/gitlab-connect.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added .assets/gitlab-create-project.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
1 change: 1 addition & 0 deletions .gitlab-ci.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
include: /04-Application/.gitlab-ci.yml
32 changes: 18 additions & 14 deletions 01-AmazonWebService/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,22 +15,26 @@ Don't worry if you don't know AWS or the service that we will use. Everything is
- [ ] Once connected, click on your name on the top right menu and select **[Security Credentials](https://us-east-1.console.aws.amazon.com/iam/home?region=eu-west-3#/security_credentials)**
- [ ] Next, in the section "Access keys for CLI, SDK, & API access" click on "Create access Key"
- [ ] Your Access key is created, now you need to record it on your machine
- [ ] Open your shell and create a directory `.aws` at the root of your home directory
- [ ] Create a file `credentials` inside this directory and copy the following content with the values from the AWS console
- There are two methods
- [ ] **Method 1:** Let yourself be guided by `aws configure` command
- [ ] **Method 2:** Copy the values and create the file manually

```
[default]
aws_access_key_id=AKIAxxxxxx
aws_secret_access_key=xxxxxx
```
- [ ] Open your shell and create a directory `.aws` at the root of your home directory
- [ ] Create a file `credentials` inside this directory and copy the following content with the values from the AWS console

- [ ] Create a file `config` inside the same directory and copy the following content
```
[default]
aws_access_key_id=AKIAxxxxxx
aws_secret_access_key=xxxxxx
```
```
[default]
region=eu-west-3
output=json
```
- [ ] Create a file `config` inside the same directory and copy the following content
```
[default]
region=eu-west-3
output=json
```
- [ ] Now you can test your access to AWS
Expand All @@ -48,4 +52,4 @@ The command will output something like this:
}
```
If it's ok, go to the next part, otherwize, ask for help 🙌
If it's ok, go to the next part, otherwize, ask for help 🙌
4 changes: 2 additions & 2 deletions 02-InfrastructureAsCode/01-locals.tf
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
locals {

owner = "FIX_ME" # lower case only
my_cidr = "FIX_ME" # (x.x.x.x/xx)
my_cidr = "10.0.0.0/20" # (x.x.x.x/xx)
my_ip = "${jsondecode(data.curl.public_ip.response)["ip"]}/32"
context = {

Expand All @@ -23,7 +23,7 @@ locals {
#eks = {
# cluster = {
# name = local.owner
# version = "1.21"
# version = "1.25"
# }
# node = {
# name = "eks_node"
Expand Down
19 changes: 19 additions & 0 deletions 02-InfrastructureAsCode/05-outputs.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
output cluster_name {
value = aws_eks_cluster.this.name
description = "EKS Cluster name"
}

output cluster_autoscaler_role_arn {
value = aws_iam_role.cluster_autoscaler.arn
description = "EKS Cluster Autoscaler IAM Role ARN"
}

output gitlab_runner_role_arn {
value = aws_iam_role.gitlab_runner.arn
description = "Gitlab Runner IAM Role ARN"
}

output ecr_repository {
value = aws_ecr_repository.this.name
description = "ECR Repository for docker images"
}
10 changes: 5 additions & 5 deletions 03-Kubernetes/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,13 +18,13 @@ From [Wikipedia](https://en.wikipedia.org/wiki/Kubernetes)
- We are doing to deploy our first component on Kubernetes : `cluster-autoscaler`. It will be used to scale the number of nodes regarding the needs of pods.
- We need to configure a little bit cluster-autoscaler beforing deploying it

- [ ] Edit the file `cluster-autoscaler/values.yaml` and set your cluster name for `clusterName` parameter (should be the same value that you put in `01-locals.tf` in `owner` parameter)
- [ ] In the same file, set the value for `eks.amazonaws.com/role-arn`: use the value of `owner` that your put in `01-locals.tf` in `owner` parameter)
- [ ] Edit the file `cluster-autoscaler/values.yaml` and set your cluster name for `clusterName` parameter (you can get it with `terraform output`).
- [ ] In the same file, set the value for `eks.amazonaws.com/role-arn` parameter. You can get the value with `terraform output`
- [ ] Then, deploy cluster-autoscaler on your cluster : `helm upgrade --install cluster-autoscaler cluster-autoscaler -n cluster-autoscaler --create-namespace=true`
- [ ] Once finished, check your deployment status with `kubectl`. Don't forget that resources are deployed in namespaces !
- [ ] For the Gitlab runner, edit the file `gitlab-runner/values.yaml` and replace the value for `runnerRegistrationToken`. To get a token, go to your `Gitlab Repository > Settings > CI/CD > Runner > Specific runners` and copy the token.
- [ ] Also, change the value for `eks.amazonaws.com/role-arn` like for `cluster-autoscaler`
- [ ] Then, install `gitlab-runner`
- [ ] Then, install `gitlab-runner`
- [ ] Process in the same way for `ingress-nginx` (no configuration is needed)
- [ ] Check the status of your new deployments
- [ ] Last, deploy a component to get metrics (monitoring) for our pods:
Expand Down Expand Up @@ -78,6 +78,6 @@ To deploy Gitlab runner and Nginx Ingress

```
helm upgrade --install gitlab-runner gitlab-runner -n gitlab --create-namespace=true
helm upgrade --install ingress-nginx ingress-nginx -n nginx --create-namespace=true
helm upgrade --install ingress-nginx ingress-nginx -n ingress-nginx --create-namespace=true
```
</details>
</details>
2 changes: 1 addition & 1 deletion 03-Kubernetes/cluster-autoscaler/Chart.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -6,4 +6,4 @@ version: 0.1.0
dependencies:
- name: cluster-autoscaler
repository: https://kubernetes.github.io/autoscaler
version: 9.19.0
version: 9.28.0
Binary file not shown.
Binary file not shown.
4 changes: 2 additions & 2 deletions 03-Kubernetes/cluster-autoscaler/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -29,12 +29,12 @@ cluster-autoscaler:
automountServiceAccountToken: true

annotations:
eks.amazonaws.com/role-arn: arn:aws:iam::334033969502:role/cluster_autoscaler_<FIX_ME> # FIX_ME with the value of owner
eks.amazonaws.com/role-arn: FIX_ME # Set the ARN of the role you created for the cluster-autoscaler, see terraform outputs

image:
repository: k8s.gcr.io/autoscaling/cluster-autoscaler
pullPolicy: IfNotPresent
tag: v1.21.1 # Follows cluster versions schemes: controle plane 1.18 -> tag v1.18.x
tag: v1.25.1 # Follows cluster versions schemes: controle plane 1.18 -> tag v1.18.x

# Defaults to "ClusterFirst". Valid values are
# 'ClusterFirstWithHostNet', 'ClusterFirst', 'Default' or 'None'
Expand Down
2 changes: 1 addition & 1 deletion 03-Kubernetes/gitlab-runner/Chart.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -6,4 +6,4 @@ version: 0.1.0
dependencies:
- name: gitlab-runner
repository: https://charts.gitlab.io
version: 0.41.0
version: 0.52.0
Binary file not shown.
Binary file not shown.
2 changes: 1 addition & 1 deletion 03-Kubernetes/gitlab-runner/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ gitlab-runner:
verbs: ["*"]
clusterWideAccess: true
serviceAccountAnnotations:
eks.amazonaws.com/role-arn: arn:aws:iam::334033969502:role/gitlab_runner_<FIX_ME>
eks.amazonaws.com/role-arn: FIX_ME # Set the ARN of the role you created for the gitlab-runner, see terraform outputs

runners:
config: |
Expand Down
2 changes: 1 addition & 1 deletion 03-Kubernetes/ingress-nginx/Chart.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -6,4 +6,4 @@ version: 0.1.0
dependencies:
- name: ingress-nginx
repository: https://kubernetes.github.io/ingress-nginx
version: 4.1.2
version: 4.6.0
Binary file not shown.
Binary file not shown.
2 changes: 2 additions & 0 deletions 04-Application/.dockerignore
Original file line number Diff line number Diff line change
@@ -1,2 +1,4 @@
*
!*.go
!go.mod
!go.sum
7 changes: 5 additions & 2 deletions 04-Application/.gitlab-ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,9 @@ stages:
- kubernetes
- padok

before_script:
- cd 04-Application # Change directory to the application folder if needed

# This job build our docker image with our App
# Once the image built, it push the image to our private ECR registry
# Permissions are already set to allow Gitlab to access the registry
Expand All @@ -18,7 +21,7 @@ build:
variables:
AWS_DEFAULT_REGION: eu-west-3
ECR: dkr.ecr.eu-west-3.amazonaws.com
REPO: <FIX_ME>/demo-k8s # replace FIX_ME with your name
REPO: FIX_ME/demo-k8s # replace FIX_ME with your name
TAG: latest
script:
- echo "Login to ECR ..."
Expand All @@ -37,4 +40,4 @@ release:
extends:
- .tags
script:
- helm upgrade --install -n demo --create-namespace=true demo-k8s chart
- helm upgrade --install -n demo --create-namespace=true demo-k8s chart
8 changes: 6 additions & 2 deletions 04-Application/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,8 +1,12 @@
FROM public.ecr.aws/docker/library/golang:alpine AS builder
FROM public.ecr.aws/docker/library/golang:1.20 AS builder
WORKDIR /app

COPY go.* .
RUN go mod download

COPY main.go .
ARG VERSION
RUN go env -w GO111MODULE=off && CGO_ENABLED=0 go build -ldflags="-s -w -X main.version=${VERSION}" -o=k8s-hello
RUN CGO_ENABLED=0 go build -ldflags="-s -w -X main.version=${VERSION}" -o=k8s-hello

FROM scratch AS runner
COPY --from=builder /app/k8s-hello /k8s-hello
Expand Down
4 changes: 2 additions & 2 deletions 04-Application/chart/templates/hpa.yaml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
apiVersion: autoscaling/v2beta2
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
name: {{ include "chart.fullname" . }}
Expand All @@ -17,4 +17,4 @@ spec:
name: cpu
target:
type: Utilization
averageUtilization: {{ .Values.hpa.averageCpuUtilization }}
averageUtilization: {{ .Values.hpa.averageCpuUtilization }}
4 changes: 2 additions & 2 deletions 04-Application/chart/values.yaml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@

container:
image:
repository: 334033969502.dkr.ecr.eu-west-3.amazonaws.com/FIX_ME/demo-k8s
repository: FIX_ME # Use terraform output or console
tag: latest
pullPolicy: Always
port: 8080
Expand Down Expand Up @@ -29,4 +29,4 @@ hpa:
averageCpuUtilization: 70
service:
type: ClusterIP
port: 80
port: 80
3 changes: 3 additions & 0 deletions 04-Application/go.mod
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
module main

go 1.20
6 changes: 3 additions & 3 deletions 05-PlayWithKube/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,18 +49,18 @@ The system responsible of scaling the pod (horizontally) is named [HPA](https://

Since the default route of our application doesn't consome too much CPU even under high load, there is another root that use more CPU for each request avaiable: `/load`. Calling this endpoint at high frequency will consomme a lot of CPU that will help to trigger the HPA

- [ ] We will use `vegata` to overload our API. Create a file `targets.txt` like the example below:
- [ ] We will use `vegeta` to overload our API. Create a file `targets.txt` like the example below:

```
GET http://<YOUR_PUBLIC_ENDPOINT>/panic
GET http://<YOUR_PUBLIC_ENDPOINT>/load
```

- `vegeta` is a load testing tool useful to stress application.

- [ ] Use `vegeta` like in the example below to overload your application:

```
vegeta attack -rate=15/s -targets target > /dev/null
vegeta attack -rate=15/s -targets targets.txt > /dev/null
```

- [ ] Watch the CPU use of your current pod. After a few seconds, you will see that the number increase (almost `1000Mi`)
Expand Down
25 changes: 21 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,14 +23,31 @@ If you have any question, any issue or you want to know more about something, do

For each exercise, there is a hint or the solution provided. First, try to do the exercise and find the solution by yourself. You can also ask to a member of Padok (and of course you can ask Google too). Don't get stuck on something and use the hint wisely.

# Part 0 - Initialization (~5 mins)
# Part 0 - Initialization (~5-10 mins)

While this repo is hosted on Github, we will use Gitlab.com for the CI/CD part. So, we will push our code on Gitlab.com.

- [ ] Clone this repo

```bash
git clone https://github.com/padok-team/dojo-devops.git
```

- [ ] Connect to Gitlab.com with your account

![Gitlab connect](./.assets/gitlab-connect.png)

- [ ] Create a new project Blank Project (don't check `Initialize repository with a README`)
- [ ] And Clone this repo on your desktop

```
git clone [email protected]:padok-team/dojo-devops.git
![Gitlab create project](./.assets/gitlab-create-project.png)

- [ ] Push this repo into Gitlab. You'll need to add an SSH key or get an access token.

```bash
git remote rename origin github
git remote add origin https://gitlab.com/<your_repo>.git
git push -u origin --all
git push -u origin --tags
```

That's all for now, we will back later to Gitlab. You can move to the next part.
Expand Down
6 changes: 6 additions & 0 deletions renovate.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
{
"$schema": "https://docs.renovatebot.com/renovate-schema.json",
"extends": [
"local>padok-team/renovate-config"
]
}

0 comments on commit dccba46

Please sign in to comment.