Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added Daemonset to istio-cni and fixed the script #2782

Merged
merged 2 commits into from
Jul 4, 2024
Merged

Added Daemonset to istio-cni and fixed the script #2782

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
931336f
Added Daemonset to istio-cni and fixed the script
biswajit-9776 Jul 2, 2024
904b2a0
Removed --cluster-specific from script in hack
biswajit-9776 Jul 2, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
265 changes: 264 additions & 1 deletion common/istio-cni-1-22/istio-install/base/install.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,17 @@
apiVersion: v1
kind: ServiceAccount
metadata:
labels:
app: istio-cni
install.operator.istio.io/owning-resource: unknown
istio.io/rev: default
operator.istio.io/component: Cni
release: istio
name: istio-cni
namespace: kube-system
---
apiVersion: v1
kind: ServiceAccount
metadata:
labels:
app: istio-ingressgateway
Expand Down Expand Up @@ -31,6 +43,55 @@ metadata:
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
labels:
app: istio-cni
install.operator.istio.io/owning-resource: unknown
istio.io/rev: default
operator.istio.io/component: Cni
release: istio
name: istio-cni
rules:
- apiGroups:
- ''
resources:
- pods
- nodes
- namespaces
verbs:
- get
- list
- watch
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
labels:
app: istio-cni
install.operator.istio.io/owning-resource: unknown
istio.io/rev: default
operator.istio.io/component: Cni
release: istio
name: istio-cni-repair-role
rules:
- apiGroups:
- ''
resources:
- events
verbs:
- create
- patch
- apiGroups:
- ''
resources:
- pods
verbs:
- watch
- get
- list
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
labels:
app: istio-reader
Expand Down Expand Up @@ -367,6 +428,43 @@ rules:
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
labels:
app: istio-cni
install.operator.istio.io/owning-resource: unknown
istio.io/rev: default
operator.istio.io/component: Cni
release: istio
name: istio-cni
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: istio-cni
subjects:
- kind: ServiceAccount
name: istio-cni
namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
labels:
install.operator.istio.io/owning-resource: unknown
istio.io/rev: default
k8s-app: istio-cni-repair
operator.istio.io/component: Cni
name: istio-cni-repair-rolebinding
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: istio-cni-repair-role
subjects:
- kind: ServiceAccount
name: istio-cni
namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
labels:
app: istio-reader
Expand Down Expand Up @@ -481,6 +579,25 @@ metadata:
namespace: istio-system
---
apiVersion: v1
data:
cni_network_config: "{\n \"cniVersion\": \"0.3.1\",\n \"name\": \"istio-cni\"\
,\n \"type\": \"istio-cni\",\n \"log_level\": \"debug\",\n \"log_uds_address\"\
: \"__LOG_UDS_ADDRESS__\",\n \n \"cni_event_address\": \"__CNI_EVENT_ADDRESS__\"\
,\n \"kubernetes\": {\n \"kubeconfig\": \"__KUBECONFIG_FILEPATH__\",\n \
\ \"cni_bin_dir\": \"/opt/cni/bin\",\n \"exclude_namespaces\": [ \"kube-system\"\
\ ]\n }\n}"
kind: ConfigMap
metadata:
labels:
app: istio-cni
install.operator.istio.io/owning-resource: unknown
istio.io/rev: default
operator.istio.io/component: Cni
release: istio
name: istio-cni-config
namespace: kube-system
---
apiVersion: v1
data:
config: |-
# defaultTemplates defines the default template to use for pods that do not explicitly specify a template
Expand Down Expand Up @@ -2412,7 +2529,7 @@ data:
},
"istio_cni": {
"chained": true,
"enabled": false,
"enabled": true,
"provider": "default"
},
"pilot": {
Expand Down Expand Up @@ -2600,6 +2717,152 @@ webhooks:
sideEffects: None
---
apiVersion: apps/v1
kind: DaemonSet
metadata:
labels:
install.operator.istio.io/owning-resource: unknown
istio.io/rev: default
k8s-app: istio-cni-node
operator.istio.io/component: Cni
release: istio
name: istio-cni-node
namespace: kube-system
spec:
selector:
matchLabels:
k8s-app: istio-cni-node
template:
metadata:
annotations:
prometheus.io/path: /metrics
prometheus.io/port: '15014'
prometheus.io/scrape: 'true'
sidecar.istio.io/inject: 'false'
labels:
istio.io/dataplane-mode: none
k8s-app: istio-cni-node
sidecar.istio.io/inject: 'false'
spec:
containers:
- args:
- --log_output_level=default:info,cni:info
command:
- install-cni
env:
- name: CNI_NETWORK_CONFIG
valueFrom:
configMapKeyRef:
key: cni_network_config
name: istio-cni-config
- name: CNI_NET_DIR
value: /etc/cni/net.d
- name: CHAINED_CNI_PLUGIN
value: 'true'
- name: REPAIR_ENABLED
value: 'true'
- name: REPAIR_NODE_NAME
valueFrom:
fieldRef:
fieldPath: spec.nodeName
- name: REPAIR_LABEL_PODS
value: 'false'
- name: REPAIR_DELETE_PODS
value: 'false'
- name: REPAIR_REPAIR_PODS
value: 'true'
- name: REPAIR_RUN_AS_DAEMON
value: 'true'
- name: REPAIR_SIDECAR_ANNOTATION
value: sidecar.istio.io/status
- name: REPAIR_INIT_CONTAINER_NAME
value: istio-validation
- name: REPAIR_BROKEN_POD_LABEL_KEY
value: cni.istio.io/uninitialized
- name: REPAIR_BROKEN_POD_LABEL_VALUE
value: 'true'
- name: NODE_NAME
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: spec.nodeName
- name: LOG_LEVEL
value: debug
- name: GOMEMLIMIT
valueFrom:
resourceFieldRef:
resource: limits.memory
- name: GOMAXPROCS
valueFrom:
resourceFieldRef:
resource: limits.cpu
image: docker.io/istio/install-cni:1.22.1
name: install-cni
readinessProbe:
httpGet:
path: /readyz
port: 8000
resources:
requests:
cpu: 100m
memory: 100Mi
securityContext:
capabilities:
add:
- NET_ADMIN
- NET_RAW
- SYS_ADMIN
drop:
- ALL
privileged: true
runAsGroup: 0
runAsNonRoot: false
runAsUser: 0
volumeMounts:
- mountPath: /host/opt/cni/bin
name: cni-bin-dir
- mountPath: /host/proc
name: cni-host-procfs
readOnly: true
- mountPath: /host/etc/cni/net.d
name: cni-net-dir
- mountPath: /var/run/istio-cni
name: cni-socket-dir
nodeSelector:
kubernetes.io/os: linux
priorityClassName: system-node-critical
serviceAccountName: istio-cni
terminationGracePeriodSeconds: 5
tolerations:
- effect: NoSchedule
operator: Exists
- key: CriticalAddonsOnly
operator: Exists
- effect: NoExecute
operator: Exists
volumes:
- hostPath:
path: /opt/cni/bin
name: cni-bin-dir
- hostPath:
path: /proc
type: Directory
name: cni-host-procfs
- hostPath:
path: /etc/cni/net.d
name: cni-net-dir
- hostPath:
path: /var/run/istio-cni
name: cni-socket-dir
- hostPath:
path: /var/run/netns
type: DirectoryOrCreate
name: cni-netns-dir
updateStrategy:
rollingUpdate:
maxUnavailable: 1
type: RollingUpdate
---
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
Expand Down
2 changes: 1 addition & 1 deletion hack/synchronize-istio-cni-manifests.sh
View file
Open in desktop
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you need to drop --cluster-specific such that the hack script works without a running cluster. Why are you using the default profile in line 66? Isn't there a CNI profile?

Copy link
Contributor Author

@biswajit-9776 biswajit-9776 Jul 2, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more. 

[biswa@fedora bin]$ istioctl profile list
Istio configuration profiles:
    ambient
    default
    demo
    empty
    minimal
    openshift
    openshift-ambient
    preview
    remote
    stable

I don't see it explicitly here and it's default in istio-cni-1-17 README.md:
https://github.com/kubeflow/manifests/tree/v1.8-branch/common/istio-cni-1-17#:~:text=%24%20cd%20%24ISTIO_NEW%0A%24%20istioctl%20profile%20dump%20default%20%3E%20profile.yaml

Copy link
Contributor Author

@biswajit-9776 biswajit-9776 Jul 2, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also I found this:
https://istio.io/latest/docs/setup/additional-setup/config-profiles/#:~:text=The%20components%20marked,minimal
Do you think CNI is a component of ambient profile instead of default?

Copy link
Member

@juliusvonkohout juliusvonkohout Jul 2, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, maybe it changed recently. Then let's drop only the --cluster-specific and test on your cluster whether it really works.

Copy link
Contributor Author

@biswajit-9776 biswajit-9776 Jul 2, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have pushed the commit for the updated script and here are the cluster resources:

[biswa@fedora manifests]$ kubectl get daemonsets -n kube-system
NAME             DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR            AGE
istio-cni-node   3         3         3       3            3           kubernetes.io/os=linux   103s
kindnet          3         3         3       3            3           kubernetes.io/os=linux   2m29s
kube-proxy       3         3         3       3            3           kubernetes.io/os=linux   2m30s

I also see 3 healthy istio-cni pods in kube-system namespace in each cluster node:

[biswa@fedora manifests]$ kubectl get pods -n kube-system
NAME                                         READY   STATUS    RESTARTS   AGE
coredns-76f75df574-db5lj                     1/1     Running   0          3m57s
coredns-76f75df574-pdml9                     1/1     Running   0          3m57s
etcd-kind-control-plane                      1/1     Running   0          4m12s
istio-cni-node-28r44                         1/1     Running   0          3m25s
istio-cni-node-rpx22                         1/1     Running   0          3m25s
istio-cni-node-tlg2b                         1/1     Running   0          3m25s
kindnet-bmfbv                                1/1     Running   0          3m57s
kindnet-mk64l                                1/1     Running   0          3m51s
kindnet-pjz6s                                1/1     Running   0          3m52s
kube-apiserver-kind-control-plane            1/1     Running   0          4m12s
kube-controller-manager-kind-control-plane   1/1     Running   0          4m12s
kube-proxy-k5rcg                             1/1     Running   0          3m51s
kube-proxy-qkbk7                             1/1     Running   0          3m57s
kube-proxy-zjvpn                             1/1     Running   0          3m52s
kube-scheduler-kind-control-plane            1/1     Running   0          4m12s

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you think we should maybe check if the ip tables are modified?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, please check a few initcontainers

Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,7 @@ $ISTIOCTL profile dump default > profile.yaml

# cd $ISTIO_NEW
# export PATH="$MANIFESTS_DIR/scripts:$PATH"
$ISTIOCTL manifest generate -f profile.yaml -f profile-overlay.yaml > dump.yaml
$ISTIOCTL manifest generate -f profile.yaml -f profile-overlay.yaml --set components.cni.enabled=true --set components.cni.namespace=kube-system > dump.yaml
./split-istio-packages -f dump.yaml
mv $ISTIO_NEW/crd.yaml $ISTIO_NEW/istio-crds/base
mv $ISTIO_NEW/install.yaml $ISTIO_NEW/istio-install/base
Expand Down
Loading