Flannel no network/dns connection with vxlan backend

[pscloud-patrick]

I tried to deploy flannel v0.26.2 with the default vxlan backend. Afterwards I noticed that I havent had any chance to get dns resolutions or connection to other pods

Expected Behavior

DNS resolution and pod network working

Current Behavior

no dns resolution, no network connection to other pods

Possible Solution

I dont have one

Steps to Reproduce (for bugs)

k apply -f https://github.com/flannel-io/flannel/releases/download/v0.26.2/kube-flannel.yml
k apply -f https://k8s.io/examples/admin/dns/dnsutils.yaml
k exec -i -t dnsutils -- nslookup kubernetes.default

Context

I first noticed when I deployed a cloud controller who wants to connect to the api of the provider and this failed every time. When I changed the backend to host-gw it immediatly started working

I now tried to deploy an basic echo server and want to access it but its only working from the node itself. In the flannel logs I can see:

│ kube-flannel I0111 17:09:04.933346       1 route_network.go:56] Watching for new subnet leases                                                                                    │
│ kube-flannel I0111 17:09:04.933513       1 subnet.go:152] Batch elem [0] is { lease.Event{Type:0, Lease:lease.Lease{EnableIPv4:true, EnableIPv6:false, Subnet:ip.IP4Net{IP:0xaf40 │
│ 000, PrefixLen:0x18}, IPv6Subnet:ip.IP6Net{IP:(*ip.IP6)(nil), PrefixLen:0x0}, Attrs:lease.LeaseAttrs{PublicIP:0x8ac7969b, PublicIPv6:(*ip.IP6)(nil), BackendType:"host-gw", Backe │
│ ndData:json.RawMessage{0x6e, 0x75, 0x6c, 0x6c}, BackendV6Data:json.RawMessage(nil)}, Expiration:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Asof:0}} }                   │
│ kube-flannel I0111 17:09:04.933781       1 subnet.go:152] Batch elem [0] is { lease.Event{Type:0, Lease:lease.Lease{EnableIPv4:true, EnableIPv6:false, Subnet:ip.IP4Net{IP:0xaf40 │
│ 500, PrefixLen:0x18}, IPv6Subnet:ip.IP6Net{IP:(*ip.IP6)(nil), PrefixLen:0x0}, Attrs:lease.LeaseAttrs{PublicIP:0x8ac7969a, PublicIPv6:(*ip.IP6)(nil), BackendType:"host-gw", Backe │
│ ndData:json.RawMessage{0x6e, 0x75, 0x6c, 0x6c}, BackendV6Data:json.RawMessage(nil)}, Expiration:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Asof:0}} }                   │
│ kube-flannel I0111 17:09:04.933796       1 route_network.go:93] Subnet added: 10.244.0.0/24 via 138.199.150.155                                                                   │
│ kube-flannel E0111 17:09:04.935297       1 route_network.go:168] Error adding route to {Ifindex: 2 Dst: 10.244.0.0/24 Src: <nil> Gw: 138.199.150.155 Flags: [] Table: 0 Realm: 0} │
│ : network is unreachable                                                                                                                                                          │
│ kube-flannel I0111 17:09:04.935491       1 route_network.go:93] Subnet added: 10.244.5.0/24 via 138.199.150.154                                                                   │
│ kube-flannel E0111 17:09:04.935821       1 route_network.go:168] Error adding route to {Ifindex: 2 Dst: 10.244.5.0/24 Src: <nil> Gw: 138.199.150.154 Flags: [] Table: 0 Realm: 0} │
│ : network is unreachable                                                                                                                                                          │
│ kube-flannel I0111 17:09:04.935992       1 subnet.go:152] Batch elem [0] is { lease.Event{Type:0, Lease:lease.Lease{EnableIPv4:true, EnableIPv6:false, Subnet:ip.IP4Net{IP:0xaf40 │
│ 400, PrefixLen:0x18}, IPv6Subnet:ip.IP6Net{IP:(*ip.IP6)(nil), PrefixLen:0x0}, Attrs:lease.LeaseAttrs{PublicIP:0x8ac7969d, PublicIPv6:(*ip.IP6)(nil), BackendType:"host-gw", Backe │
│ ndData:json.RawMessage{0x6e, 0x75, 0x6c, 0x6c}, BackendV6Data:json.RawMessage(nil)}, Expiration:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Asof:0}} }                   │
│ kube-flannel I0111 17:09:04.936150       1 subnet.go:152] Batch elem [0] is { lease.Event{Type:0, Lease:lease.Lease{EnableIPv4:true, EnableIPv6:false, Subnet:ip.IP4Net{IP:0xaf40 │
│ 200, PrefixLen:0x18}, IPv6Subnet:ip.IP6Net{IP:(*ip.IP6)(nil), PrefixLen:0x0}, Attrs:lease.LeaseAttrs{PublicIP:0x251bf2eb, PublicIPv6:(*ip.IP6)(nil), BackendType:"host-gw", Backe │
│ ndData:json.RawMessage{0x6e, 0x75, 0x6c, 0x6c}, BackendV6Data:json.RawMessage(nil)}, Expiration:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Asof:0}} }                   │
│ kube-flannel I0111 17:09:04.936077       1 route_network.go:93] Subnet added: 10.244.4.0/24 via 138.199.150.157                                                                   │
│ kube-flannel E0111 17:09:04.936547       1 route_network.go:168] Error adding route to {Ifindex: 2 Dst: 10.244.4.0/24 Src: <nil> Gw: 138.199.150.157 Flags: [] Table: 0 Realm: 0} │
│ : network is unreachable                                                                                                                                                          │
│ kube-flannel I0111 17:09:04.936663       1 route_network.go:93] Subnet added: 10.244.2.0/24 via 37.27.242.235                                                                     │
│ kube-flannel I0111 17:09:04.936857       1 subnet.go:152] Batch elem [0] is { lease.Event{Type:0, Lease:lease.Lease{EnableIPv4:true, EnableIPv6:false, Subnet:ip.IP4Net{IP:0xaf40 │
│ 100, PrefixLen:0x18}, IPv6Subnet:ip.IP6Net{IP:(*ip.IP6)(nil), PrefixLen:0x0}, Attrs:lease.LeaseAttrs{PublicIP:0x74cb2246, PublicIPv6:(*ip.IP6)(nil), BackendType:"host-gw", Backe │
│ ndData:json.RawMessage{0x6e, 0x75, 0x6c, 0x6c}, BackendV6Data:json.RawMessage(nil)}, Expiration:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Asof:0}} }                   │
│ kube-flannel E0111 17:09:04.937838       1 route_network.go:168] Error adding route to {Ifindex: 2 Dst: 10.244.2.0/24 Src: <nil> Gw: 37.27.242.235 Flags: [] Table: 0 Realm: 0}:  │
│ network is unreachable                                                                                                                                                            │
│ kube-flannel I0111 17:09:04.938020       1 route_network.go:93] Subnet added: 10.244.1.0/24 via 116.203.34.70                                                                     │
│ kube-flannel E0111 17:09:04.939707       1 route_network.go:168] Error adding route to {Ifindex: 2 Dst: 10.244.1.0/24 Src: <nil> Gw: 116.203.34.70 Flags: [] Table: 0 Realm: 0}:  │
│ network is unreachable                                                                                                                                                            │
│ kube-flannel I0111 17:09:04.940267       1 iptables.go:372] bootstrap done                                                                                                        │
│ kube-flannel I0111 17:09:04.947690       1 iptables.go:372] bootstrap done                                                                                                        │
│ kube-flannel I0111 17:09:04.955372       1 main.go:437] Waiting for all goroutines to exit

Your Environment

• Flannel version: 0.26.2
• Backend used (e.g. vxlan or udp): vxlan
• Etcd version: 3.5.15
• Kubernetes version (if used): 1.31
• Operating System and version: Ubuntu 24.04

kube-flannel-cfg

│ Name:         kube-flannel-cfg                                                                                                                                                    │
│ Namespace:    kube-flannel                                                                                                                                                        │
│ Labels:       app=flannel                                                                                                                                                         │
│               k8s-app=flannel                                                                                                                                                     │
│               tier=node                                                                                                                                                           │
│ Annotations:  <none>                                                                                                                                                              │
│                                                                                                                                                                                   │
│ Data                                                                                                                                                                              │
│ ====                                                                                                                                                                              │
│ cni-conf.json:                                                                                                                                                                    │
│ ----                                                                                                                                                                              │
│ {                                                                                                                                                                                 │
│   "name": "cbr0",                                                                                                                                                                 │
│   "cniVersion": "0.3.1",                                                                                                                                                          │
│   "plugins": [                                                                                                                                                                    │
│     {                                                                                                                                                                             │
│       "type": "flannel",                                                                                                                                                          │
│       "delegate": {                                                                                                                                                               │
│         "hairpinMode": true,                                                                                                                                                      │
│         "isDefaultGateway": true                                                                                                                                                  │
│       }                                                                                                                                                                           │
│     },                                                                                                                                                                            │
│     {                                                                                                                                                                             │
│       "type": "portmap",                                                                                                                                                          │
│       "capabilities": {                                                                                                                                                           │
│         "portMappings": true                                                                                                                                                      │
│       }                                                                                                                                                                           │
│     }                                                                                                                                                                             │
│   ]                                                                                                                                                                               │
│ }                                                                                                                                                                                 │
│                                                                                                                                                                                   │
│ net-conf.json:                                                                                                                                                                    │
│ ----                                                                                                                                                                              │
│ {                                                                                                                                                                                 │
│   "Network": "10.244.0.0/16",                                                                                                                                                     │
│   "EnableNFTables": false,                                                                                                                                                        │
│   "Backend": {                                                                                                                                                                    │
│     "Type": "host-gw"                                                                                                                                                             │
│   }                                                                                                                                                                               │
│ }                                                                                                                                                                                 │
│                                                                                                                                                                                   │
│                                                                                                                                                                                   │
│ BinaryData                                                                                                                                                                        │
│ ====                                                                                                                                                                              │
│                                                                                                                                                                                   │
│ Events:  <none>