pfSense

Comments Cookbook
1

This is an addendum to the MetalLB recipe, explaining how to configure MetalLB to perform BGP peering with a pfSense firewall.

This is a companion discussion topic for the original entry at https://geek-cookbook.funkypenguin.co.nz/kubernetes/loadbalancer/metallb/pfsense/
2

Hello, if I am with a digitalocean, I should create a new droplet or where install pfsense?

4

Pfsense would make “sense” (haha) for a self-hosted, bare-metal install. If you’re with a cloud provider, you’ll typically use their tooling / services for firewalling / load-balancing…

5

Thank you for your work on these, it has been very helpful.

I’m attempting to follow this one, but my pfsense does not seem to be receiving the advertisements from metallb.

I can see in the logs that metallb updated the advertisements:
{“caller”:“bgp_controller.go:285”,“event”:“updatedAdvertisements”,“ips”:[“172.22.210.1”],“level”:“info”,“msg”:“making advertisements using BGP”,“numAds”:1,“pool”:“metallb-pool”,“protocol”:“bgp”,“ts”:“2024-05-09T20:56:31Z”}
{“caller”:“main.go:344”,“event”:“serviceAnnounced”,“ips”:[“172.22.210.1”],“level”:“info”,“msg”:“service has IP, announcing”,“pool”:“metallb-pool”,“protocol”:“bgp”,“ts”:“2024-05-09T20:56:31Z”}

and pfsense shows the neighbor as established:
BGP version 4, remote router ID 172.22.200.41, local router ID 172.22.200.1
BGP state = Established, up for 01:48:47

Yet nothing shows up under BGP routes and I get no results when running:
show ip bgp neighbors 172.22.200.41 advertised-routes
show ip bgp neighbors 172.22.200.41 received-routes

I ran a packet trace on pfsense when metallb advertised a route, but I only captured keepalive messages, no route updates.

Any ideas?

Update: it was the tainted master node. Set the speaker tolerance and now it works.

6

Aah, nice find! I was about to go down a rabbit hole of prefix lists, filters, etc, but glad we didn’t need to go there!

7

Running into a weird issue where kubectl is reporting that 3 services have external IPs, but pfSense is only receiving routes to podinfo.

kubectl get service -A -o wide
NAMESPACE        NAME                      TYPE           CLUSTER-IP      EXTERNAL-IP    PORT(S)                               AGE     SELECTOR
podinfo          podinfo                   LoadBalancer   10.43.207.8     172.22.210.1   9898:32685/TCP,9999:30375/TCP         3h40m   app.kubernetes.io/name=podinfo
kasten-io        gateway-ext               LoadBalancer   10.43.221.255   172.22.210.2   80:31390/TCP                          3h4m    service=gateway
default          nginx                     LoadBalancer   10.43.151.173   172.22.210.3   80:32741/TCP                          173m    app=nginx

But on pfSense:

BGP table version is 44, local router ID is 172.22.200.1, vrf id 0
Default local pref 100, local AS 64513
Status codes:  s suppressed, d damped, h history, * valid, > best, = multipath,
               i internal, r RIB-failure, S Stale, R Removed
Nexthop codes: @NNN nexthop's vrf id, < announce-nh-self
Origin codes:  i - IGP, e - EGP, ? - incomplete

   Network          Next Hop            Metric LocPrf Weight Path
*> 172.22.210.1/32  172.22.200.41            0             0 64512 i
*=                  172.22.200.212           0             0 64512 i
*=                  172.22.200.213           0             0 64512 i
*=                  172.22.200.210           0             0 64512 i
*=                  172.22.200.211           0             0 64512 i

Displayed  1 routes and 5 total paths

I’m not really sure where to look to find the issue. Any ideas?