Hello fellow TCP users,
I am moving my homelab from Docker to Kubernetes (because I have nothing to do with my homelab anymore) and I am having an issue with services that need to be accessible both within the cluster and from the outside world on the same hostname.
For an example, supposedly you have two pods: A and B which are accessible via the Gateway with hostname a.example.com and b.example.com respectively. Pod A also need to contact Pod B so there are two ways to do this:
- Via
b.example.com. This works but in this case, the traffic will go from pod A -> the boarder internet -> the loadbalancer -> the gateway -> pod B which is not very optimal. - Via
b.default.svc.cluster.local. This also works but in this case you lose:- TLS: because you bypass the gateway.
- Configuration transparency: now you have to maintain a different internal hostname and another external hostname. For example Keycloak has a specific setting just for this case: https://www.keycloak.org/server/hostname#_utilizing_an_internal_url_for_communication_among_clients.
In Docker case, I can just set the alias of my reverse-proxy container to b.example.com and it is done. I am wondering is there anything I can do to get the traffic goes from pod A -> the gateway -> pod B in Kubernetes. Also is this a common issue or not because I don’t see a lot of articles about this issue on the internet :/
Thank you very much!
You can configure CoreDNS for what you are trying to do, there is usually a ConfigMap containing the CoreDNS config in the kube-system namespace. We did that as a quick hack on one of our clusters at work, but I wouldn’t recommend it tbh. Instead I would do what others already suggested and deploy a Pi-Hole or AdGuard Home, or even a second CoreDNS for this purpose.
Run a local DNS server with local records.
yeah but K8S already has coredns so I am hoping that I can customize it somehow
Local DNS with rewrites is the answer. Don’t mess with CoreDNS unless you know what you’re doing. It has the potential to fubar the cluster DNS.
Running a local DNS server is super easy, especially compared to managing a kubernetes cluster. Install Adguard Home or Pihole, set your devices to use this server, and add some rewrites. That’s it.
sometimes the easiest solution is the best solution.
I just use an internal proxy with the same endpoint and same cert.
Could you tell me more about this :D
Dunno how well it would work with kubernates internal networks, but my DNS is configured with different views for internal and external clients. So, when letsencrypt does a lookup, they get the external IP, but when an internal client looks up the same name, they get the internal IP. TLS is happy, because the certificate matches the name. I’m happy because it works even when the ISP is down.
