Use case: certain services available only from home network, but still with subdomain and certificate, that all browsers will accept. For example UI for traefik or longorn – I would not necessarily share them outside. Of course you can use self-signed, but there is little inconvenience here and there. The idea is to get the wildcard certificate, and configure local DNS server with IP from private network range.

To quickly recap, there are three classes of private IP ranges:

• class A: 10.0.0.0 to 10.255.255.255, 16mln addresses
• class B: 172.16.0.0 to 172.31.255.255, 1mln addresses
• class C: 192.168.0.0 to 192.168.255.255, 65k addressees

Those addresses are non routable, and require NAT.

I started with creating another cluster issuer. Not sure why, but I had a problem with traefik solver. Maybe I am not that patient. But this time I created cloudflare solver. Using cloudflare API tends to be more reliable. Cert manager will use provided token for cloudflare API, and create TXT record.

apiVersion: v1
kind: Secret
metadata:
  name: cloudflare-api-token-secret
  namespace: cert-manager
type: Opaque
stringData:
  api-token: "take it from your cloudflare account"

apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt-prod-api
spec:
  acme:
    email: mail@mail.pl
    server: https://acme-v02.api.letsencrypt.org/directory
    privateKeySecretRef:
      # Secret resource that will be used to store the account's private key.
      name: issuer-acct-key
    solvers:
    - dns01:
        cloudflare:
          email: mail@mail.pl
          apiTokenSecretRef:
            name: cloudflare-api-token-secret
            key: api-token
      selector:
        dnsZones:
        - 'example.com'
        - '*.example.com'

Now, request wildcard certificate from LetsEncrypt.

apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: wildcard-example-com
  namespace: kube-system
spec:
  secretName: wildcard-example-com-tls
  issuerRef:
    name: letsencrypt-prod-api
    kind: ClusterIssuer
  dnsNames:
    - "*.example.com"

Create those resources, and check for a minute or so. After that, your certificate should be ready. You can check the status with kubectl get cert -A

Next step would be configuring traefik to use newly obtained certificate by default. In k3s it can be done by editing the helm chart values. The traefik will be redeployed automatically, after any changes in this file: /var/lib/rancher/k3s/server/manifests/traefik.yaml. But there is a better way. Traefik has CRD called TLSStore. Instead of redeploying the app, you can create such TLSStore and point to a secret with the certificate. It will be used as default.

apiVersion: traefik.containo.us/v1alpha1
kind: TLSStore
metadata:
  name: default
  namespace: kube-system
spec:
  defaultCertificate:
    secretName: my-default-cert-secret

Now, add a local record to pihole. Unfortunately pihole UI do not accept multiple entries for a single domain. So first dirty hack would be creating similar entries, like service.domain and service2.domain and double the ingresses. There is another way, by editing /etc/dnsmasq.d/02-custom.conf. If you inspect pihole deployment, you will see:

        volumeMounts:
        - mountPath: /etc/pihole
          name: config
        - mountPath: /etc/dnsmasq.d/02-custom.conf
(...)
      - configMap:
          defaultMode: 420
          name: pihole-custom-dnsmasq
        name: custom-dnsmasq

It means, that we can modify this file, by editing custom-dnsmasq configmap.

Add something like this, and delete the pod

data:
  02-custom.conf: |
    addn-hosts=/etc/addn-hosts
    host-record=pihole.piasecki.it,192.168.1.23
    host-record=pihole.piasecki.it,192.168.1.43

after that we can check if it is working:

dig @192.168.1.23 pihole.piasecki.it                                                                                                             

;; ANSWER SECTION:
pihole.piasecki.it.     0       IN      A       192.168.1.43
pihole.piasecki.it.     0       IN      A       192.168.1.23

Lastly, create ingress

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    traefik.ingress.kubernetes.io/router.entrypoints: web,websecure
    traefik.ingress.kubernetes.io/router.tls: "true"
  name: pihole-web
  namespace: pihole
spec:
  ingressClassName: traefik
  rules:
  - host: pihole.example.com
    http:
      paths:
      - backend:
          service:
            name: pihole-web
            port:
              number: 80
        path: /
        pathType: Prefix
  tls:
  - hosts:
    - pihole.example.com

finally, we can check if certificate is set up correctly

openssl s_client -connect pihole.piasecki.it:443 -showcerts                                                                                
CONNECTED(00000003)
depth=2 C = US, O = Internet Security Research Group, CN = ISRG Root X1
verify return:1
depth=1 C = US, O = Let's Encrypt, CN = R11
verify return:1
depth=0 CN = *.piasecki.it
verify return:1
---
Certificate chain
 0 s:CN = *.piasecki.it
   i:C = US, O = Let's Encrypt, CN = R11
   a:PKEY: rsaEncryption, 2048 (bit); sigalg: RSA-SHA256
   v:NotBefore: Nov  5 20:38:06 2024 GMT; NotAfter: Feb  3 20:38:05 2025 GMT

And it’s done. Frankly, it was much more problematic than I expected. Traefik logs are garbage. Debugging those issues would be much faster, If traefik would log something, when it encounter an issue with configuration. Anyway, I’m glad I documented it (to some degree). Hopefully, it will be useful to someone.