Ingress and Networking

How external traffic reaches your pods, and how pods reach each other or are restricted.

Service types recap

ClusterIP (default, in-cluster), NodePort (rarely used directly), LoadBalancer (provisions a cloud LB), ExternalName (DNS alias). For internet traffic, you typically pair a Service: LoadBalancer with an Ingress for L7 features.

Ingress

The HTTP-aware front door. One LB → many services routed by host + path.

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: public
  namespace: prod
  annotations:
    # or alb, traefik, ...
    kubernetes.io/ingress.class: nginx
    cert-manager.io/cluster-issuer: letsencrypt-prod
spec:
  tls:
  - hosts: [api.example.com]
    secretName: api-tls
  rules:
  - host: api.example.com
    http:
      paths:
      - { path: /v1/orders,   pathType: Prefix, backend: { service: { name: orders,   port: { number: 80 } } } }
      - { path: /v1/users,    pathType: Prefix, backend: { service: { name: users,    port: { number: 80 } } } }
      - { path: /v1/payments, pathType: Prefix, backend: { service: { name: payments, port: { number: 80 } } } }

The Ingress object is just a spec; it does nothing without an Ingress Controller to implement it (nginx-ingress, AWS Load Balancer Controller, Traefik, Contour).

Ingress controllers

Modern direction: Gateway API (the successor to Ingress, k8s 1.29+ GA), with proper resource types Gateway, HTTPRoute, GRPCRoute, etc. Cleaner separation of “I run an LB” from “I route paths”.

TLS

cert-manager is the de-facto: it watches Ingress resources, requests certs from Let’s Encrypt / ACME / private CA, stores them as Secrets, and renews.

apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata: { name: letsencrypt-prod }
spec:
  acme:
    email: ops@example.com
    server: https://acme-v02.api.letsencrypt.org/directory
    privateKeySecretRef: { name: letsencrypt-prod-key }
    solvers:
    - http01:
        ingress: { class: nginx }

The cert-manager.io/cluster-issuer annotation on the Ingress triggers cert issuance/renewal automatically.

Network Policies

Default k8s networking: flat — every pod can talk to every other pod. Network Policies restrict this.

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata: { name: orders-allow, namespace: prod }
spec:
  podSelector: { matchLabels: { app: orders } }
  policyTypes: [Ingress, Egress]
  ingress:
  - from:
    - podSelector: { matchLabels: { app: api-gateway } }
    ports: [{ port: 8000 }]
  egress:
  - to:
    - podSelector: { matchLabels: { app: postgres } }
    ports: [{ port: 5432 }]
  - to:
    - namespaceSelector: { matchLabels: { name: kube-system } }
    ports: [{ port: 53, protocol: UDP }]   # DNS

“orders pods accept ingress only from api-gateway, egress only to postgres and DNS.” Default-deny once you create any policy for a pod.

Network policies need a CNI that enforces them: Calico, Cilium, Weave. Flannel doesn’t enforce; AWS VPC CNI requires Calico add-on or recent versions.

DNS inside the cluster

CoreDNS resolves:

orders                    → orders.<current-ns>.svc.cluster.local  (in-namespace short name)
orders.prod               → orders.prod.svc.cluster.local
orders.prod.svc.cluster.local → ClusterIP
<pod-ip-dotted>.<ns>.pod.cluster.local → pod IP (rarely useful)

DNS gotchas:

• ndots:5 in /etc/resolv.conf makes external.com first try external.com.prod.svc.cluster.local, etc. — 5 failed lookups before falling back. For chatty external calls, set dnsPolicy: ClusterFirstWithHostNet carefully or override ndots.
• DNS caching in JVMs / language runtimes — Python’s socket.getaddrinfo doesn’t cache by default; requests/httpx don’t cache (relies on the OS / glibc). Usually fine; jvm/glibc issues are language-specific.
• A pod restart changes the pod IP. Service ClusterIPs stay stable until the Service is deleted.

North-south vs east-west

The mesh is the layer that adds mTLS + retries + tracing for east-west traffic without touching app code.

AWS specifics: ALB Ingress Controller

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: public
  annotations:
    alb.ingress.kubernetes.io/scheme: internet-facing
    # bypass NodePort
    alb.ingress.kubernetes.io/target-type: ip
    alb.ingress.kubernetes.io/listen-ports: '[{"HTTPS":443}]'
    alb.ingress.kubernetes.io/certificate-arn: arn:aws:acm:...
spec:
  ...

The AWS Load Balancer Controller watches Ingress objects, provisions an ALB, creates Target Groups, registers pod IPs (with target-type: ip for direct, no NodePort hop). Most efficient pattern on EKS.

Egress: how pods reach the internet

By default, pods egress out via the node’s IP (NAT). On EKS:

• Public subnets → node IP is public; pods egress directly.
• Private subnets + NAT Gateway → pods egress via NAT GW. Costs $$ at high throughput.
• VPC Endpoints (Interface / Gateway) → in-cluster traffic to S3, DynamoDB, etc. without NAT.

For cost-sensitive workloads, VPC Endpoints for S3 + ECR save a lot vs NAT egress.

Common bugs

• “Service exists, no endpoints.” Selector doesn’t match pod labels, or pods are not ready. kubectl describe svc + kubectl get endpoints.
• “Ingress works internally, 502 externally.” ALB security group doesn’t allow 443 from internet; or target group health checks fail (wrong path or no readiness probe at that path).
• “DNS lookup is slow.” ndots:5 thrashing on external hostnames. Use FQDNs (s3.amazonaws.com.) with trailing dot, or override.
• “Network policy applied, nothing works.” Default-deny took effect; you forgot to allow egress to DNS (53/UDP to kube-system).
• “TLS cert renewal failing.” ACME challenge fails because Ingress wasn’t routing /.well-known/acme-challenge/... correctly. cert-manager logs show this.