Service Mesh

A service mesh moves “cross-cutting service-to-service concerns” out of your application code and into a sidecar proxy that runs next to every pod. Mutual TLS, retries, timeouts, circuit breaking, traffic shaping, metrics — all configured at the mesh layer, transparently to the app.

What problem it solves

Without a mesh, every service has to reimplement (or import a library for):

• mTLS between services
• retries with backoff + jitter
• circuit breaking
• per-request timeouts
• canary / weighted traffic splits
• per-call metrics + tracing
• rate limiting

In a polyglot stack, you write that 5 times — Python, Go, Java, Node, Rust. The mesh does it once at the network layer.

The architecture

+--------------+       +------------------+
|   Pod A      |       |   Pod B          |
|  +--------+  |       |  +--------+      |
|  | app    |  |       |  | app    |      |
|  +---+----+  |       |  +---+----+      |
|      | localhost                        |
|  +---v----+  |  mTLS  |  +---v----+     |
|  | proxy  +--+--------+->+ proxy  |     |
|  | (envoy)|  |        |  | (envoy)|     |
|  +--------+  |        |  +--------+     |
+--------------+        +------------------+
       ▲                        ▲
       │                        │
       └──── control plane (Istio, Linkerd) ────┘
            pushes config: mTLS certs, routes,
            traffic policies, telemetry

• Data plane — the sidecar proxies (Envoy, Linkerd-proxy). Intercept all inbound/outbound traffic of the pod.
• Control plane — pushes configuration to all proxies (Istio’s istiod, Linkerd’s controller).

The app code calls http://payments like before. The local proxy intercepts, terminates TLS, picks an instance, retries, records metrics, then forwards.

The big two

Others: Consul Connect, Cilium Service Mesh (eBPF-based, no sidecar), AWS App Mesh (deprecated 2024 — AWS pivoting to VPC Lattice).

What you actually configure

mTLS

Default Istio config: every service talks to every service over mTLS, certificates rotated automatically by istiod. Your app code says http://; the proxy upgrades to https:// and presents the pod’s identity.

apiVersion: security.istio.io/v1beta1
kind: PeerAuthentication
metadata: { name: default, namespace: prod }
spec:
  mtls: { mode: STRICT }   # reject non-mTLS traffic

Retry + timeout

apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata: { name: payments }
spec:
  hosts: [payments]
  http:
  - timeout: 2s
    retries:
      attempts: 3
      perTryTimeout: 500ms
      retryOn: 5xx,connect-failure,reset
    route:
    - destination: { host: payments, subset: v1 }

Same retry rules you’d put in tenacity or httpx-retry, but applied uniformly across every language.

Traffic split (canary)

spec:
  http:
  - route:
    - destination: { host: payments, subset: v1 }
      weight: 90
    - destination: { host: payments, subset: v2 }
      weight: 10

90/10 traffic split between deployments. Combine with metrics-based rollouts (Argo Rollouts, Flagger) for automated canary analysis.

Authorization

apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata: { name: payments-allow-orders }
spec:
  selector: { matchLabels: { app: payments } }
  rules:
  - from:
    - source: { principals: ["cluster.local/ns/prod/sa/orders"] }
    to:
    - operation: { methods: ["POST"], paths: ["/v1/charge"] }

“Only the orders service account in prod namespace may POST to /v1/charge on payments.” Cryptographically enforced by mTLS identities, not just IP/network rules.

What the mesh does NOT solve

• Business logic retries. The mesh retries 5xx and timeouts. It doesn’t know about your payment semantics. App-level idempotency keys are still your problem.
• Async messaging. Meshes intercept HTTP/gRPC, not Kafka/SQS. Kafka clients live inside the app.
• Data consistency. Meshes don’t do sagas, outbox, distributed transactions.
• Application observability. Metrics like “orders placed per second” are domain-specific and must be emitted by the app.

Costs

• Latency — each hop adds a sidecar round-trip (sub-ms in practice; still real).
• Memory — Envoy is ~50-150MB per pod. Multiply by pod count.
• Operational complexity — you now run a control plane. Upgrades are non-trivial.
• Debugging — when “the call failed”, is it the app, the local proxy, the remote proxy, or the control plane?

For < ~10 services, the mesh is usually overkill. For 50+ services and a polyglot stack, it’s the lowest-friction way to enforce mTLS + observability.

Alternatives at smaller scale

• Library-based — httpx + tenacity + manual mTLS via cert-manager. Works in a single-language shop.
• API gateway only — for north-south traffic (client → cluster). East-west (service → service) is on you.
• Cilium without sidecars — eBPF kernel-level networking; mTLS without sidecar memory cost. Newer; rapidly maturing.