Kubernetes Tutorial: Orchestration Basics

When your containerized application crashes at 3 AM because a single node failed, or when you're manually SSH-ing into servers to restart containers, you're experiencing the exact problem that kubernetes container orchestration solves. In 2025, teams running containerized workloads without proper orchestration face cascading failures, unpredictable resource utilization, and deployment processes that can't keep pace with modern CI/CD pipelines. The cost isn't just operational—it's measured in lost revenue during outages, engineering hours spent on manual interventions, and the inability to scale services during traffic spikes.

The stakes have escalated significantly. Modern applications must handle AI inference workloads with variable resource demands, comply with data residency regulations requiring precise pod placement, and maintain sub-100ms response times across globally distributed clusters. A misconfigured orchestration layer doesn't just slow deployments—it creates security vulnerabilities through improper network policies, wastes thousands in cloud costs through inefficient resource allocation, and violates SLAs when health checks fail silently.

Why Manual Container Management Fails at Scale

Running containers with Docker alone worked when teams managed five services across three servers. In 2025, that approach collapses under the weight of modern requirements. Consider a typical e-commerce platform: 40+ microservices, each requiring specific CPU and memory allocations, automatic failover, zero-downtime deployments, and dynamic scaling based on real-time traffic patterns.

Manual orchestration creates several critical failure points. When a container crashes, there's no automatic restart mechanism. When traffic doubles during a product launch, there's no way to automatically provision additional container instances. When you need to update a service, you're forced into maintenance windows and service interruptions. Network discovery between services requires hardcoded IP addresses that break when containers restart on different nodes.

The infrastructure landscape has fundamentally shifted. Cloud providers now charge for sustained CPU usage with per-second billing granularity. AI workloads require GPU scheduling and fractional resource allocation. Privacy regulations mandate that certain data never leaves specific geographic regions. These constraints make manual container management not just inefficient but architecturally impossible.

Understanding Kubernetes Core Architecture

Kubernetes provides a declarative API for managing containerized workloads across a cluster of machines. Instead of telling Kubernetes how to run your application, you describe the desired state, and Kubernetes continuously works to maintain that state.

The control plane manages the cluster through several components. The API server acts as the central communication hub, validating and processing all cluster operations. The scheduler assigns pods to nodes based on resource requirements and constraints. The controller manager runs control loops that watch cluster state and make changes to match desired specifications. The etcd datastore maintains all cluster state with strong consistency guarantees.

Worker nodes run your actual workloads. Each node runs a kubelet agent that communicates with the control plane, a container runtime (typically containerd in 2025), and kube-proxy for network routing. This separation between control plane and data plane enables horizontal scaling and fault tolerance.

Pods: The Fundamental Deployment Unit

Pods represent the smallest deployable unit in Kubernetes—one or more containers that share network and storage resources. Understanding pod design patterns is critical for building resilient applications.

Here's a production-grade pod specification for a web application with a sidecar logging container:

apiVersion: v1
kind: Pod
metadata:
  name: web-app-pod
  labels:
    app: web-app
    tier: frontend
    version: v2.1.0
spec:
  containers:
  - name: web-server
    image: registry.company.com/web-app:2.1.0
    ports:
    - containerPort: 8080
      name: http
      protocol: TCP
    resources:
      requests:
        memory: "256Mi"
        cpu: "250m"
      limits:
        memory: "512Mi"
        cpu: "500m"
    livenessProbe:
      httpGet:
        path: /health/live
        port: 8080
      initialDelaySeconds: 30
      periodSeconds: 10
      timeoutSeconds: 5
      failureThreshold: 3
    readinessProbe:
      httpGet:
        path: /health/ready
        port: 8080
      initialDelaySeconds: 10
      periodSeconds: 5
      timeoutSeconds: 3
      failureThreshold: 2
    env:
    - name: DATABASE_URL
      valueFrom:
        secretKeyRef:
          name: db-credentials
          key: connection-string
    - name: LOG_LEVEL
      value: "info"
  - name: log-forwarder
    image: fluent/fluent-bit:2.2
    volumeMounts:
    - name: shared-logs
      mountPath: /var/log/app
  volumes:
  - name: shared-logs
    emptyDir: {}
  securityContext:
    runAsNonRoot: true
    runAsUser: 1000
    fsGroup: 2000
  nodeSelector:
    workload-type: web
  tolerations:
  - key: "high-priority"
    operator: "Equal"
    value: "true"
    effect: "NoSchedule"

This specification demonstrates several critical production patterns. Resource requests and limits prevent resource starvation and enable efficient bin-packing. Liveness probes detect deadlocked applications and trigger restarts. Readiness probes prevent traffic routing to containers that aren't ready to serve requests. Security contexts enforce least-privilege principles. Node selectors and tolerations control pod placement for compliance and performance requirements.

Deployments: Managing Application Lifecycle

While pods are the fundamental unit, you rarely create them directly. Deployments provide declarative updates, rollback capabilities, and replica management. They're the primary abstraction for running stateless applications.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: web-app-deployment
  namespace: production
  labels:
    app: web-app
spec:
  replicas: 5
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 2
      maxUnavailable: 1
  selector:
    matchLabels:
      app: web-app
      tier: frontend
  template:
    metadata:
      labels:
        app: web-app
        tier: frontend
        version: v2.1.0
      annotations:
        prometheus.io/scrape: "true"
        prometheus.io/port: "8080"
        prometheus.io/path: "/metrics"
    spec:
      containers:
      - name: web-server
        image: registry.company.com/web-app:2.1.0
        ports:
        - containerPort: 8080
        resources:
          requests:
            memory: "256Mi"
            cpu: "250m"
          limits:
            memory: "512Mi"
            cpu: "500m"
        livenessProbe:
          httpGet:
            path: /health/live
            port: 8080
          initialDelaySeconds: 30
          periodSeconds: 10
        readinessProbe:
          httpGet:
            path: /health/ready
            port: 8080
          initialDelaySeconds: 10
          periodSeconds: 5
      affinity:
        podAntiAffinity:
          preferredDuringSchedulingIgnoredDuringExecution:
          - weight: 100
            podAffinityTerm:
              labelSelector:
                matchExpressions:
                - key: app
                  operator: In
                  values:
                  - web-app
              topologyKey: kubernetes.io/hostname

The rolling update strategy ensures zero-downtime deployments. With maxSurge: 2, Kubernetes creates two extra pods before terminating old ones. With maxUnavailable: 1, at most one pod can be unavailable during updates. The pod anti-affinity rule spreads replicas across different nodes, preventing a single node failure from taking down multiple replicas.

Services and Network Discovery

Pods are ephemeral—they get created, destroyed, and rescheduled constantly. Services provide stable network endpoints and load balancing across pod replicas.

apiVersion: v1
kind: Service
metadata:
  name: web-app-service
  namespace: production
  annotations:
    service.beta.kubernetes.io/aws-load-balancer-type: "nlb"
    service.beta.kubernetes.io/aws-load-balancer-cross-zone-load-balancing-enabled: "true"
spec:
  type: LoadBalancer
  selector:
    app: web-app
    tier: frontend
  ports:
  - name: http
    protocol: TCP
    port: 80
    targetPort: 8080
  sessionAffinity: ClientIP
  sessionAffinityConfig:
    clientIP:
      timeoutSeconds: 3600

This LoadBalancer service creates a cloud provider load balancer that distributes traffic across all pods matching the selector. Session affinity ensures requests from the same client IP route to the same pod, critical for applications maintaining in-memory session state.

For internal service-to-service communication, ClusterIP services provide DNS-based discovery:

apiVersion: v1
kind: Service
metadata:
  name: database-service
  namespace: production
spec:
  type: ClusterIP
  selector:
    app: postgres
    tier: database
  ports:
  - name: postgres
    protocol: TCP
    port: 5432
    targetPort: 5432

Applications can now connect to database-service.production.svc.cluster.local:5432, and Kubernetes handles routing to healthy database pods.

Horizontal Pod Autoscaling

Modern applications must scale dynamically based on actual demand. The Horizontal Pod Autoscaler adjusts replica counts based on observed metrics.

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: web-app-hpa
  namespace: production
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: web-app-deployment
  minReplicas: 5
  maxReplicas: 50
  metrics:
  - type: Resource
    resource:
      name: cpu
      target:
        type: Utilization
        averageUtilization: 70
  - type: Resource
    resource:
      name: memory
      target:
        type: Utilization
        averageUtilization: 80
  - type: Pods
    pods:
      metric:
        name: http_requests_per_second
      target:
        type: AverageValue
        averageValue: "1000"
  behavior:
    scaleDown:
      stabilizationWindowSeconds: 300
      policies:
      - type: Percent
        value: 50
        periodSeconds: 60
    scaleUp:
      stabilizationWindowSeconds: 0
      policies:
      - type: Percent
        value: 100
        periodSeconds: 30
      - type: Pods
        value: 5
        periodSeconds: 30
      selectPolicy: Max

This HPA configuration scales based on CPU, memory, and custom application metrics. The behavior section prevents flapping—rapid scaling up and down—by defining stabilization windows and rate limits. Scale-up happens aggressively (doubling capacity or adding 5 pods every 30 seconds), while scale-down is conservative (reducing by 50% every minute after a 5-minute stabilization period).

ConfigMaps and Secrets Management

Separating configuration from application code is fundamental to cloud-native design. ConfigMaps store non-sensitive configuration, while Secrets handle sensitive data.

apiVersion: v1
kind: ConfigMap
metadata:
  name: app-config
  namespace: production
data:
  app.properties: |
    server.port=8080
    logging.level=info
    feature.new-checkout=true
    cache.ttl=3600
  nginx.conf: |
    worker_processes auto;
    events {
      worker_connections 1024;
    }
    http {
      upstream backend {
        server backend-service:8080;
      }
      server {
        listen 80;
        location / {
          proxy_pass http://backend;
        }
      }
    }
---
apiVersion: v1
kind: Secret
metadata:
  name: db-credentials
  namespace: production
type: Opaque
stringData:
  connection-string: "postgresql://user:password@postgres-service:5432/appdb?sslmode=require"
  api-key: "sk-prod-abc123xyz789"

In production environments, never commit secrets to version control. Use external secret management systems like HashiCorp Vault, AWS Secrets Manager, or Azure Key Vault with the External Secrets Operator to sync secrets into Kubernetes.

Persistent Storage with StatefulSets

Stateful applications like databases require stable network identities and persistent storage. StatefulSets provide ordered deployment, stable hostnames, and persistent volume claims.

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: postgres
  namespace: production
spec:
  serviceName: postgres-headless
  replicas: 3
  selector:
    matchLabels:
      app: postgres
  template:
    metadata:
      labels:
        app: postgres
    spec:
      containers:
      - name: postgres
        image: postgres:16.1
        ports:
        - containerPort: 5432
          name: postgres
        env:
        - name: POSTGRES_PASSWORD
          valueFrom:
            secretKeyRef:
              name: postgres-secret
              key: password
        - name: PGDATA
          value: /var/lib/postgresql/data/pgdata
        volumeMounts:
        - name: postgres-storage
          mountPath: /var/lib/postgresql/data
        resources:
          requests:
            memory: "2Gi"
            cpu: "1000m"
          limits:
            memory: "4Gi"
            cpu: "2000m"
  volumeClaimTemplates:
  - metadata:
      name: postgres-storage
    spec:
      accessModes: [ "ReadWriteOnce" ]
      storageClassName: fast-ssd
      resources:
        requests:
          storage: 100Gi

Each pod in a StatefulSet gets a stable hostname (postgres-0, postgres-1, postgres-2) and a dedicated persistent volume. When pods restart, they reattach to the same storage, preserving data across failures.

Common Pitfalls and Failure Modes

Resource limits misconfiguration: Setting limits too low causes OOMKilled errors and CPU throttling. Setting them too high wastes cluster capacity. Monitor actual resource usage over time and set requests at P50 usage and limits at P95.

Missing readiness probes: Without readiness probes, Kubernetes routes traffic to pods before they're ready to serve requests, causing 502 errors during deployments. Always implement both liveness and readiness probes with appropriate thresholds.

Insufficient replica counts: Running a single replica creates a single point of failure. During deployments with rolling updates, you'll have zero available pods if maxUnavailable isn't carefully configured. Run at least three replicas for critical services.

Ignoring pod disruption budgets: Cluster maintenance or node failures can terminate multiple pods simultaneously. Pod Disruption Budgets ensure a minimum number of replicas remain available during voluntary disruptions.

Inadequate monitoring: Kubernetes provides orchestration, not observability. Implement comprehensive monitoring with Prometheus, structured logging with the ELK stack or Loki, and distributed tracing with Jaeger or Tempo.

Security misconfigurations: Running containers as root, using default service accounts with excessive permissions, and storing secrets in ConfigMaps create security vulnerabilities. Implement Pod Security Standards, use dedicated service accounts with minimal RBAC permissions, and encrypt secrets at rest.

Network policy gaps: By default, all pods can communicate with all other pods. Implement network policies to enforce zero-trust networking and limit blast radius during security incidents.

Production Best Practices

Implement health checks correctly: Liveness probes should detect deadlocks and unrecoverable errors. Readiness probes should check dependencies like database connections. Use different endpoints for each probe type.

Use namespaces for isolation: Separate environments (dev, staging, production) and teams into different namespaces. Apply resource quotas and limit ranges to prevent resource exhaustion.

Version everything explicitly: Never use latest tags in production. Use semantic versioning and immutable tags. Implement image scanning in your CI/CD pipeline.

Apply resource quotas: Prevent runaway pods from consuming all cluster resources by setting namespace-level quotas for CPU, memory, and persistent storage.

Implement GitOps workflows: Store all Kubernetes manifests in version control. Use tools like ArgoCD or Flux to automatically sync cluster state with Git repositories, providing audit trails and easy rollbacks.

Plan for disaster recovery: Regularly backup etcd and persistent volumes. Test restore procedures. Document runbooks for common failure scenarios.

Use init containers for dependencies: When pods require setup tasks before the main container starts, use init containers to handle database migrations, configuration validation, or dependency checks.

Implement proper logging: Configure containers to write logs to stdout/stderr. Use a centralized logging solution to aggregate logs across all pods. Include correlation IDs for request tracing.

Frequently Asked Questions

What is the difference between a pod and a container in Kubernetes?

A container is a single running process with its own filesystem and resources. A pod is a Kubernetes abstraction that wraps one or more containers that share network namespace, storage volumes, and lifecycle. Pods represent the smallest deployable unit in Kubernetes, while containers are the actual runtime instances within pods.

How does Kubernetes service discovery work in 2025?

Kubernetes provides DNS-based service discovery through CoreDNS. When you create a service, Kubernetes automatically creates DNS records in the format service-name.namespace.svc.cluster.local. Applications query this DNS name, and Kubernetes returns IP addresses of healthy pods matching the service selector. For external services, LoadBalancer or Ingress resources provide stable external endpoints.

What is the best way to handle secrets in Kubernetes production environments?

Never store secrets directly in manifests or ConfigMaps. Use external secret management systems like HashiCorp Vault, AWS Secrets Manager, or Azure Key Vault integrated through the External Secrets Operator. Enable encryption at rest for etcd. Use dedicated service accounts with minimal RBAC permissions. Rotate secrets regularly and implement secret scanning in CI/CD pipelines.

When should you avoid using Kubernetes for container orchestration?

Avoid Kubernetes for simple applications with minimal scaling requirements, where the operational overhead exceeds the benefits. For single-server deployments, Docker Compose or systemd services are simpler. For serverless workloads with unpredictable traffic patterns, managed services like AWS Lambda or Cloud Run provide better cost efficiency. For edge computing with severe resource constraints, lightweight alternatives like K3s or MicroK8s are more appropriate.

How do you scale Kubernetes clusters for AI and ML workloads in 2025?

AI workloads require GPU scheduling, which Kubernetes supports through device plugins. Use node pools with GPU instances and configure resource requests with nvidia.com/gpu limits. Implement cluster autoscaling to provision GPU nodes on demand. Use gang scheduling with tools like Volcano or Kubeflow to ensure all pods in a distributed training job start simultaneously. Consider fractional GPU sharing with NVIDIA MIG or time-slicing for inference workloads.

What causes pods to remain in Pending state and how do you troubleshoot it?

Pods stay Pending when the scheduler cannot find a suitable node. Common causes include insufficient cluster resources (CPU, memory, or GPU), unsatisfied node selectors or affinity rules, missing persistent volumes, or taints on nodes without matching tolerations. Use kubectl describe pod <pod-name> to view scheduling events and identify the specific constraint preventing scheduling.

How do you implement zero-downtime deployments with Kubernetes?

Use Deployment resources with rolling update strategy. Configure maxSurge and maxUnavailable to control update velocity. Implement readiness probes so Kubernetes only routes traffic to ready pods. Set appropriate terminationGracePeriodSeconds to allow in-flight requests to complete. Use Pod Disruption Budgets to ensure minimum availability during voluntary disruptions. Consider blue-