DevOps and Operations
Azure Kubernetes Service (AKS)
Azure Kubernetes Service (AKS) is a managed container orchestration platform that simplifies deploying, scaling, and managing containerized applications in the cloud. It matters because it abstracts the underlying infrastructure complexity, allowing teams to focus on application logic rather than cluster maintenance. You reach for it when your application architecture requires high availability, granular scaling, and automated container lifecycle management at scale.
Core Architecture and Control Plane
To understand AKS, you must first recognize the distinction between the control plane and the node pools. Microsoft manages the control plane, which acts as the 'brain' of the cluster, handling API requests, scheduling, and state storage. When you deploy a cluster, you are effectively offloading the burden of maintaining the etcd database and API server to a fully managed environment. This is crucial because it ensures that your cluster's management layer remains highly available and patched without manual intervention. You interact with this control plane using standard tools, treating it as the authoritative source of truth for your application's desired state. The system works by constantly reconciling the actual state of your running containers against the desired configuration defined in your manifests. This reconciliation loop is the fundamental mechanism that enables self-healing; if a container dies, the scheduler immediately identifies the discrepancy and initiates a new instance to restore balance.
# Deploying a basic AKS cluster using Azure CLI
az aks create --resource-group MyResourceGroup --name MyAKSCluster --node-count 3 --enable-addons monitoring --generate-ssh-keysNodes and Pool Configuration
Nodes represent the actual compute resources where your containerized workloads execute. In AKS, these nodes are organized into node pools, which are groups of virtual machines that share the same configuration. Understanding this is vital for cost optimization and performance tuning. You can create system node pools for core infrastructure services and separate user node pools for your specific applications, allowing you to isolate workloads. This segregation is beneficial when you need to apply different hardware specifications—such as memory-optimized or compute-optimized machines—to specific parts of your service architecture. Furthermore, by using node pools, you can scale your compute capacity independently of the control plane. When demand surges, the cluster autoscaler detects pending pods and automatically provisions new virtual machines in the node pool to handle the load, ensuring that your application maintains performance levels without requiring constant human oversight or pre-provisioned over-capacity.
# Adding a new user node pool with a specific VM size
az aks nodepool add --resource-group MyResourceGroup --cluster-name MyAKSCluster --name userpool1 --node-count 3 --node-vm-size Standard_DS2_v2Networking and Load Balancing
Networking in AKS is typically handled through the Azure CNI (Container Networking Interface), which provides every pod with an IP address directly from the underlying virtual network. This direct integration is superior to overlay networking because it simplifies troubleshooting and security policy application by allowing standard network security groups to recognize pod traffic. When exposing services, the Azure Load Balancer acts as the primary entry point for external traffic. It works by routing incoming requests to the specific nodes running your application pods, leveraging health probes to ensure traffic only hits healthy instances. By integrating natively with Azure networking primitives, AKS allows you to apply enterprise-grade security, such as restricting traffic flow with network policies or using private links to secure your internal traffic paths. This architecture is designed to scale horizontally; as you add more replicas, the load balancer automatically distributes the request load across all available endpoints to maintain responsiveness.
# Creating a LoadBalancer service manifest for a web app
apiVersion: v1
kind: Service
metadata:
name: web-service
spec:
selector:
app: web
ports:
- port: 80
targetPort: 8080
type: LoadBalancer # Provisions an Azure Load BalancerPersistent Storage and Data Persistence
Containers are inherently ephemeral; when they stop, their local storage is wiped. To persist data, AKS leverages Azure Managed Disks or Azure Files. Managed Disks provide high-performance block storage suitable for databases, while Azure Files offers file shares that can be accessed by multiple pods simultaneously. The key to using these effectively is the StorageClass and PersistentVolumeClaim mechanism. When a developer defines a persistent volume claim, the AKS scheduler automatically provisions the requested Azure storage resource and attaches it to the node. This decoupling of storage lifecycle from pod lifecycle is what allows your applications to be stateful even within a highly dynamic environment. By using these abstractions, you ensure that your data persists across pod restarts or rescheduling events, providing a reliable foundation for stateful services that require high availability and data integrity in the event of hardware or software failure.
# Defining a PersistentVolumeClaim using Azure Managed Disk
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: data-pvc
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 10Gi
storageClassName: managed-premiumMonitoring and Operational Visibility
Operational visibility in AKS is primarily achieved through Container Insights, which integrates directly with Azure Monitor and Log Analytics. It works by deploying a containerized agent to every node, which streams telemetry, performance metrics, and logs into a centralized repository. This allows you to visualize cluster health, monitor resource consumption, and set up alerts for anomalies like high CPU usage or frequent pod restarts. Understanding this flow is critical for maintaining long-term reliability; you cannot manage what you cannot measure. By aggregating data across nodes and pods, you gain a holistic view of the entire ecosystem, enabling proactive capacity planning and rapid incident response. Furthermore, these logs are essential for auditing and compliance, as they provide an immutable record of cluster activity. When issues occur, you can run complex queries against this data to pinpoint the exact sequence of events that led to a service degradation or failure.
# Querying pod restart logs in Log Analytics
KubePodInventory
| where TimeGenerated > ago(1h)
| project Name, RestartCount, PodStatus
| order by RestartCount descKey points
- AKS manages the complex control plane, freeing operators from manual orchestration tasks.
- Node pools allow for granular resource allocation by grouping similar virtual machines together.
- The Kubernetes scheduler maintains the desired state by constantly reconciling current and target configurations.
- Azure CNI networking enables pods to receive unique IP addresses directly from the virtual network.
- Horizontal pod autoscaling adjusts replica counts based on observed CPU or memory metrics.
- Persistent storage is handled through volume claims that decouple data from ephemeral container lifecycles.
- Container Insights provides necessary telemetry for proactive monitoring and automated alerting.
- Load balancers facilitate external access by distributing incoming traffic across healthy container instances.
Common mistakes
- Mistake: Manually modifying AKS node VMs. Why it's wrong: AKS manages these nodes automatically, and manual changes will be overwritten during upgrades or scaling operations. Fix: Use Kubernetes manifests, Helm charts, or Terraform to define desired state configurations.
- Mistake: Storing sensitive data in plain text Kubernetes Secrets. Why it's wrong: By default, these are only base64 encoded and are not secure for production credentials. Fix: Integrate Azure Key Vault with the Secrets Store CSI driver to inject secrets securely into pods.
- Mistake: Neglecting to set resource requests and limits on pods. Why it's wrong: Without these, a single pod can consume all resources on a node, leading to 'noisy neighbor' issues and cluster instability. Fix: Define specific CPU and memory requests and limits in your deployment YAML.
- Mistake: Using a single namespace for all applications in a production cluster. Why it's wrong: This makes managing resource quotas, RBAC, and network policies significantly more complex and increases blast radius. Fix: Implement a multi-tenancy strategy using namespaces to isolate environments and services.
- Mistake: Ignoring Azure CNI IP exhaustion. Why it's wrong: Azure CNI assigns a real IP address from your VNet to every single pod, which can quickly deplete the subnet if not planned correctly. Fix: Use the Azure CNI Overlay plugin or properly size your subnets based on maximum pod count per node.
Interview questions
What is Azure Kubernetes Service (AKS) and why would you choose it for your applications?
Azure Kubernetes Service (AKS) is a managed container orchestration service provided by Microsoft Azure that simplifies deploying, managing, and scaling containerized applications using Kubernetes. You would choose AKS because it offloads the operational burden of maintaining the Kubernetes control plane to Microsoft. This allows your team to focus on development rather than patching the master nodes or managing complex infrastructure, while still benefiting from integrated Azure networking, identity, and monitoring tools.
How does Azure Active Directory (Azure AD) integration improve the security of an AKS cluster?
Integrating AKS with Microsoft Entra ID (formerly Azure AD) improves security by moving away from static, shared cluster certificates toward centralized, identity-based access control. With this integration, you can use Role-Based Access Control (RBAC) to manage access to the Kubernetes API based on existing Azure organizational roles. This ensures that only authenticated users with specific permissions can execute commands like `kubectl get pods`, and it provides an audit trail of who accessed the cluster and when.
What are the primary differences between using Azure CNI and Kubenet for networking in AKS?
Azure CNI gives each pod an individual IP address from the virtual network subnet, which simplifies communication and security group management because pods appear as first-class citizens in the Azure network. Kubenet, conversely, uses a separate address space for pods that is NATed behind the node's IP. You should choose Azure CNI when you have a large available IP range and require deep integration with Azure virtual network policies and performance, whereas Kubenet is preferred when IP address scarcity is a concern.
How do you achieve high availability for your services running on AKS?
To achieve high availability in AKS, you must design for resilience at multiple layers. First, ensure you are using a multi-node cluster spread across multiple Azure Availability Zones. Next, utilize Horizontal Pod Autoscaler (HPA) to scale pods based on CPU or memory usage. Finally, implement Azure Load Balancer or Azure Application Gateway with an Ingress controller to manage incoming traffic, ensuring that if one pod or node fails, traffic is automatically rerouted to healthy replicas to maintain uptime.
Compare the use of Azure Container Registry (ACR) versus using a public registry for hosting your images on AKS.
Using Azure Container Registry (ACR) provides a private, secure repository that resides within the Azure network, offering significant latency and security advantages over public registries. When using ACR, you can leverage Service Principals or Managed Identities to pull images without embedding secrets in your Kubernetes manifests. Furthermore, ACR supports geo-replication and content trust, which ensures that your images are cryptographically signed and always available near your cluster nodes, drastically reducing deployment times and enhancing the overall security posture.
How can you implement fine-grained traffic management and security using the Azure Service Mesh add-on for AKS?
The Azure Service Mesh (ASM) add-on, powered by Istio, provides a robust control plane to manage communication between microservices within an AKS cluster. It allows you to implement traffic splitting for canary deployments, mutual TLS (mTLS) for encrypted service-to-service communication, and detailed observability metrics without changing application code. By injecting a sidecar proxy, ASM captures all network traffic, giving you the power to define advanced egress/ingress gateway rules and policy-driven load balancing to secure and optimize complex, distributed Azure-based application architectures.
Check yourself
1. An administrator wants to ensure that specific sensitive keys are only available to a pod without being persisted in the cluster's etcd database. Which approach is best?
- A.Create a base64 encoded Kubernetes secret and mount it as an environment variable.
- B.Store the keys in Azure Key Vault and use the Secrets Store CSI Driver to mount them as a volume.
- C.Include the keys in a ConfigMap and mount it to the container filesystem.
- D.Hardcode the credentials in the container image and pass them via command-line arguments.
Show answer
B. Store the keys in Azure Key Vault and use the Secrets Store CSI Driver to mount them as a volume.
The CSI driver fetches secrets from Azure Key Vault directly at runtime, avoiding storage in etcd. Option 0 is insecure, Option 2 exposes configuration data, and Option 3 is a major security violation.
2. A team is experiencing 'Noisy Neighbor' issues where one workload consumes all node memory, causing other pods to crash. What is the most effective way to prevent this?
- A.Increase the number of nodes in the node pool manually.
- B.Enable Cluster Autoscaler to add more capacity on demand.
- C.Define resource requests and limits in the deployment YAML for each container.
- D.Deploy all applications into separate dedicated node pools.
Show answer
C. Define resource requests and limits in the deployment YAML for each container.
Resource requests and limits enforce boundaries, preventing one pod from starving others. Autoscaling helps with total capacity but does not solve the local 'noisy neighbor' contention problem.
3. When designing a production-grade AKS cluster, why is it recommended to use a separate Azure Container Registry (ACR) over a public registry?
- A.Public registries do not support large container images.
- B.Using ACR allows for Private Link integration, keeping image traffic within the Azure network.
- C.AKS is only compatible with images stored in ACR.
- D.ACR automatically patches the application code inside the containers.
Show answer
B. Using ACR allows for Private Link integration, keeping image traffic within the Azure network.
Private Link secures traffic and ensures the registry is not exposed to the public internet. AKS can pull from any registry, and ACR does not modify application code.
4. A developer is using Azure CNI and finds that the subnet is running out of IP addresses despite the cluster having a low number of nodes. What is the cause?
- A.Azure CNI reserves an IP for every pod, not just every node.
- B.The cluster is using too many LoadBalancer services.
- C.The node OS is consuming all available IP addresses.
- D.The Azure CNI plugin has a default hard limit of 10 nodes per cluster.
Show answer
A. Azure CNI reserves an IP for every pod, not just every node.
Azure CNI assigns VNet IPs to pods. Option 1 is incorrect as services use different IP pools. Option 2 refers to OS management, and Option 3 is simply false.
5. Which action should be taken when performing a maintenance update on an AKS cluster to ensure minimal impact on application availability?
- A.Shut down the entire node pool before starting the upgrade.
- B.Delete all existing pods before initiating the cluster upgrade.
- C.Configure Pod Disruption Budgets to ensure a minimum number of pods remain available.
- D.Use a single node pool for both system and user services to streamline the process.
Show answer
C. Configure Pod Disruption Budgets to ensure a minimum number of pods remain available.
Pod Disruption Budgets prevent voluntary disruptions from taking down too many replicas simultaneously. Shutting down nodes or deleting pods manually would cause unnecessary downtime.