Building AI-Driven Cloud-Native Applications with Kubernetes and Containerization

Abstract

Modern enterprises increasingly deploy AI-driven services in cloud environments, demanding scalable infrastructure that aligns machine learning operations (MLOps) with cloud-native principles. This paper proposes a Kubernetes-based architecture for developing and deploying AI applications, emphasizing containerization, orchestration, and continuous delivery. The architecture supports end-to-end MLOps workflows from training and versioning to real-time inference and monitoring using open-source tools on managed Kubernetes services (e.g., Amazon EKS, Azure AKS, Google GKE). Core components include Kubeflow Pipelines for orchestration, Ml flow for model registry, Argo Workflows for automation, and serving frameworks such as TensorFlow Serving and ONNX Runtime for scalable inference. Cloud-native features like autoscaling, service mesh, observability, and security are integrated using tools such as Prometheus, Grafana, Trivy, and Vault. The architecture is validated through two use cases: an e-commerce recommendation service and an IoT anomaly detection pipeline, with a proof-of-concept deployed on AWS. Experimental results demonstrate low-latency inference (95th percentile latency under 120, ms at 100 requests/s) and efficient resource utilization. The platform enhances reproducibility, monitoring, and deployment speed over traditional ML deployment approaches. These findings highlight the advantages of Kubernetes-native MLOps for scalable, reliable AI systems in production environments.