DEVKUBEDOC [2 min read]¶
Documentation for Kubernetes for development and data science purposes. This repository serves as complementary configuration and documentation for Kubernetes cluster deployed at the CTTC for Sustainable AI Research Unit.
Main purpose of this Kubernetes implementation is to deliver an experimental data science programming and testing environment to Sustainable AI Research Unit. An environment where all computational resources could be effectively shared among the Research Unit members and/or internal/external collaborators. An environment that is easy to extend by new servers financed by new grants. An environment that is easy to monitor and administer.
Other advantages include: 1. use of Kubernetes infrastructure for simulation of Federated learning scenarios, e.g EasyFL or KubeFL 2. simulation of IIoT/IoT sensor deployment and analysis of the traffic between deployed pods(sensors), e.g. IoTSimulator, Kubeshark 3. MLOps, monitoring of models performance, ML pipelines etc. by easy helm chart deployment of e.g. MLflow, AirFlow 4. and many more, only sky is the limit...
I chose Microk8s Kubernetes implementation as it includes all necessary elements for JupyterHub, it is lightweitgh supports NFS, GPU, NGINX and Ingress out of the box and is easy to administer, e.g see this comparison.