On July 25th, The Cloud Native Computing Foundation (CNCF) accepted Kubeflow as an incubation project. This move signals an important stride for the open source Kubernetes machine learning platform.
Kubeflow is an open source project founded by Google. The project was internally started at Google to run TensorFlow at scale on containers.
At KubeCon + CloudNativeCon Europe 2018, the first Kubeflow release (Kubeflow 0.1) was announced with claims of already ranking in the top 2% of all GitHub projects. A public blog post announcing the graduation of numerous Kubeflow components to “stable status”—i.e., the point at which they are now prepared for use in production—was used to announce the release of Kubeflow 1.0 in March 2020.
Some of the key contributors of Kubeflow include AWS, Bloomberg, Google, IBM, NVIDIA, Nutanix, Red Hat, and Arrikto. AWS and Microsoft have built connectors for Kubeflow that integrate with their ML PaaS offerings – Amazon SageMaker and Azure ML.
Enterprises considering a portable and open source machine learning platform rely on Kubeflow. It can be deployed on any Kubernetes cluster running in the public cloud or in an on-premises environment without any lock-in.
Kubeflow can run mainstream ML framework including TensorFlow, PyTorch and Apache MXNet. It supports a scalable development environment based on Jupyter Notebooks, which is a standard tool used by data scientists and ML developers.
Kubeflow project is not just confined to training. It is capable of running an end-to-end MLOps pipelines that include data preprocessing, feature engineering, model training, hyperparameter tuning, AutoML, and model serving.
The CNCF Technical Oversight Committee (TOC) has voted to accept Kubeflow as a CNCF incubating project. Kubeflow already integrates with a number of CNCF projects, including Argo, Cert-Manager, and Istio. Kubeflow’s affiliation with the CNCF will help to solidify these integrations and promote increased collaboration among cloud native projects, resulting in even more durable and cutting-edge solutions for customers.
As a project hosted by CNCF, Kubeflow becomes a part of a neutral foundation that aligns with its technical interests and the larger Linux Foundation, which offers governance, marketing assistance, and community outreach. The initiative joins 38 other in-development technologies, including Backstage, Cilium, Istio, Knative, OpenTelemetry, and others.
Kubeflow is getting ready for the 1.8 release scheduled for October 2023. Some of the new features include the general availability of Kubeflow Pipelines 2.0, new AutoML experiment features. The 1.8 release will be tested with defined dependency versions of Kubernetes, Kustomize, Istio, Certificate Manager, Argo, and Knative.
With the growing interest and adoption of large language models and generative AI foundation models, enterprises need a reliable platform to manage the lifecycle of these models. Kubeflow is positioned as a strong candidate for hosting the foundation models and also enabling advanced scenarios such as fine-tuning and integration with existing systems.
CNCF’s acceptance of Kubeflow as an incubation project marks an important milestone in the evolution of the open source MLOps platform.
Read the full article here
