Kedro Joins the Linux Foundation to Become an Open Standard for Machine Learning Engineering
Yetunde Dada, Director of Product Management; Ivan Danov, Principal Machine Learning Engineer; Joel Schwarzmann, Product Manager; Jeremy Palmer and Matt Fitzpatrick, Co-Leaders, QuantumBlack Labs; Alex Singla and Alex Sukharevsky, Global Co-Leaders, QuantumBlack
QuantumBlack, a McKinsey company, is delighted to share that we have donated Kedro to the Linux Foundation (LF), one of the world’s largest vendor-neutral consortiums for open-source projects. Specifically, Kedro will be incubated in LF AI & Data, an umbrella foundation launched in 2018 with the mission to support and accelerate the development and innovation of open-source projects in artificial intelligence and data. The foundation also facilitates continued collaboration on these projects between companies and the global AI and Data developer communities.
What this means for the Kedro community
The move marks a next step for QuantumBlack and McKinsey’s open-source journey. Kedro is no longer a sole QuantumBlack-owned product but a project in the hands of the worldwide community, to be grown and maintained by an active global developer network.
As an LF AI Data Incubation project, our team will be adopting an open governance approach that makes it easier than ever to play a significant role in Kedro’s continued growth. Developers can explore what the team is working on, view the backlog or status of the current sprint and contribute according to the Kedro roadmap.
This new standardised contribution workflow means that anyone can join in Kedro’s continued development and eventually progress into becoming an official maintainer on the project, writing code to continually improve the central framework. In return for a weekly time commitment, maintainers will join Kedro’s Technical Steering Committee and help shape product strategy and roadmap decisions through regular voting. We’ll be updating our existing contributing documentation in the near future to explain the route and requirements to becoming an official project maintainer.
LF AI & Data has created a community of over 18,000 individual developers as well as a significant number of partnering companies that have donated their own projects. These projects continue to be maintained and updated in collaboration with other organisations, following a neutral governance model and supported by a host of enabling services offered by the LF AI & Data Foundation. Other incubation stage projects include the likes of Feast(originally developed by Gojek and Google Cloud) as well as Amundsen and Flyte (both donated by Lyft). Joining the LF AI & Data community provides tremendous opportunities to collaborate with these exciting projects tackling different parts of the ecosystem.
Why we’re donating Kedro
This move now offers a tremendous springboard to launch the project into its next phase and marks a new step for QuantumBlack and McKinsey as we continue to embrace open-source values. Moreover, there will be plenty of ways for the community to become even more closely involved with Kedro’s future as we switch to an open development model.
First founded more than 20 years ago, the Linux Foundation has become known as the organisation of choice for the world’s top developers and companies to build ecosystems that accelerate open technology development and commercial adoption. The network counts more than 1,800 corporate organisations as members and Kedro joins a collection of hundreds of other cutting edge open-source projects hosted by the Linux Foundation.
Acceptance into LF AI & Data is a significant mark of validation for Kedro as a de-facto industry standard tool, similar to other LF projects such as Kubernetes (donated by Google), GraphQL (donated by Facebook) or MLFlowand Delta Lake (donated by Databricks). Moreover, it’s an exciting next step for the project — entering the incubation phase with LF AI & Data means that other member companies and organisations will have the opportunity to collaborate on the project.
From a personal perspective, we’re incredibly excited about what this donation symbolises for QuantumBlack and McKinsey. It’s an enormous mark of success after five years of hard work by colleagues across our organisation, from QuantumBlack and McKinsey R&D, engineering and project teams to our tremendous product engine room QuantumBlack Labs. Moreover, it’s a substantial step forward for our organisation on our open-source journey. The consultancy sector has traditionally been highly protective of intellectual property, but it’s clear that open, collaborative innovation will help power the next phase for analytics technology and unlock truly transformative solutions. We can’t wait to watch Kedro’s progress as a Linux Foundation product and how it evolves with the support of the worldwide developer community.
We would like to thank the existing global Kedro community for your support to date and we look forward to working even more closely with you as we enter this new phase. You can follow the next stage of the Kedro project on GitHub — Kedro v0.18.0 will be our next release, with the path towards v1.0 and a stable API clearly in our sights. Be sure to subscribe to the Kedro-Announce and Kedro-Technical-Discuss mailing lists to stay connected on the latest updates, alongside joining the community on the Discord server and the GitHub repo.
