Understanding and unlocking the potential of streaming analytics with Kedro — As a data practitioner, there are scenarios in which working with real-time data streams is essential to delivering timely insights. Unlike batch analytics, which involves processing large volumes of data in predefined batches, streaming analytics processes real-time data as it arrives, offering immediate insights and enabling real-time decision-making. Streaming data…

For both new and experienced data practitioners, managing data is not a trivial matter. Organizations increasingly rely on efficient data collection, storage, and access so cross-functional teams can use it effectively, across the business. The management of these vast quantities of data residing in a central data lake can be…

Kedro is an open-source Python toolbox that applies software engineering principles to data science code, making it easier to transition from prototype to production. Created by QuantumBlack, AI by McKinsey, it was open-sourced in 2019, and donated to the Linux Foundation in 2021. Kedro is constantly being improved and in…

The recruitment and retention of women in tech is key to the sector’s growth. That’s according to a recent McKinsey report, which proposes that doubling the number of women in the tech workforce by 2027 could close the talent gap it currently faces and increase its economic output. …

Our team has shared the most effective ways to keep data safe, including key techniques such as tokenisation, suppression and cryptographic encryption. Data-driven solutions help organisations make better decisions, improve efficiency, create better experiences for customers and ultimately bring in more revenue. But the growth of big data is outpacing…

There’s never been a more exciting time to work in data. As artificial intelligence (AI) and machine learning (ML) continue to evolve, the potential of what can be achieved appears to be near limitless. Innovations in AI will continue to be some of the most transformative technologies impacting how we…

We’ve added datasets and documentation enhancements to the recent 0.18.4 release of Kedro — Jo Stichbury, Technical Writer, QuantumBlack Labs Since its inception there have been some major milestones in the lifetime of Kedro. From being open-sourced in 2019, to being donated to the Linux Foundation. Kedro is constantly being developed and the latest release, made in December 2022, brings a raft of changes…

We have identified 8 key traits of a data engineer that will help practitioners become highly productive and build an ecosystem for success. About five years ago, Maxime Beauchemin, creator of Apache Airflow and Apache Superset, talked about data engineers (DE) having the worst seat at the table. Thankfully, much…

Reducing bias through model development is only one part of dealing with fairness in AI. Practitioners can take these steps to increase AI model fairness. At a basic level, AI learns from our history. Unfortunately, much of societal history includes some discrimination and inequality. It’s therefore essential that data practitioners…