Search for a command to run...
Series
Here, I will discuss different data engineering concepts and tools. Might mess around with different tools and produce how-to posts. Only time will tell tbh 馃. stay tuned!!
Before diving in, let us discuss what data quality entails. IBM puts it so well - Data quality measures how well a dataset meets criteria for accuracy, completeness, validity, consistency, uniqueness,
This post aims to provide a historical picture of the evolution of the typical data stack over a span of about 5 decades. A lot has happened, but I will try to keep it as simple and digestible as possible. Fig 1: Evolution of the tools, concepts and...
Introduction This article will discuss compression in the Big Data context, covering the types and methods of compression. I will also highlight why and when each type and method should be used. Diving in What is compression? According to the gene...
There's often a huge fuss about making data-driven decisions, leveraging data analytics, using data science and data-centred thinking. From a technological point of view, data is usually stored and accessed using databases. Databases, often referred ...
This post draws extensively from a book I am currently reading and my experience managing data systems (primarily, core data infrastructure). The book is Designing Data-Intensive Applications by Martin Kleppmann. Hey! What is even a Data System? Firs...
