Observability: A Data Engineering Challenge

In recent years, observability has emerged as a cornerstone for ensuring system reliability, performance, and security. At its core, observability is about gaining a deep understanding of what’s happening inside a system by collecting, analyzing, and acting on data. While it’s tempting to view observability through the lens of monitoring tools and practices, a significant portion of the challenge - and opportunity - lies within the realm of data engineering....

February 8, 2024

Rethinking Custom Solutions

The decision to create bespoke software solutions from the ground up often comes from a place of seeking perfect alignment with unique business needs. However, this approach may overlook the substantial benefits and strategic advantages of harnessing existing open-source and proprietary software. The Temptation of Custom Development Custom-built software projects are frequently driven by the allure of a perfect fit - a solution meticulously tailored to an organization’s specific requirements....

February 7, 2024

Should You Use Kubernetes?

I get this question a lot: “Should I use Kubernetes?” It’s no surprise, as Kubernetes has taken the tech world by storm in recent years. As the popularity of containerization and microservices architecture continues to rise, Kubernetes has become a go-to solution for container orchestration. However, deciding whether or not to adopt Kubernetes for your projects can be a daunting task. In this post, I will guide you through the pros and cons of Kubernetes to help you make an informed decision....

February 6, 2024

Inner Source: A Path to Enhanced Collaboration and Innovation

The concept of Inner Source, or Internal Open Source, is gaining significant traction among forward-thinking organizations. This approach, inspired by the principles of open source software, involves leveraging open source methodologies within the confines of an organization. It encourages transparency, collaboration, and shared responsibility in software development, leading to a myriad of benefits that can propel organizations toward greater innovation and efficiency. The Essence of Inner Source Inner Source takes the best practices from the open source world, such as collaborative development, code reuse, and community building, and applies them internally....

February 5, 2024

What Observability is Not

Observability is a critical concept for software engineering. It refers to the ability to gain insights into the inner workings of your software applications, helping you to understand, troubleshoot, and optimize them effectively. While observability has gained immense popularity, there are still some misconceptions surrounding it. Observability Is Just Another Term for Monitoring One of the most common misconceptions is that observability and monitoring are interchangeable terms. However, they are not the same thing....

February 2, 2024