My Experience with DevOps and DataOps | by Madison Schott | Nov, 2023

How these two data roles are similar yet very different

When I first started as a data engineer, I worked on a DevOps-focused team. While it wasn’t exactly what I wanted to be doing in my first role, it taught me a lot. Now looking back, if I hadn’t worked in that type of role then, I probably wouldn’t have the experience I have as an analytics engineer today.

Now, working as an analytics engineer, I focus on something called DataOps. While this may sound similar to DevOps, they are very different. DevOps focuses on software as the product while DataOps focuses on producing high-quality data. For those focusing on DataOps, data is the product!

While working as a DevOps data engineer, I supported software engineers making code changes to our web application. I focused on testing changes in the UI after each deployment rather than looking into the specifics of the data. Not once did I check on the number of rows in a table or if values in a field were populated. Instead, I made sure no errors were being thrown on the backend.

As an analytics engineer, any time I make a code change or push something to production, I need to focus on the metadata—or the data about the data. This involves writing validation queries to ensure things like row count, column count, and distribution of values look as they did before I pushed a change. Or, if I want them to look different than they did before, they reflect those changes!

Although DevOps and DataOps sound similar, they serve two different purposes. In this article we will dive deeper into the differences, touching on the product they aim to serve and the varying metrics of success.

DevOps involves the deployment and testing of software code changes. When I worked as a DevOps engineer it often involved long deployment nights, testing code changes in many different environments, and validating the changes with the software engineers that made them.

Software as a product