The Data Standard

The Data Standard Audio Experience with Daniel Peck from PepsiCo
Daniel Peck, Director of Engineering, Data Platform at PepsiCo, joins The Data Standard to discuss the similarities and differences between data engineering and software engineering.

Episode Summary

As the world consumes more and more data, businesses increasingly require experts who know how to handle large amounts of customer information, competitor research and product performance results. They also need professional developers to build and manage their increasingly complex websites and apps. These two distinct needs have led to two exciting tech fields: data engineering and software engineering. The two sometimes overlap, but actually have different areas of expertise.

To those outside the tech world, terms like “data engineer” and “software engineer” may sound interchangeable, since both rely heavily on programming skills. Some may even assume that data engineers and software engineers do the same thing.

In reality, software engineers usually only engage in data infrastructure to a limited extent. Their primary focus is on building websites that function well for the end-user. The task of data engineers, however, is to maintain large datasets and build the structures that house them. So while each profession involves a common background of data management, software engineers and data engineers have vastly different responsibilities.

In short, data engineers examine the practical applications of data collection and help in the process of analysis. Data engineers work closely with large datasets, and build the structures that house that data long-term. Data engineers should be well versed in programming languages like SQL that allow them to build, maintain and secure big data. Most data engineers have been software engineers at some time in their career and have moved on to specializing in data engineering.

Software engineers are also skilled in computer programming, but they perform work that includes designing and developing operating systems, as well as front and back-end development. While some software engineers rely heavily on data, you’re not guaranteed to focus on data analysis if you pursue a career in engineering. Engineers who focus on front-end web development, for example, are much more concerned with the look and feel of a website, rather than organizing the data on the back end.

Meet the Hosts

Catherine Tao

Data scientist at The Data Standard
Catherine Tao is a tech enthusiast looking for new methods for building connections with businesses around the world. Her extensive knowledge of data science allowed her to develop new solutions and implement them into existing ecosystems. She is currently working as a Data scientist and Exclusive Podcast Producer at The Data Standard.

Daniel Peck

Director of Engineering, Data Platform at PepsiCo
I’m a technical people person with expertise in building & scaling distributed systems and teams. I’m still a hacker at heart and I love moving fast and making things. But my favorite thing is helping people and the organizations that they’re a part of to level up, and become more effective and efficient at whatever they’re focused on. Outside of work I love to cook and bringing people together for meals and celebrations of the ordinary and extraordinary parts of life. More recently I’ve taken up cycling and playing guitar.