Data Architecture

There are some really messy things about data architecture that I enjoy talking about, but most people go glassy-eyed when I talk about them. Typically I will only talk about these subjects with other data developers and engineers, but I think it’s time that others outside that domain get a little peek into data architecture work. The primary responsibility of a data architect is to develop scalable solutions to data challenges. The data architect designs plans for integrating, centralizing, protecting, and maintaining organizational data assets, so that employees or customers can get access to critical information at the right time and in the right place. He or she is familiar with how data flows from sources all the way through to their destination and final resting places. The data architect uses a variety of tools for ETL/ELT (Extract-Transform-Load, and Extract-Load-Transform), structured or unstructured databases (SQL vs. Hadoop, etc), and even design tools such as ERWin or ER Studio. Many data architects may also use software tools in a category called data warehouse automation, some of which include Informatica, Dimodelo, Wherescape, and Kalido. The data architect is an expert in SQL and at least one other programming language like Java, C#, or Python.

Interestingly, data architects play different roles depending on the organization that they work for. For example, in smaller organizations where he or she may be the only data architect, he or she may be involved in every single aspect from strategic planning all the way through coding and scripting ETL jobs. Indeed, the data architect has a diverse skill set and many of these skills can be used on a daily basis. In other organizations, perhaps a data architect works with a team of other data architects, analysts, and solution designers, so he or she may play more of a role of technical leader and high-level designer. Most data architects are still very hands-on and are involved directly in their data-related projects. A blend of technical and managerial abilities are needed to be successful as a data architect. Perhaps communication skills and team skills are one of the most important aspects of this type of role, and the ability to explain technical concepts to the uninitiated (from executives to end-users and everyone in-between). Senior level data architects may also lead other developers and architects in data-related projects, so the ability to coach and lead others becomes as equally important as technical skills.