Principal Data Engineer

iO Associates is currently partnered with a custom software and technology consultancy that is seeking a skilled Principal Data Engineer.

Their in-house product teams' partner with Fortune 500s to navigate the unknown - that space between an identified problem and a future-focused solution. They are currently seeking a Senior Data Engineer, as they continue to build out their development team.

Responsibilities

• Lead the design, development, and optimisation of ETL data pipelines using Golang or Python.


• Collaborate with cross-functional teams to architect scalable data solutions on platforms such as Databricks, Snowflake, or similar.


• Ensure data quality, security, and compliance standards are upheld across all pipelines.


• Mentor and guide junior team members, fostering a culture of continuous improvement.


• Troubleshoot and resolve complex technical issues to ensure seamless data flow.


• Contribute to strategic decisions regarding data architecture and technology stack.

Essential Skills & Experience

• Extensive experience developing ETL pipelines using Golang and/or Python.


• Proven expertise working with data platforms such as Databricks, Snowflake, or comparable solutions.


• Strong knowledge of distributed data processing and scalability best practices.


• Demonstrable leadership experience, with the ability to guide technical teams and influence project outcomes.


• Excellent problem-solving skills and attention to detail.


• Experience working in fast-paced, collaborative environments.


• Senior/ Principal level with Consulting experience

Desirable Skills & Experience

• Familiarity with cloud infrastructure services such as AWS, Azure, or Google Cloud Platform.


• Knowledge of containerisation and orchestration tools like Docker and Kubernetes.


• Prior experience in data engineering within large-scale enterprise environments.


• Exposure to DevOps practices and CI/CD pipelines.


• Advanced understanding of data security and compliance standards.

If you are seeking an opportunity to learn new technologies, make decisions that influence the path of a product roadmap, and impact hundreds and thousands of users every day with the products you build, apply now to dive deeper into this opportunity!