Data Engineer, Specialist

{"description": " Role Summary

The Data Engineer, Specialist is responsible for building and maintaining scalable data pipelines to ensure seamless integration and efficient data flow across various platforms. This role involves designing, developing, and optimizing ETL (Extract, Transform, Load) processes, managing databases and leveraging big data technologies to support analytics and business intelligence initiatives.

Responsibilites
1. Design, develop and maintain scalable ETL (Extract, Transform, Load) processes to efficiently extract data from various structured and unstructured sources, ensuring accuracy, consistency and performance optimization.
2. Architect and manage database systems to support large-scale data storage and retrieval, ensuring high availability, security and efficiency in handling complex datasets.
3. Integrate and transform data from multiple sources including APIs, on-premises databases and cloud storage, creating unified datasets to support data-driven decision-making across the organization.
4. Collaborate with business intelligence analysts, data scientists and other stakeholders to understand specific data needs, ensuring the delivery of high-quality, business-relevant datasets.
5. Monitor, troubleshoot and optimize data pipelines and workflows to resolve performance bottlenecks, improve processing efficiency and ensure data integrity.
6. Develop automation frameworks for data ingestion, transformation and reporting to streamline data operations and reduce manual effort.
7. Work with cloud-based data platforms and technologies such as AWS (Redshift, Glue, S3), Google Cloud (Big Query, Dataflow), or Azure (Synapse, Data Factory) to build scalable data solutions.
8. Optimize data storage, indexing, and query performance to support real-time analytics and reporting, ensuring cost-effective and high-performing data solutions.
9. Lead or contribute to special projects involving data architecture improvements, migration to modern data platforms, or advanced analytics initiatives.

Qualifications and Skills

• Minimum of five years data analytics, programming, database administration, or data management experience.
  Undergraduate degree or equivalent combination of training and experience.
• Strong expertise in SQL, Python, or Scala for data processing, automation and transformation.
• Hands-on experience with big data frameworks such as Apache Spark, Hadoop, or Kafka for real-time and batch processing.
• Experience in cloud data platforms including AWS Redshift, Google Big Query, or Azure Synapse, with proficiency in cloud-native ETL and data pipeline tools.
• Strong understanding of data modeling principles, relational and NoSQL databases (PostgreSQL, MySQL, MongoDB, Cassandra).

Special Factors

Sponsorship
Vanguard is not offering visa sponsorship for this position.

About Vanguard

At Vanguard, we don't just have a mission-we're on a mission.

To work for the long-term financial wellbeing of our clients. To lead through product and services that transform our clients' lives. To learn and develop our skills as individuals and as a team. From Malvern to Melbourne, our mission drives us forward and inspires us to be our best.

How We Work

Vanguard has implemented a hybrid working model for the majority of our crew members, designed to capture the benefits of enhanced flexibility while enabling in-person learning, collaboration, and connection. We believe our mission-driven and highly collaborative culture is a critical enabler to support long-term client outcomes and enrich the employee experience.", "salary_raw": "Row(double=None, string=None)"}