Data Engineer

Job Description:

We are seeking a skilled and motivated Data Engineer to design, build, and optimize scalable data pipelines and architectures in support of advanced analytics and business intelligence initiatives. The ideal candidate will have hands-on experience with modern data platforms and a strong foundation in data modeling, pipeline orchestration, and cloud-native data services. This role involves collaborating cross-functionally to ensure the availability, integrity, and performance of data assets across the enterprise.

Job description:

• Design and develop end-to-end ETL/ELT pipelines using Python, SQL, and Apache Spark to process large-scale structured and semi-structured data.
• Build and maintain real-time and batch data ingestion workflows using tools like AWS Glue, S3, Snowpipe, and Databricks.
• Orchestrate and automate data workflows using Apache Airflow, Snowflake Tasks, and parameterized scripts to ensure reliable, scalable operations.
• Develop star schema and dimensional data models to support scalable data warehousing and self-service BI.
• Flatten complex data structures into semantic layers to support easy consumption by analytics and reporting tools.
• Collaborate with data analysts, BI developers, and business stakeholders to define data requirements and deliver actionable insights.
• Monitor and optimize performance of Snowflake queries and pipelines to ensure efficient resource utilization and data freshness.
• Implement robust data quality checks, conduct unit and integration testing, and ensure data consistency across systems.
• Maintain clear documentation for data pipelines, data dictionaries, and data lineage to support data governance.
• Participate in Agile ceremonies, including stand-ups and sprint reviews, and use Git for version control and collaborative development.
• Support ad-hoc data analysis and reporting by writing complex SQL queries and transforming raw data into meaningful insights.