Assist in building and maintaining ETL/ELT pipelines for acquisition reporting using Databricks, PySpark, SQL, and Unity Catalog under the guidance of senior engineers.
Support the migration of existing Snowflake SQL scripts and transformations into Databricks UC by updating queries, validating outputs, and helping implement governance best practices.
Contribute to developing ingestion processes for marketing vendor data, including data parsing, normalization, and basic quality validations.
Top 5 must have skills needed for this role:
4+ years of experience as a Data Engineer or similar role.
Strong proficiency in SQL (analytical SQL, complex joins, window functions).
Hands-on experience with PySpark and/or Spark SQL in production.
Good understanding of data modeling, ETL/ELT design patterns, and distributed data processing.
Experience building pipelines in Databricks, including Delta Lake, Unity Catalog, data governance, and Lakehouse patterns.
Key Responsibilities
Assist in building and maintaining ETL/ELT pipelines for acquisition reporting using Databricks, PySpark, SQL, and Unity Catalog under the guidance of senior engineers.
Support the migration of existing Snowflake SQL scripts and transformations into Databricks UC by updating queries, validating outputs, and helping implement governance best practices.
Contribute to developing ingestion processes for marketing vendor data, including data parsing, normalization, and basic quality validations.
Implement and maintain foundational data quality checks, monitoring alerts, and issue triage workflows using Databricks, Snowflake, Airflow, and internal tooling.
Partner with the Data Reliability Engineering team to assist with SLA monitoring, simple incident troubleshooting, and logging improvements.
Collaborate with analytics and marketing partners to understand data requirements and ensure accuracy of datasets used in dashboards and reporting.
Support performance tuning, logging improvements, and general pipeline reliability work.
Participate in engineering best practices, including code reviews, documentation, and contributing to shared frameworks and tools.
Basic Qualifications
4+ years of experience as a Data Engineer or similar role.
Strong proficiency in SQL (analytical SQL, complex joins, window functions).
Hands-on experience with PySpark and/or Spark SQL in production.
Good understanding of data modeling, ETL/ELT design patterns, and distributed data processing.
Experience building pipelines in Databricks, including Delta Lake, Unity Catalog, data governance, and Lakehouse patterns.
Experience in AWS (S3, IAM, EC2, Glue, Lambda, or related services).
Experience with Airflow or similar orchestration tools.
Experience building robust ingestion pipelines and working with semistructured formats (JSON, Parquet, CSV).
Experience with Git/GitHub, CI/CD, and modern DevOps practices.
Excellent communication skills and ability to work with crossfunctional partners.
