Data Engineer

Job title: Data Engineer

Mode of work: Remote (Preferably from PST or MST)

Duration: 12 months

Job description:

• Design, build, and maintain scalable and auditable data pipelines using Databricks (Spark/Scala/Python/SQL) and Snowflake
• Own the data validation lifecycle including source-to-target reconciliation using Oracle, data profiling, and anomaly detection
• Conduct deep-dive data mining to uncover patterns, inconsistencies, and opportunities for business action.
• Develop and maintain data visualizations and dashboards in Tableau to support transparency and decision-making
• Implement data quality checks and automated alerting for pipeline failures, anomalies, and audit flags
• Collaborate with analysts, scientists, and business stakeholders to gather requirements, and deliver reliable data products.
• Optimize existing pipelines for performance, scalability, and operational simplicity
• Document data lineage, pipeline logic, and metadata clearly and consistently

Required Skills:

• -Hands-on experience designing and maintaining scalable data pipelines using Databricks (Spark/Scala/Python/Sql) and Snowflake
• -Strong SQL development skills for data extraction, transformation, and performance tuning
• -Ability to conduct source-to-target data reconciliation, auditing, and anomaly detection across large datasets
• -Proven experience troubleshooting Spark performance issues
• -Expertise in Snowflake data loading and update patterns
• -Tableau dashboard development, including parameters, performance optimization, and data-driven visualization
• -Solid understanding of data validation lifecycle
• -Ability to document pipelines, workflows, and requirements

Preferred Skills:

• -Familiarity with advanced data profiling, anomaly detection techniques, and predictive modeling for identifying inconsistencies
• -Experience collaborating with business stakeholders to translate analytical requirements into reliable data products
• -Knowledge documenting data lineage, pipeline logic, and metadata management practices
• -Strong communication skills with the ability to clearly explain technical findings to non-technical stakeholders
• -Demonstrated ability to work independently with minimal oversight while managing priorities effectively