We are seeking a highly skilled, solution-oriented Senior Data Integration Engineer with deep expertise in modern data engineering, cloud-native architectures, and robust pipeline development. In this role, you will be a senior technical driver in creating, optimizing, and modernizing advanced data integration systems. You will collaborate closely with product, engineering, and architecture teams to bridge raw source environments and analytical databases. If you excel in cloud ecosystems, thrive on automating complex data workflows, and value precision, reliability, and data quality above all, we encourage you to apply! Responsibilities Pipeline Architecture & Development: Design, build, and optimize scalable, reliable batch and near-real-time ETL/ELT pipelines using Python, PySpark, SQL, and modern cloud integration engines Orchestration & Automation: Develop and manage complex workflow orchestrations (using Apache Airflow or cloud native schedulers) and automate ingestion routines to minimize manual operations Data Modeling & Warehousing: Design and implement modern data warehouse/lakehouse layers (using Snowflake, ClickHouse, Azure Synapse, or Redshift), establishing optimal partitioning, indexing, and Slowly Changing Dimension (SCD Type 2) patterns Data Quality & Testing Integration: Establish rigorous data quality checks and validation frameworks utilizing tools like dbt (data build tool), Soda, or customized PySpark testing suites Collaboration & Design: Work closely with product owners, business analysts, and systems architects to define data requirements, analyze technical constraints, design Source-to-Target Mappings (STTM), and make critical architectural decisions Code Quality & DevOps: Maintain a clean, modular code repository. Lead code reviews, enforce engineering standards, and configure robust CI/CD pipelines (Azure DevOps, GitLab CI, or GitHub Actions) with Docker containers Technical Documentation: Deliver comprehensive, clear technical specs, metadata lineage documentation, architectural diagrams, and data dictionaries Requirements Experience: 5+ years of hands-on experience in data engineering, data warehousing, database design, and end-to-end data integration ETL & Integration Tools: Advanced knowledge of Cloud Integration tools such as Azure Data Factory (ADF), AWS Glue, or Google Cloud Platform Dataflow Orchestration & Real-Time Ingestion: Proficiency in workflow orchestrators like Apache Airflow and exposure to CDC (Change Data Capture) or real-time streaming tools (e.g., Kafka, Debezium) Core Technical Stack: Strong production-level coding skills in SQL (advanced optimization/stored procedures), Python, and PySpark / Apache Spark Analytical Databases & Cloud Warehouses: Experience working with high-performance databases and cloud-native systems (e.g., Snowflake, ClickHouse, PostgreSQL, MS SQL Server, or Azure Synapse) Methodologies: Master-level understanding of data modeling practices (OLAP, OLTP, Star/Snowflake schemas, Delta Lake/Lakehouse patterns, and Data staging processes) DevOps & CI/CD: Hands-on experience with version control (Git) and building automated deployment pipelines (CI/CD) for data products Communication & English: Proven ability to articulate complex technical ideas clearly to both business stakeholders and developers. Fluency in English (Upper-Intermediate level or higher) Nice to have Data Transformation & Quality Tools: Deep knowledge of dbt (data build tool) and schema validation practices Containerization: Experience using Docker or Kubernetes to package and deploy data applications Serverless Engineering: Experience building lightweight, serverless ingestion services (e.g., using AWS Lambda / Azure Functions and RESTful APIs)
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.
- Dice Id: 10330481
- Position Id: 3d69779a9741be75a0d33237d625b0a7
- Posted 7 days ago
Never miss an opportunity! Create an alert based on the job you applied for.
