Lead Data Engineer in Irving, TX

Amtex Systems Inc is an information technology and talent solutions company offering talent and BI consulting to the companies in US for over 25 years.

Our solutions are designed to fill resource gaps, by providing the right candidates who deliver value to the organization. Our propensity to nurture and build strong relationships with our clients helps us better understand their business demands and gives us the ability to provide services that are on time and rise above the rest.

Title: Lead Data Engineer 

Irving, TX

Duration: 6+ Months C2H

Need Locals for Onsite Interview

We are seeking a highly experienced Senior Data Engineer to play a pivotal role in modernizing, scaling, and continuously improving our Azure-based data platform. You will design, build, and maintain robust, scalable data pipelines and contribute significantly to data architecture decisions that power mission-critical internal applications, business operations, analytics, and strategic insights. This hands-on senior role demands deep technical expertise in Azure data services, strong collaboration skills, DataOps practices, and a commitment to delivering reliable, high-quality data that drives measurable business value.

Key Responsibilities

• Design, develop, and maintain scalable, secure ETL/ELT pipelines in Azure Data Factory (v2) to ingest, transform, and load live, near-real-time, and batch data from diverse sources (legacy RDBMS such as MySQL/MariaDB, flat files/CSV, APIs, and event-driven feeds) into Azure SQL Database (OLTP) and Azure SQL Data Warehouse / Synapse environments.
• Serve as a strong technical contributor and influencer in the full architecture, design, implementation, and population of a new high-performance OLTP transactional database in Azure SQL Database to support internal applications — ensuring high concurrency, ACID compliance, data integrity, and excellent performance under load.
• Apply expert-level dimensional modeling techniques (star and snowflake schemas, fact/dimension tables, slowly changing dimensions (SCD Type 1/2/4/6), surrogate keys, degenerate/junk dimensions, bridge tables), partitioning, and indexing strategies optimized for both transactional (OLTP) and analytical workloads.
• Write, tune, and maintain production-grade T-SQL code — including complex stored procedures, functions, views, triggers, analytical queries — with deep focus on query optimization, execution plan analysis, index tuning, and cost-effective scaling.
• Develop and utilize Python scripts for custom transformations, data validation, API integrations, custom Azure Data Factory activities, automation, and supporting logic.
• Implement and govern mature CI/CD pipelines (Azure DevOps, Git) for automated, repeatable, version-controlled, and auditable deployment of pipelines, T-SQL objects, Python code, database schema changes (using DACPACs, Flyway, Liquibase or similar), and infrastructure-as-code components.
• Establish comprehensive data quality frameworks — including automated profiling, validation checks, reconciliation, anomaly detection, monitoring, observability, lineage, and metadata management — embedded within pipelines and workflows.
• Design and implement API-based (REST) and event-driven integrations for real-time / near-real-time data flows into Azure SQL platforms.
• Apply DataOps principles to promote automation, collaboration, monitoring, versioning, continuous testing, and incremental improvement across the data engineering lifecycle.
• Manage data environment lifecycle (dev, test, prod), including provisioning, configuration consistency, promotion workflows, and drift prevention.
• Partner closely with Data Analysts, Report Writers (Power BI), Subject Matter Experts, application developers, business stakeholders, and Executives to gather, refine, document, and translate requirements into robust data models, pipeline designs, and delivery plans.
• Proactively monitor pipeline health, data freshness, query performance, SLAs, reliability, and costs; diagnose and resolve issues efficiently.
• Contribute meaningfully to long-term data platform architecture, identify and integrate new data sources, ensure compatibility/cleanliness/compliance, and actively participate in Agile ceremonies (sprint planning, stand-ups, refinement, demos, retrospectives).
• Provide technical mentorship, guidance, and support on data engineering initiatives as needed.

Required Qualifications

• 7+ years of hands-on data engineering experience, with at least 4–5 years heavily focused on the Microsoft Azure data stack.
• Expert-level mastery of T-SQL in Azure SQL Database and Azure SQL Data Warehouse / Synapse contexts — advanced query authoring, performance tuning, stored procedure development, and execution plan optimization.
• Strong proficiency in Python for data engineering tasks: transformations, scripting, validation, API integrations, and Azure service interactions.
• Deep expertise in dimensional modeling, data warehousing architecture, and database design across OLTP and analytical workloads.
• Advanced, production experience designing and operating complex Azure Data Factory pipelines for reliable ETL/ELT at scale, including migrations from legacy RDBMS (MySQL, MariaDB) and file-based sources.
• Proven success implementing CI/CD for data assets (pipelines, databases, code) and applying DataOps best practices in Agile environments.
• Strong troubleshooting, problem-solving, communication, documentation, and stakeholder management skills.
• Comfortable collaborating with both technical and non-technical stakeholders to elicit, clarify, and influence requirements and design decisions.

Preferred Qualifications

• Experience with Azure Synapse Analytics, Azure Data Lake Gen2, Azure Databricks, Event Grid / Event Hubs, or related event-driven services.
• Familiarity with Infrastructure as Code (Bicep, Terraform) in data platform contexts.
• Exposure to modern data quality, testing, and governance tools (Great Expectations, Soda, dbt tests, Azure Purview).
• Track record of migrating or modernizing legacy database systems to Azure-native architectures.
• Relevant Microsoft certification (e.g., DP-203: Data Engineering on Microsoft Azure).
• Bonus: Experience in the automotive repair, insurance, or related industries.

Thanks and Regards!

Lavkesh Dwivedi

 

 
Amtex System Inc.

28 Liberty Street, 6^th Floor | New York, NY - 10005