Azure Data Engineer

Job Title: Azure Data Engineer

Location: Cary, NC
Work Mode: 100% Onsite
Duration: 6 Months (Contract)
Experience: 10+ Years

Job Description

We are seeking a highly experienced Azure Data Engineer to design, build, and optimize large-scale data pipelines and analytical platforms. The ideal candidate will have strong hands-on experience with Azure cloud services, big data technologies, and performance-optimized ETL frameworks for both batch and real-time analytics use cases.

Key Responsibilities

• Design, develop, and maintain scalable ETL/ELT pipelines for ingesting and processing large volumes of structured and unstructured data.
• Build reusable data engineering frameworks to improve development efficiency and reduce project costs.
• Implement high-performance data ingestion and curation processes using big data tools.
• Optimize performance at the development stage through efficient coding, tuning, and best practices.
• Develop and manage data warehouses and analytical data stores for enterprise reporting and analytics.
• Support real-time and batch processing use cases across on-prem and cloud environments.
• Collaborate with global teams to deliver data solutions and recommend performance improvements.
• Ensure data quality, validation, and anomaly detection using strong SQL and analytical techniques.
• Continuously learn and adopt new cloud and big data technologies.

Technical Skills Required

Cloud & Big Data

• Strong hands-on experience with Microsoft Azure, including:
• Azure Databricks
• Azure Data Factory (ADF)
• Azure Data Lake (ADLS Gen2)
• Azure Synapse Analytics
• Experience with Big Data ecosystems, including:
• Apache Spark (Scala / Python / Java)
• Hive (Partitioning & Bucketing MUST)
• HDFS, Kafka, Sqoop, Impala
• Kerberos security
• Cloudera CDP 7.x

Programming & Data Processing

• Strong proficiency in:
• Spark (Scala & PySpark) Performance tuning is a MUST
• Python, Scala, SQL
• Expertise in writing high-performance, reliable, and maintainable ETL code.
• Strong experience handling unstructured and semi-structured datasets.

Data Modeling & Warehousing

• Strong understanding of relational data modeling.
• Hands-on experience designing and implementing Star and Snowflake schemas.
• Experience building analytical data warehouses for enterprise analytics.

Preferred Skills

• Experience with:
• Big Query, Big SQL
• Cloud Composer
• Dataproc
• Snowflake
• Knowledge of MySQL and other relational databases.
• Exposure to multi-cloud data platforms (Azure + Google Cloud Platform is a plus).