Azure Data Engineer

Job Title: Azure Data Engineer

Location: Boston, MA

Role Type: Contract

Role Overview:

We are seeking a highly skilled Azure Data Engineer with expertise in data migration from Big Data platforms to Azure Data Factory (ADF) to join our team. The ideal candidate will have a strong background in designing and executing complex data migration projects, with a focus on leveraging Azure cloud technologies. As an Azure Data Engineer, you will play a key role in migrating data from various Big Data platforms to Azure Data Factory while ensuring data integrity, security, and performance.

Key Responsibilities:

• Lead the planning, design, and execution of data migration projects from Big Data platforms to Azure Data Factory.
• Collaborate with stakeholders to understand data migration requirements and develop migration strategies and plans.
• Assess source data systems and identify data migration requirements, challenges, and opportunities.
• Design and implement scalable and efficient data migration pipelines using Azure Data Factory.
• Ensure data quality and integrity throughout the migration process by implementing data validation and reconciliation mechanisms.
• Optimize data migration processes for performance, reliability, and scalability.
• Work closely with cross-functional teams to integrate migrated data into Azure-based data solutions and applications.
• Provide technical guidance and support to junior team members and stakeholders throughout the migration process.

Requirements:

• Bachelor's degree in Computer Science, Engineering, or related field.
• Proven experience as a Data Engineer with a focus on data migration from Big Data platforms to Azure cloud environments.
• Strong proficiency in Azure Data Factory (ADF) and other Azure cloud services.
• In-depth understanding of data migration strategies, methodologies, and best practices.
• Experience with Big Data technologies such as Hadoop, Spark, or Kafka.
• Proficiency in SQL, Python, or other programming languages.
• Excellent analytical and problem-solving skills.
• Strong communication and interpersonal skills.
• Ability to work independently and collaboratively in a fast-paced environment.

Preferred Qualifications:

• Microsoft Azure certifications (e.g., Azure Data Engineer Associate, Azure Solutions Architect Expert).
• Experience with other Azure services such as Azure SQL Database, Azure Synapse Analytics, or Azure HDInsight.
• Knowledge of data governance, compliance, and security principles.
• Familiarity with DevOps practices and tools for CI/CD pipelines.
• Experience with Agile methodologies.