Data Engineer With Databricks Experience, Onsite in Chicago, IL

Introduction:

We are seeking a highly skilled Data Engineer with experience in Databricks to join our client in NYC. The successful candidate will be responsible for designing and implementing data pipelines, processing large datasets, and optimizing data workflows.

Responsibilities:

• Design and build scalable data pipelines using Databricks
• Work closely with data scientists and analysts to understand data requirements
• Optimize data workflows for performance and efficiency
• Implement data quality and data governance best practices
• Collaborate with cross-functional teams to integrate data from various sources
• Troubleshoot and resolve data processing issues
• Stay up-to-date with the latest trends and technologies in data engineering

Requirements:

Required Skills:

• 8+ years of experience in data engineering
• Proficiency in Databricks
• Strong knowledge of SQL and data modeling
• Experience with big data technologies such as Hadoop, Spark, or Kafka
• Excellent problem-solving skills
• Ability to work independently and in a team environment

Preferred Skills:

• Experience with cloud platforms such as AWS or Azure
• Knowledge of machine learning concepts
• Experience with data visualization tools