W2-Columbus, OH (Onsite) :: AWS Data Engineer with Specialization in Databricks

AWS Data Engineer with Specialization in Databricks

6+Months

Columbus, OH (Onsite 5 days a week)

Role Summary:
We are seeking a Data Engineer with a strong focus on Databricks and AWS platforms, bringing at least 6 years of experience to our team.

Primary Duties:
Construct and Sustain Advanced Data Architectures: Employ PySpark and Spark to devise and maintain advanced data architectures within the Databricks ecosystem.
Architect and Implement the Medallion Design: Utilize the medallion model's raw, trusted, and refined zones to craft dependable data workflows.
Synthesize Varying Data Sources: Fuse data from multiple origins including Kafka streams, extraction endpoints, and third-party APIs.
Data Catalog Creation and Management: Architect and catalog datasets within a corporate data library, guaranteeing strong governance and easy access.
Access Management: Oversee stringent, role-based access configurations to facilitate data analytics, artificial intelligence, and machine learning projects.
Cooperative Team Effort: Engage actively with fellow team members to meet code quality standards and deliver thoroughly tested, high-caliber solutions.
Necessary Qualifications & Proficiencies:
Mastery in Databricks: Demonstrated expert knowledge
PySpark/Spark Expertise: Extensive practical engagement
AWS Expertise: Proficient knowledge in AWS services such as S3 and adeptness with Terraform for managing infrastructure as code
Data Architecture Acumen: In-depth understanding of medallion architecture and data warehousing methodologies
Data Pipeline Engineering: Established track record in the development, optimization, and management of large-scale data pipelines