Cloud Data Engineer (AWS / Databricks)

Overview

On Site
Full Time
Part Time
Accepts corp to corp applications
Contract - W2
Contract - Independent

Skills

Management
Analytics
Provisioning
Collaboration
Performance Tuning
Scalability
Optimization
Testing
DevOps
Data Engineering
Cloud Computing
Amazon Web Services
Big Data
Electronic Health Record (EHR)
Amazon S3
Scala
Python
Java
Streaming
Terraform
Continuous Integration
Continuous Delivery
Docker
Kubernetes
Agile
Machine Learning (ML)
Workflow
Databricks
Orchestration

Job Details

Key Responsibilities:

  • Design, build, and optimize scalable data pipelines and frameworks on AWS leveraging EMR, EKS, S3, and related services.

  • Develop, deploy, and manage Databricks (DBX) workflows for big data processing, analytics, and machine learning.

  • Implement infrastructure as code (IaC) solutions using Terraform for automated provisioning and environment consistency.

  • Write efficient, reusable, and optimized code in Scala, Python, and Java for data transformation, streaming, and integration.

  • Collaborate with architects, data scientists, and business stakeholders to translate requirements into technical solutions.

  • Ensure performance tuning, scalability, and cost optimization across cloud-based data platforms.

  • Enforce best practices in security, CI/CD, testing, and DevOps integration for cloud-native applications.


Required Skills & Experience:

  • 7 10 years of hands-on experience in data engineering and cloud frameworks.

  • Proven expertise with AWS big data services (EMR, EKS, S3, Lambda, Glue).

  • Strong development skills in Scala, Python, and Java.

  • Deep knowledge of Databricks (DBX) for batch and streaming workloads.

  • Solid experience with Terraform for infrastructure automation.

  • Familiarity with CI/CD, containerization (Docker, Kubernetes), and Agile methodologies.


Nice-to-Have Skills:

  • Exposure to machine learning workflows on Databricks.

  • Knowledge of Airflow or other orchestration tools.

  • Experience with data lakehouse architectures.


Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.

About Purple Drive Technologies LLC