Data Platform Engineer w/ Python

Data Platform Engineer

6 month Contract

rate: 90 to 120hr W2 only 

3 Regional Office Locations:  New York City, NT; or San Francisco, CA or Austin TX metropolitan area

Overview:
We are seeking a Data Platform Engineer who can build the data infrastructure that transforms large-scale healthcare data into reliable, production-ready datasets for AI/ML and product teams. This is a hands-on data engineering role with a strong emphasis on Apache Spark, Databricks, and distributed data processing. You will work across Spark, Databricks, Python and/or Scala, CI/CD, cloud infrastructure, and data quality to build and operate pipelines at scale. The ideal candidate is not simply focused on individual data transformations—they understand how data moves through a platform, how pipelines perform at scale, and how engineering decisions translate into reliable, high-quality data products.

Requirements:

1. Must have demonstrated hands-on experience building production data pipelines using Apache Spark and Databricks.
2. Must have strong Python and/or Scala development experience for data engineering and distributed processing.
3. Proven ability to design, build, deploy, and maintain large-scale data pipelines processing complex datasets.
4. Demonstrated experience building CI/CD pipelines and automated engineering workflows, including build, testing, deployment, and staging processes.
5. Demonstrated experience working with AWS, Azure, or Google Cloud Platform in production data engineering environments.
6. Proven ability to implement data transformation, validation, monitoring, and data quality controls for production pipelines.
7. Demonstrated breadth of contribution across the data platform rather than deep specialization in only one narrow technical area.
8. Experience troubleshooting and optimizing distributed data workloads for performance, reliability, scalability, and throughput.
9. Experience in healthcare data, revenue cycle management (RCM), claims, patient encounters, clinical data, or other regulated/high-accuracy data environments.
10. Strong builder and problem-solving mindset; comfortable moving quickly from requirements or data problems to production-ready pipeline solutions.