Data Engineer Databricks Integration

Overview

Hybrid
Depends on Experience
Accepts corp to corp applications
Contract - Independent
Contract - W2
Contract - 12 Month(s)

Skills

Data Engineer
Databricks
ETL/ELT
SQL

Job Details

Title: Data Engineer Databricks Integration

Onshore Location: local to Chicago

Type: Hybrid (twice/thrice a week)

Job Type: Contract

About the Role

We are seeking a highly skilled Data Engineer to design, develop, and maintain data pipelines that extract data from Oracle Symphony via APIs, process and store it in the Databricks Lakehouse platform, and then integrate it into Oracle EPM (Enterprise Performance Management). This role requires deep expertise in data integration, ETL/ELT, APIs, and Databricks. The candidate will work closely with business stakeholders, architects, and analysts to ensure seamless data flow, transformation, and availability for financial planning, reporting, and analytics.

Key Responsibilities

  • Design and implement end-to-end pipelines from Oracle Symphony (API extraction) into Databricks Lakehouse.
  • Develop efficient ETL/ELT processes in Databricks (PySpark, Delta Lake) to transform, cleanse, and enrich data.
  • Build and maintain data flows from Databricks into Oracle EPM to support reporting, forecasting, and planning.
  • Ensure data quality, consistency, and governance across Symphony, Databricks, and EPM.
  • Optimize pipeline performance, scalability, and reliability.
  • Collaborate with data architects, finance teams, and Oracle specialists to meet business needs.
  • Troubleshoot pipeline issues and provide production support for data integration processes.
  • Document architecture, pipeline logic, and integration workflows.
  • Stay current on Databricks, Oracle, and API integration best practices.

Required Skills & Qualifications

  • Bachelor s or Master s degree in Computer Science, Data Engineering, or related field.
  • 5+ years of experience in data engineering, ETL/ELT, and data pipeline development.
  • Hands-on experience with Databricks (PySpark, Delta Lake, MLflow).
  • Strong experience with APIs (REST, SOAP, JSON, XML) for data extraction and integration.
  • Proficiency in SQL, Python, and Spark for data processing.
  • Experience with cloud platforms (Azure, AWS, or Google Cloud Platform) for hosting Databricks and related services.
  • Knowledge of data modeling, data governance, and performance tuning.
  • Strong problem-solving skills and ability to work in cross-functional teams.

Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.