Senior Data Engineer

Overview

On Site
$DOE
Full Time
Part Time
Accepts corp to corp applications
Contract - W2
Contract - Independent
Contract - 12+ Month(s)

Skills

(PySpark + Databricks + Oracle/PostgreSQL Migration)

Job Details

Position: Senior Data Engineer (PySpark + Databricks + Oracle/PostgreSQL Migration)

Location: 5800 Windward Parkway, Alpharetta, GA 3000 (Only Locals) 3 Days onsite

Job type: Contract

Exp Level: Min 9+ Years

  • Expert-level proficiency in PySpark and Databricks.
  • Primary Skill Focus: PySpark, Databricks (Hands-on coding test expected)

Skills Required:

1. RDBMS Expertise:

  • Strong hands-on experience with Oracle and PostgreSQL databases.
  • Deep understanding of database architecture, design, indexing, and performance tuning.
  • Expert-level SQL skills: queries, stored procedures, functions, triggers, and views.

2. Database Migration

  • Proven experience in end-to-end database migration projects (preferred: Oracle PostgreSQL).
  • Strong ability to perform data mapping, transformation, validation, and reconciliation.
  • Experience using migration tools, scripts, and automation frameworks.

3. Data Engineering & Analysis

  • Advanced proficiency in Databricks for large-scale data processing.
  • Expert in PySpark and Python for data transformation and analytics.
  • Ability to build, enhance, and optimize complex ETL/ELT data pipelines.

4. Job Scheduling & Automation

  • Experience creating and maintaining Databricks jobs for scheduled reporting.
  • Familiarity with workflow orchestration tools (Airflow, ADF, Step Functions, etc.).

5. Performance Optimization

  • Strong background in performance tuning for Oracle and PostgreSQL.
  • Experience with index strategies, query optimization, execution plan analysis, and caching.

6. Cloud Platforms

  • Good understanding of AWS Cloud, including RDS, S3, EMR, Lambda, Glue, or similar services.

Responsibilities:

1. Database Design & Development:

  • Design and develop robust database solutions that meet data storage and retrieval requirements.
  • Create scripts and procedures to automate routine database tasks.

2. Migration & Implementation:

Lead the migration process from Oracle to PostgreSQL, ensuring data integrity and minimal downtime.

Develop comprehensive migration plans and execute them proficiently.

3. Support & Maintenance:

Monitor database performance and implement necessary improvements.

4. Reporting & Analytics:

Develop and maintain Databricks jobs for generating business reports and analytics.

Provide insights from data analysis to support decision-making.

Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.