Senior Data Engineer (PySpark Databricks Oracle/PostgreSQL Migration)

Senior Data Engineer (PySpark + Databricks + Oracle/PostgreSQL Migration)

Location: Alpharetta, GA - Need Locals

LOOKING F , Citizen and EAD

Exp Level: Min 12+ Years

Expert-level proficiency in PySpark and Databricks.

Primary Skill Focus: PySpark, Databricks (Hands-on coding test expected)

Skills Required:

1.RDBMS Expertise:

Strong hands-on experience with Oracle and PostgreSQL databases.

Deep understanding of database architecture, design, indexing, and performance tuning.

Expert-level SQL skills: queries, stored procedures, functions, triggers, and views.

2. Database Migration

Strong ability to perform data mapping, transformation, validation, and reconciliation.

Experience using migration tools, scripts, and automation frameworks.

3. Data Engineering & Analysis

Advanced proficiency in Databricks for large-scale data processing.

Expert in PySpark and Python for data transformation and analytics.

Ability to build, enhance, and optimize complex ETL/ELT data pipelines.

4. Job Scheduling & Automation

Experience creating and maintaining Databricks jobs for scheduled reporting.

Familiarity with workflow orchestration tools (Airflow, ADF, Step Functions, etc.).

5. Performance Optimization

Strong background in performance tuning for Oracle and PostgreSQL.

Experience with index strategies, query optimization, execution plan analysis, and caching.

6. Cloud Platforms

Good understanding of AWS Cloud, including RDS, S3, EMR, Lambda, Glue, or similar services.

Responsibilities:

Design and develop robust database solutions that meet data storage and retrieval requirements.

Create scripts Database Design & Development:

and procedures to automate routine database tasks.

Migration & Implementation:

Lead the migration process from Oracle to PostgreSQL, ensuring data integrity and minimal downtime.

Develop comprehensive migration plans and execute them proficiently.

Support & Maintenance:

Monitor database performance and implement necessary improvements.

Reporting & Analytics:

Develop and maintain Databricks jobs for generating business reports and analytics.

Provide insights from data analysis to support decision-making.