Senior Python Developer - (Data Pipeline Migration Engineer)

Overview

Hybrid
$80 - $90
Accepts corp to corp applications
Contract - Independent
Contract - W2
Contract - 12 Month(s)

Skills

Python
Databricks
Jupyter
Lambda
Jupyter Notebooks/Workflows
PostgreSQL
Data Pipeline Migration

Job Details

Title: Senior Python Developer - (Data Pipeline Migration Engineer)

Location: Hybrid work in Princeton, NJ

Client: Major Financial Services Company

Duration: one year

Project Overview:

We are undertaking a project to migrate legacy data ingestion pipelines to a centralized shared data management system (BDMS). This initiative involves:

Loading datasets into BDMS

Developing new tools enabling data analysts to modify datasets

Validating and cleansing data before delivering it to AWS S3

Enriching data using tools like Databricks, AWS Lambda & Step Functions, and Jupyter Notebooks/Workflows, and loading into Delta tables

Your Responsibilities:

You will be expected to:

Collaborate with product owners and engineers to develop effective solutions

Work closely with data analysts/scientists to optimize data collection processes and identify opportunities to automate manual workflows

Develop Infrastructure as Code (IaC) and follow DevOps principles

Utilize AWS cloud services to implement microservices using serverless technologies

Required Qualifications:

Minimum 4 years of experience programming in Python or Java

Strong knowledge of database systems and SQL (including Oracle and PostgreSQL)

Bachelor s degree in Computer Science, Engineering, Mathematics, or a related field (or equivalent experience)

Hands-on experience with AWS services such as S3, Lambda, CloudWatch

Preferred Qualifications:

Experience with data migrations from legacy systems to the cloud

Knowledge of Apache Spark, Databricks, and Infrastructure as Code tools such as Terraform or CloudFormation

Top Skills:

Python

Databricks

AWS Lambda & Step Functions

Jupyter Notebooks/Workflows

Oracle and PostgreSQL

AWS

Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.