Senior data Engineer

  • New York, NY
  • Posted 23 hours ago | Updated 23 hours ago

Overview

On Site
Depends on Experience
Full Time

Skills

ETL
Pyspark
Python Frameworks

Job Details

Sr Data Engineer (8+ yrs)

Design, build, and optimize scalable ETL/ELT pipelines using PySpark, Databricks, or Spark on EMR.

Develop reusable components and frameworks for data ingestion and transformation using Python.

Write, tune, and optimize PL/SQL queries, stored procedures, and data models for high-performance analytics.

Proficiency in PySpark (RDDs, DataFrames, Spark SQL).

Experience with Python frameworks (e.g., pandas, Airflow, FastAPI, or Flask for orchestration and APIs).

Understanding of object-oriented programming and software engineering best practices.

Expertise in PL/SQL for Oracle or similar relational databases.

Experience with ETL tools (Informatica, Talend, or custom-built frameworks).

Exposure to containerization (Docker, Kubernetes) and CI/CD pipelines (Jenkins, GitHub Actions).

Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.