Data Engineer - Pyspark

Overview

On Site
$100,000 - $110,000
Full Time

Skills

Big Data
Apache Spark
Pyspark

Job Details

Job Description

Required Qualifications:

  • All applicants authorized to work in the United States are encouraged to apply

  • At least 4 years of experience with Information Technology.

  • At least 2 years of experience in PySpark

  • Strong understanding of distributed computing principles and big data technologies

  • At least 2 years of experience working with Apache Spark and Spark SQL

  • Knowledge of data serialization formats such as Parquet, Avro, or ORC

  • Familiarity with data processing and transformation techniques

Preferred Qualifications:

  • Experience with data lakes, data warehouses, and ETL processes

  • Good understanding of Agile software development frameworks

  • Experience in Banking domain

  • Strong communication and Analytical skills

  • Ability to work in teams in a diverse, multi-stakeholder environment comprising of Business and Technology teams

  • Experience and desire to work in a global delivery environment

Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.