Sr. Pyspark Developer with AWS

Overview

On Site
Accepts corp to corp applications
Contract - W2
Contract - 12

Skills

AWS
Python
Pyspark

Job Details

Hi

Hope you are doing well! Kindly do a glance at the below JD and share your updated resume.

Contract description

Skills:

  • Primary Skills Required
    • 4+ years of exp in PySpark
    • 6+ yrs of exp in SQL
    • Excellent hands-on experience with Python
  • Secondary Skills Required:
    • Data Engineering: Experience with data ingestion, ETL, and data warehousing.
    • Databases: Knowledge of relational databases
    • Cloud Platforms: Experience with AWS
    • Data Modeling: Ability to design and implement data models for analytical and reporting purposes.
    • Security: Understanding of data security principles and best practices.
    • Communication and Collaboration: Excellent communication and collaboration skills.

Responsibilities:

  • Development of Pyspark Jobs for data integration
  • Data Flow Design and Implementation:
    • Design ,document and implement data pipelines for various business needs, including data ingestion, transformation, and loading.
  • Data Pipeline Optimization:
    • Optimize data flows for performance, scalability, and efficiency, addressing bottlenecks and improving processing times.
  • Troubleshooting and Debugging:
    • Identify and resolve issues within data pipelines, including data quality issues, performance problems, and security vulnerabilities.
  • Data Quality and Security:
    • Ensure data quality and security throughout the data pipeline, implementing appropriate data validation, cleansing, and security measures.
  • Collaboration and Communication:
Collaborate with other data engineers, developers, and business stakeholders to understand requirements, design solutions, and ensure project success.
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.