Big Data Engineer__ Irving, TX- Hybrid 3 days from office

Overview

Compensation information provided in the description
Full Time

Skills

ETL
hadoop
SAS
Hive
Pyspark
banking domain

Job Details

Hi,

Hope you are doing well!

Please have a look below JD * if you are comfortable, please share me update profile.

Role- : Big Data Engineer

Location- Irving, TX- Hybrid 3 days from office

Job Type: Only Fulltime

Expected salary : $100k

Primary Skills: Hadoop, Hive & Spark Framework

Responsibilities:

  • Experience with big data processing and distributed computing systems like Spark.
  • Implement ETL pipelines and data transformation processes.
  • Ensure data quality and integrity in all data processing workflows.
  • Troubleshoot and resolve issues related to PySpark applications and workflows.
  • Understand source, dependencies and data flow from converted PySpark code.
  • Strong programming skills in Python and SQL.
  • Experience with big data technologies like Hadoop, Hive, and Kafka.
  • Understanding of data warehousing concepts and relational databases like SQL.
  • Demonstrate and document code lineage.
  • Integrate PySpark code with frameworks such as Ingestion Framework, DataLens, etc.,
  • Ensure compliance with data security, privacy regulations, and organizational standards.
  • Knowledge of CI/CD pipelines and DevOps practices.
  • Strong problem-solving and analytical skills.
  • Excellent communication and leadership abilities.

Qualifications:

  • 4+ years of experience in big data development, Hadoop, Hive & Spark framework.
  • Good to have experience in SAS.
  • Strong Python, PySpark Development and SQL knowledge.
  • Certification in big data or cloud technologies is preferred.
  • Excellent communication, collaboration, and problem-solving skills.

Ravi Kumar

Desk: |Cell-

Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.