Overview
Hybrid
Depends on Experience
Contract - W2
Contract - Independent
Contract - 12 Month(s)
Skills
Python programming
Hadoop
Pyspark
Apache Airflow
HDFC
Job Details
Job Title: Data Engineer
Location: Irving, TX. Tampa, FL and Rutherford, NJ
Position Overview:
We are seeking a skilled Data Engineer to join our dynamic team. The ideal candidate will have a strong background in Python programming and Big Data technologies such as Hadoop and experience with PySpark on Cloudera platforms. Proficiency in Apache Airflow for workflow management is essential. Knowledge of Java is preferred but not mandatory.
Key Responsibilities:
- Design, develop, and maintain scalable data pipelines and infrastructure using Python, PySpark, and Cloudera technologies.
- Implement data ingestion, processing, and transformation processes to support analytical and operational requirements.
- Optimize and tune existing data pipelines for performance and efficiency.
- Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and deliver solutions.
- Troubleshoot and resolve data quality, performance, and reliability issues.
- Stay updated with the latest advancements in Big Data technologies and recommend innovative solutions to enhance our data engineering capabilities.
Requirements:
- Proven experience as a Data Engineer or in a similar role, with a minimum of 8-15 years of relevant work experience.
- Strong proficiency in Python programming and experience with Big Data frameworks such as Hadoop/HDFC.
- Hands-on experience with PySpark on Cloudera platforms.
- Hands-on experience with Apache Airflow for workflow management.
- Knowledge of Java is preferred but not mandatory.
- Solid understanding of distributed computing principles and data processing techniques.
- Excellent problem-solving skills and the ability to work independently and as part of a team.
- Strong communication skills and the ability to collaborate effectively with cross-functional teams.
- Bachelor's degree in Computer Science, Engineering, or a related field.
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.