Overview
On Site
$45 - $50
Contract - W2
Contract - 1 month(s)
No Travel Required
Skills
Big Data Developer
Job Details
Job Description:
Pay Range: $45hr - $50hr
Responsibilities:
- XXgn, develop, and maintain scalable ETL pipelines using PySpark and Python to process large-scale datasets across distributed environments.
- Implement complex data transformation logic and optimize Spark jobs for performance and cost-efficiency.
- CollaboXX with data scientists, analysts, and business stakeholders to understand data requirements and deliver robust solutions.
- IntegXX data from multiple sources (structured, semi-structured, unstructured) into unified data models.
- Ensure data quality, consistency, and governance through validation, monitoring, and error handling mechanisms.
- Participate in system design discussions and contribute to architectural decisions for big data platforms.
- Document technical solutions and maintain version control using Git and CI/CD tools like Jenkins or Azure DevOps.
- Strong programming skills in Python and PySpark.
- Hands-on experience with Apache Spark, Hadoop, Hive, and HDFS.
- Proficiency in SQL and working with both relational and NoSQL databases.
- Strong debugging, performance tuning, and problem-solving skills.
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.