Job Title: Data Engineer - Spark, Hadoop, Ozone CH, Flink
Location: Berkley Heights, NJ (5 Days Onsite)
Duration: long term
Type: Contract W2 only
Experience: 7 - 10 Yrs
Job Description
We are seeking a highly skilled Big Data Engineer with strong experience in Apache Spark, Hadoop ecosystem, and Apache Ozone. The ideal candidate will design, develop, and optimize large-scale data processing systems, ensuring high performance, scalability, and reliability for enterprise-level applications.
Key Responsibilities:
- Design and implement distributed data processing solutions using Apache Spark, Hadoop, Flink
- Develop and maintain Spark applications for data transformation, aggregation, and ETL processes using Scala, Java, or Python
- Utilize Apache Ozone for storing large-scale datasets, ensuring efficient data access and management in a distributed environment
- Manage and optimize HDFS and Apache Ozone, Kafka for scalable and fault-tolerant storage.
- Develop ETL pipelines for batch and real-time data ingestion and transformation.
- Implement and ensure data validation, data security, integrity, and compliance across big data platforms.
- Monitor and troubleshoot performance issues in large-scale clusters.
- Collaborate with data scientists, analysts, and application teams to deliver high-quality data solutions.
- Automate workflows and improve operational efficiency using scripting and orchestration tools.
Required Skills & Qualifications:
- Strong expertise in Apache Spark (Core, SQL, Streaming).
- Hands-on experience with Hadoop ecosystem (HDFS, YARN, MapReduce).
- Proficiency in Apache Ozone for object storage and integration with Hadoop.
- Solid programming skills in Java, Scala, or Python.
- Experience with Hive, HBase, and Kafka is a plus.
- Knowledge of cluster management and resource optimization.
- Familiarity with Linux/Unix environments and shell scripting.
- Understanding of data security, governance, and compliance standards.
- Experience with cloud-based big data platforms
- Exposure to containerization (Docker, Kubernetes) for big data workloads.
- Knowledge of CI/CD pipelines for data engineering projects.
Qualifications:
- Bachelor’s degree in computer science, Software Engineering, or a related field.
- Proficiency in business process modeling and documentation tools.
- Product implementation experience is preferred
About Next Gen Software Solutions:
Next Gen Software Solutions is a trusted provider of IT Staffing and consulting services dedicated to empowering businesses with cutting-edge technology solutions and exceptional talent. We specialize in delivering tailored IT consulting services, innovative software solutions, and connecting businesses with highly skilled IT professionals. Founded and led by a dedicated U.S. Army solider, Next Gen Software Solutions is deeply rooted in the core values of integrity, discipline, commitment, and experience-principles that guide every aspect of our operations.
Equal Employment Opportunity Statement:
Next Gen Software Solutions is an Equal Opportunity Employer. We are committed to fostering an inclusive and diverse workplace where all employees and applicants are treated respect and dignity. We do not discriminate based on race, colour, religion, sex (including pregnancy, sexual orientation, or gender identity), national origin, age, genetic information, veteran status, or any other legally protected characteristic under applicable federal, state, or local laws.
About Next Gen Software Solutions LLC:
Next Gen Software Solutions is a trusted provider of IT staffing and consulting services dedicated to empowering businesses with cutting-edge technology solutions and exceptional talent. We specialize in delivering tailored IT consulting services, innovative software solutions, and connecting businesses with highly skilled IT professionals. Founded and led by a dedicated U.S. Army soldier, Next Gen Software Solutions is deeply rooted in the core values of integrity, discipline, commitment, and excellence—principles that guide every aspect of our operations.
Never miss an opportunity! Create an alert based on the job you applied for.