Big Data Developer

Position Title : Big Data Developer
Location : Jersey City, NJ (Hybrid)
Experience : 8+ Years
Employee Type : Full Time with Benefits

Job Description
We are seeking a highly skilled and experienced Senior Big Data Engineer to join our dynamic team. The ideal candidate will have a strong background in software development, data engineering, and large-scale distributed systems. This role requires a deep understanding of big data technologies and the ability to lead and implement enterprise-scale solutions.

Primary Skills:
Java/Scala, ETL, Spark, Hadoop, Hive, Impala, Sqoop, HBase, Confluent Kafka, Oracle, Linux, Git, Jenkins, CI/CD

Responsibilities:

• Design, develop, and maintain large-scale distributed data processing systems and applications.
• Lead the development and implementation of Big Data solutions at an enterprise scale.
• Collaborate with cross-functional teams to define, design, and ship new features.
• Perform performance analysis, troubleshooting, and resolution of issues related to Cloudera/Hadoop logs.
• Work with Cloudera on open issues, implement cluster configuration changes as needed.
• Ensure data management and governance, including Hadoop security.
• Monitor and schedule batch processing jobs using AutoSys.
• Optimize and maintain databases such as SQL, Hive, Elasticsearch, HBase, etc.
• Utilize programming languages such as Java, J2EE, and Scala for development tasks.
• Implement and manage ETL processes using tools like Sqoop.
• Maintain and enhance CI/CD pipelines using Git and Jenkins.

Skills and Qualifications:

• Bachelor s Degree or Master s in Computer Science, Engineering, Software Engineering, or a relevant field.
• 8-10 years of software development experience in building large-scale distributed data processing systems/applications.
• At least 4 years of experience in developing/leading Big Data solutions at an enterprise scale with at least one end-to-end implementation.
• Strong experience in programming languages Java/J2EE/Scala.
• Proficiency in Spark/Hadoop/HDFS Architecture, YARN, Confluent Kafka, HBase, Hive, Impala, and NoSQL databases.
• Experience with batch processing and AutoSys job scheduling and monitoring.
• Strong experience with databases such as SQL, Hive, Elasticsearch, HBase, etc.
• Knowledge of Hadoop security, data management, and governance.
• Familiarity with Cloudera/Hadoop logs for performance analysis and troubleshooting.
• Experience with Linux operating systems.
  
  Disclaimer:
  Capgemini is an Equal Opportunity Employer encouraging diversity in the workplace. All qualified applicants will receive consideration for employment without regard to race, national origin, gender identity/expression, age, religion, disability, sexual orientation, genetics, veteran status, marital status or any other characteristic protected by law.

  This is a general description of the Duties, Responsibilities and Qualifications required for this position. Physical, mental, sensory or environmental demands may be referenced in an attempt to communicate the manner in which this position traditionally is performed. Whenever necessary to provide individuals with disabilities an equal employment opportunity, Capgemini will consider reasonable accommodations that might involve varying job requirements and/or changing the way this job is performed, provided that such accommodations do not pose an undue hardship.

  Capgemini is committed to providing reasonable accommodations during our recruitment process. If you need assistance or accommodation, please reach out to your recruiting contact.

  Click the following link for more information on your rights as an Applicant ;/p>