Sr Hadoop Developer

Software Guidance & Assistance, Inc., (SGA), is searching for a Sr Hadoop Developer for a contract assignment with one of our premier Healthcare Services clients in Jacksonville, FL.

Responsibilities :

• Lead the development and implementation of the next-generation Ingestion Framework (Aka Ingestion Framework Modernization) that writes data into Iceberg.
• Lead the design and development of the new Ingestion Framework using Apache Spark, ensuring it meets the requirements of scalability, performance, and reliability.
• Responsible for the design, development and operations of systems that store and manage large amounts of data.
• Most Sr Hadoop developers have a computer software background and have a degree in information systems, software engineering, computer science.
• Responsible for development, programming, coding of Information Technology solutions.
• Responsible for documenting detailed system specifications, participation in unit testing and maintenance of planned and unplanned internally developed applications, evaluation, and performance testing of purchased products.
• Responsible for including IT Controls to protect the confidentiality, integrity, as well as availability of the application and data processed or output by the application. Sr IT Developers are assigned to complex development projects.
• Write code for moderately to complex system designs. Write programs that span platforms. Code and/or create Application Programming Interfaces (APIs).
• Write code for enhancing existing programs or developing new programs.
• Review code developed by IT Developers.
• Provide input to and drive programming standards.
• Write detailed technical specifications for subsystems. Identify integration points.
• Report missing elements found in system and functional requirements and explain impacts on subsystem to team members.
• Consult with others Sr IT Developer, IT Developers, Business Analysts, Systems Analysts, Project Managers, and vendors.
• "Scope" time, resources, etc., required to complete programming projects. Seek review from other Sr IT Developers, Business Analysts, Systems Analysts or Project Managers on estimates.
• Perform unit testing and debugging. Set test conditions based upon code specifications. May need assistance from other IT Developers and team members to debug more complex errors.
• Supports transition of application throughout the Product Development life cycle. Document what has to be migrated. May require more coordination points for subsystems.
• Researches vendor products / alternatives. Conducts vendor product gap analysis / comparison.
• Accountable for including IT Controls and following standard corporate practices to protect the confidentiality, integrity, as well as availability of the application and data processed or output by the application.

Required Skills:

• Related Bachelor's degree or related work experience
• 5+ years related work experience, Professional experience with technical design and coding in the IT industry
• Specific Tools/Languages Required:
  • HADOOP
  • Spark
  • Scala
  • Python
• Technical Skills:
  • 5+ years of experience in Hadoop ecosystem (HDFS, Spark, Hive, etc.)
  • Strong expertise in Spark (Scala/Java) and experience with Spark-based data processing frameworks
  • Experience with Iceberg and data warehousing concepts
  • Proficiency in Java and/or Scala programming languages
  • Experience with JAR (Java Archive) development and deployment
  • Familiarity with CDC tools (Kafka, etc.) and Recon Frameworks
  • Proficiency in building Jars for parsing different type of files
• Ingestion Patterns Experience: Proven experience in designing and developing ingestion patterns, preferably with Spark as the base orchestration engine.
• Data Processing and Integration: Experience with data processing, integration, and reconciliation, including data quality and validation.
• Collaboration and Communication: Excellent collaboration and communication skills, with the ability to work with cross-functional teams and stakeholders.
• Problem-Solving and Troubleshooting: Strong problem-solving and troubleshooting skills, with the ability to analyze complex issues and develop effective solutions.
• Design and Develop Ingestion Patterns: Lead the design and development of a modernized ingestion patterns using Spark as the base orchestration engine, ensuring scalability, reliability, and performance.
• Create Jars for File Parsing: Develop and maintain Jars for parsing files of various formats (e.g., CSV, JSON, Avro, Parquet) and writing to Iceberg, ensuring data quality and integrity.
• Integrate with Recon Framework: Integrate the ingestion patterns with the Recon Framework through Jars, ensuring seamless data reconciliation and validation.
• Integrate with CDC Tool: Integrate the ingestion patterns with the CDC (Change Data Capture) Tool, enabling real-time data ingestion and processing.
• Collaborate with Cross-Functional Teams: Work closely with data engineers, data scientists, and other stakeholders to ensure the ingestion patterns and configurations meets business requirements and is aligned with overall data architecture.
• Troubleshoot and Optimize: Troubleshoot issues, optimize performance, and ensure the ingestion patterns of different types of files is running efficiently and effectively.
• Code Review and Quality Assurance: Perform code reviews, ensure adherence to coding standards, and maintain high-quality code.
• Experience and understanding with unit testing, release procedures, coding design and documentation protocol as well as change management procedures
• Proficiency using versioning tools
• Thorough knowledge of Information Technology fields and computer systems
• Demonstrated organizational, analytical and interpersonal skills
• Flexible team player
• Ability to manage tasks independently and take ownership of responsibilities
• Ability to learn from mistakes and apply constructive feedback to improve performance
• Must demonstrate initiative and effective independent decision-making skills
• Ability to communicate technical information clearly and articulately
• Ability to adapt to a rapidly changing environment
• In-depth understanding of the systems development life cycle
• Proficiency programming in more than one object-oriented programming language
• Proficiency using standard desktop applications such as MS Suite and flowcharting tools such as Visio
• Proficiency using debugging tools
• High critical thinking skills to evaluate alternatives and present solutions that are consistent with business objectives and strategy

SGA is a technology and resource solutions provider driven to stand out. We are a women-owned business. Our mission: to solve big IT problems with a more personal, boutique approach. Each year, we match consultants like you to more than 1,000 engagements. When we say let's work better together, we mean it. You'll join a diverse team built on these core values: customer service, employee development, and quality and integrity in everything we do. Be yourself, love what you do and find your passion at work. Please find us at .

SGA is an Equal Opportunity Employer and does not discriminate on the basis of Race, Color, Sex, Sexual Orientation, Gender Identity, Religion, National Origin, Disability, Veteran Status, Age, Marital Status, Pregnancy, Genetic Information, or Other Legally Protected Status. We are committed to providing access, equal opportunity, and reasonable accommodation for individuals with disabilities in employment, and our services, programs, and activities. Please visit our company to request an accommodation or assistance regarding our policy. #LI-SH1