Spark Architect

Spark Architect
Location: Bloomfield, NJ
Duration: 6 months+

10+ years of experience in Development, design, Architect implementation, and analytics solutions
Hands on development Design, Setup and Develop applications using AWS-Python/Pyspark /Cloud

Very Strong Spark Architecture/Development knowledge
5-10 Years of Experience on Python and Pyspark
At least 8+ Experience on AWS (Glue, S3, lambda at the minimum)
Candidates need to be very skilled in Python core programming and Spark.
Experience in Devops building pipeline etc.
Ability to design and demonstrate system Architecture with different environments.
Hands on experience on Data Warehousing and SQL
Experience on Pandas
Strong expertise in SQL is required.
Solid experience with git software version control and good
understanding of code branching strategies and organization for code reuse

Experience with DevOps & CI/CD tools Jenkins, Cloudbees, Please Build, etc.
Proficiency working with large data stores and data sets
Health care knowledge is must.
Agile methodology (Jira Preferable)

Roles & Responsibilities

Must have capability to understand the current Architecture

Architect, Design, Coding and implementation of cloud-based Spark apps
support existing code thru enhancing, trouble shooting and maintaining per requirements
maintain consistent practices for source code management in github
comply with established CICD practices, and participate in the creation of new CICD methods
attend daily scrum calls, and actively participate in updating/creating user stories and tasks in Jira
Work closely with business analysts.
Lead innovation by exploring, investigating, recommending, benchmarking, and implementing data centric technologies for the platform.
Collaborating within a project team to solve complex problems.
Lead by example coaching and mentoring those of your peers and mentoring less experienced team members. Operate as part of the extended management team.