Sr Big Data Software Engineer - Spark

micro-services, rest apis, .net core, java, node., Big Data, kafka, data lakes, Big Data Analytics, Machine Learning, spark, scala, hadoop, python, r, C#, data structures, algorithms, design patterns, Azure Data Lake Store
Contract Independent, Contract Corp-To-Corp, 12 Months
Depends on Experience
Work from home not available Travel not required

Job Description

We are looking for a senior software engineer experienced in building high performing micro-services and REST APIs using Microsoft .Net Core, Java and Node.

Additionally, they must also have deep understanding of Big Data / Analytics technologies like Apache Spark, Kafka, Data Lakes, and running/consuming machine learning models.

We are developing innovative solutions that leverage the latest technologies in open source and the cloud and are building a team of top-notch software engineers to create our next generation Big Data Analytics and Machine Learning platform.

You ll be joining a team responsible for using cutting-edge technologies to modernize our data acquisition, data processing workflows, data analytics and automation methods.

You should be passionate about technology and love learning new technologies and implementing high quality, highly scalable global solutions. We want you to bring your excitement and technical expertise to deliver high quality work and continuously improve the ways we can better leverage Big Data Analytics and Machine Learning.

Most importantly you should bring new thinking, love technology, and be excited to help us take our vision to the next level.

This is just the beginning of Big Data Analytics and Machine Learning and we need you to help us define the future and get us there a huge opportunity awaits!

Role Specific Responsibilities

  • Responsible for design and development of high performing micro-service style server applications, REST APIs and distributed processing systems using Spark/Scala.
  • Analyzes business/ functional requirements and prepares development project schedule, tasks, and estimates
  • Identify and build automated solutions for the acquisition, processing and management of data that would otherwise require manual effort
  • Build distributed, scalable, and reliable data pipelines that ingest and process data at scale
  • Build the API layer to access Machine Learning Models
  • Ensures application design, development schedule, and implementation meet or exceed documented application scope/timeline, while adhering to established standards
  • Mentor junior development staff in the art of building software systems.
  • Bring new ideas in cloud, big data, and machine learning software development
  • Leverage industry knowledge and stay close to technology developments in the open-source and cloud communities
  • Collaborate with cross-functional teams
  • Practice and enforce Agile and Scrum development methodologies

Education: Bachelor s Degree or equivalent

Years of Experience: 7+


  • BS or higher degree in Computer Science/Engineering or related field
  • Must have experience with Big Data / Analytics technologies like Hadoop, Spark, Python, Scala, R,
  • Preferred to have exposure to Machine Learning Models using REST APIs.
  • Must have experience designing and building highly concurrent and high volume REST APIs using Microsoft .Net Core using C#
  • Preferred to experience in addressing concerns with big data management (Governance, Role Based Access Control using LDAP etc.)
  • Must have solid foundation in data structures, algorithms, design patterns
  • Highly desired knowledge of Azure components like Azure Data Lake Store, SQL Server, HDInsight, Web Jobs, Functions, Key vault, Data Factory is preferred
  • Experience with NoSQL databases like MongoDB is a big plus
  • Experience with caching technologies like Redis is a plus
  • Knowledge of React, ExpressJS, Node is a plus
  • Strong aptitude for problem solving, particularly to modify and enhance processes and workflows
  • Outstanding communicator with both business and technology audiences
  • Extensive experience working across teams and operating in an Agile Scrum culture
  • Experience with continuous delivery and using agile techniques
  • Firm sense of accountability and ownership
  • Passionate for making things better and driving action
  • Desire to understand our businesses and users
  • Experience with statistical platforms is a plus, such as R and SAS.
  • Experience with other visualization tools is a plus, such as Tableau
Dice Id : 10397987
Position Id : 6365819
Originally Posted : 1 month ago
Have a Job? Post it

Similar Positions

Big Data Lead
  • Pozent
  • Raritan, NJ
Hadoop developer
  • IT America
  • Edison, NJ
Hadoop Developer
  • Sumas Corporation
  • Princeton, NJ
Big Data Architect
  • SmartIT Frame
  • Bridgewater, NJ
Java Developer with Hadoop / Big Data experience
  • Konnect Partners
  • Princeton, NJ
Hadoop and Spark Developer with Python
  • Gangboard, LLC
  • Clinton, NJ
hadoop developer
  • Kforce Technology Staffing
  • Middletown, NJ
Hadoop Developer
  • Advantage Technical Resourcing
  • North Brunswick, NJ
Sr Big Data Platform Architect
  • Bank Of America
  • Pennington, NJ
Sr. Hadoop Developer
  • SM Logics Inc
  • Edison, NJ
Hadoop and Spark Developer with Python
  • Clinton, NJ
(STA13) Solution Design Expert - Big Data
  • Axtria
  • Berkeley Heights, NJ
Sr. Hadoop Developer
  • Uniqsoft
  • Edison, NJ