Senior Big Data Developer

Big Data, Pig, Kafka, Hadoop
Full Time
Depends on Experience
Travel not required

Job Description

Job Title: Senior Big Data Developer

Location: 100% Remote

Duration: Full Time Only


Job Description:

Experience Level- 10+ years

We are looking for a Senior Big Data Engineer to help design, build, and enable new use cases leveraging our Big Data Platform.  In this role you will perform a broad range of data engineering activities including the design, creation, and support new bespoke data sets leveraging big data tools to help provide new and innovative insights to our business.    You will also be assisting in the creation of ingestion pipelines and refinement of data ingested into our environment.  Responsibilities include working with customers to define data assets that are needed and working to align with peers across data engineering (data ingestion and the cloud platform teams) to ensure needed data assets exist.  You will also be a key partner to our Data Science team to enable pipelines allowing model outputs to be consumed on a large scale and real-time basis.  This is an extremely hands on engineering role that values an engineering mind set and belief in first principles engineering.

At times you might also be expected to lead teams of engineers based on the product backlog(s).  Below is some activities you will perform:

  • Senior Engineer on the Big Data - Data Engineering team.  Design, develop, implement, and maintain code that enables data and analytics use cases across the enterprise
  • Design and build complex applications with an emphasis on performance, scalability, and high-reliability (DevOps Model).  You build it you own it!
  • Code, test, and document new or modified data systems to create robust and scalable applications for data analytics
  • Write concise code utilizing complex algorithms that can run at scale to build data driven features
  • Dive deep into performance, scalability, capacity, and reliability problems to resolve issues
  • Develop models within distributed computing frameworks
  • Take on research projects to improve data processing and implemented machine learning frameworks



  • Degree in Computer Science, Applied Mathematics, Engineering, or any other technology related field (or equivalent work experience)
    • 5+ years of experience working as a big data engineer (Required) must be able to articulate use cases supported and outcomes driven.  Expert knowledge of the following (expectation is to demonstrate these skills live during the interview):  Spark, Python, Scala, Hive, Pig, and MapReduce
    • Proven experience designing, building, and operating enterprise grade data streaming use cases leveraging one of the following:  Kafka, Spark Streaming, Storm, and/or Flink
  • Large scale data engineering and business intelligence delivery experience
    • Design of large-scale enterprise level big data platforms
    • 100’s or 1000’s of concurrent users
    • Experience working with and performing analysis using large data sets (10-1000 TB)
  • Proven and Demonstrated experience working with or on a mature, self-organized agile team
  • ***Preferred:  SAFe certification, Scrum Certification***
    • ***Preferred*** Previous exposure to cloud data platform (example):  AWS S3, Azure ADLS, Google Cloud Platform Cloud storage
    • ***Preferred*** Previous experience with Tibco Business works


Thanks and Regards,

Akash Kumar Gupta

Dice Id : 91101774
Position Id : 7501991
Originally Posted : 4 weeks ago
Have a Job? Post it