Senior Big Data Engineer

company banner
GlobalLogic, Inc.
Big Data, hadoop, scala, pyspark, python, hbase, kafka, rest based api, machine learning, solr, lucene, MapReduce, hive, pig scripts, Spark / Scala & Java, Kafka
Contract W2, Contract Independent, Contract Corp-To-Corp, 12 Months
Depends on Experience
Work from home not available Travel not required

Job Description

Job Description

  • At least 10+ years of experience in building and managing complex products/solutions.
  • Very good problem solving/analytical skills.
  • Experience developing Restful web services in any Java framework.
  • Hands-on & Expert level knowledge and experience in Scala, Java, Distributed Computing, Apache Spark,
  • PySpark, Python, HBASE, Kafka, REST-based API, Machine Learning.
  • 10+ years of experience working in Linux/Unix environment.
  • Expert-level knowledge on HBASE, Solr, Lucene
  • Master s degree in Computer Science, Management Information Systems
  • Most importantly an absolute self-starter, a good team player, willingness to learn and implement Big data,
  • AI/ML technologies.
  • Strong experience with Data Warehousing concepts and standards.

Job Responsibilities

  • Build Back end analytical applications using Hadoop ecosystem.
  • Build Back end applications using Java, Spark/Scala, and Python, distributed computing
  • ETL modules for the AI/ML use cases, developing algorithms & testing use cases
  • Work on performance optimizations on HBASE and Solr
  • Work on large scale data transfer across different Hadoop clusters, implement new technology stacks on
  • Hadoop clusters
  • Work on Performance optimization of Spark Jobs, HIVE, PIG scripts and MapReduce jobs.
  • Debug complex production scenarios

Job Duties

  • Design and development of java, Scala and spark based software modules, performance improvement and
  • testing of these modules.
  • Scripting using python and shell scripts for ETL workflow. Design and development of back end big data
  • frameworks that are built on top of Spark with features like Spark as a service, workflow, and pipeline
  • management, handling batch and streaming jobs;
  • Build a comprehensive Big Data platform for data science and engineering that can run the batch process and
  • machine-learning algorithms reliably
  • Design and development of data ingestion services that can ingest 100s of TB of data every day
  • Build data ingestion from various source systems to Hadoop using Kafka, Sqoop, Spark Streaming, etc.
  • Coding for Big Data applications on clickstream, location and demographic data for behavior analysis using
  • Spark / Scala & Java
  • Optimize resource requirements including number of executors, cores per executors, memory for Spark
  • streaming and batch jobs
  • Knowledge and exposure the AI/ML technology stack
  • Have the ability to research and assess open source technologies and components to recommend and integrated into the design and implementation.

Posted By

Shashi Singh

1741 Technology Drive Level 4 San Jose, CA, 95110131



Company Information

GlobalLogic is a full-lifecycle product development services leader that combines chip-to-cloud software engineering expertise and vertical industry experience to help our customers design, build, and deliver their next generation products and digital experiences. We expertly integrate design, complex engineering, and agile delivery capabilities to deliver superior business outcomes for global brands. Headquartered in Silicon Valley, GlobalLogic operates design studios and engineering centers around the world, extending the benefits of our true global presence to customers in telecom, automotive, healthcare, technology, media and entertainment, manufacturing, and semiconductor industries.
Dice Id : RTL65472
Position Id : 6236560
Originally Posted : 2 months ago

Similar Positions at GlobalLogic, Inc.

Big Data Developer
  • Sunnyvale, CA
  • 2 weeks ago
Big Data Developer
  • Sunnyvale, CA
  • 2 weeks ago
Lead BigData/Scala Developer
  • Sunnyvale, CA
  • 2 weeks ago