Big Data Engineer

Hadoop, Big Data, Big Data Engineer, PySpark, Scala, Spark
Full Time
Depends on Experience
Work from home available Travel not required

Job Description

Role: Big Data Engineer

Location: Dallas, TX (Initially Remote)

Duration: Fulltime with the client

Visa: Any Visa is fine (including H1 transfer)

 

Education: Minimum Bachelor’s degree in Computer Science, Engineering, Business Information Systems, or related field. Masters in Computing related to scalable and distributed computing is a major plus

 

Key Responsibilities:

  • Develop Big Data applications using PySpark or Scala-Spark on Hadoop, Hive and/or Kafka, HBase, MongoDB.
  • Build Feature Engineering, Scoring / Machine Learning models.
  • Deployment on Cloud platforms.

 

Experience & Skillset (MUST HAVE):

  • Total IT / development experience of 8+ years.
  • Experience in PySpark or Spark-Scala developing Big Data applications on Hadoop, Hive and/or Kafka, HBase, MongoDB.
  • Technical leadership and Onsite-Offshore coordination.
  • Deep knowledge of Spark libraries on Python or Scala to develop and debug complex data engineering challenges.
  • Experience in developing sustainable data driven solutions with current new generation data technologies to drive our business and technology strategies.
  • Exposure in deploying on Cloud platforms.
  • At least 4 years of development experience on designing and developing Data Pipelines for Data Ingestion or Transformation using PySpark or Spark-Scala.
  • At least 5 years of development experience in the following Big Data frameworks: File Format (Parquet, AVRO, ORC), Resource Management, Distributed Processing and RDBMS.
  • At least 4 years of developing applications in Agile with Monitoring, Build Tools, Version Control, Shell Scripting, Unit Test, TDD, CI/CD, Change Management to support DevOps.
  • Prior experience on ETL or SQL or other Data technologies.

 

GOOD-TO-HAVE:

  • Banking domain knowledge.
  • Hands-on experience in SAS toolset / statistical modelling migrating to Machine Learning models.
  • Digital Marketing Machine Learning models and use cases.
  • ETL / Data Warehousing and Data Modelling experience prior to Big Data experience.
  • Deep knowledge on AWS stack for big data and machine learning.

 

If you’re interested then please apply for this role or share your updated resume on lokesh(at)nexgeniots(dot)com

Dice Id : RTX1dcee3
Position Id : 7167727
Originally Posted : 2 months ago
Have a Job? Post it

Similar Positions

Big Data Engineer, Tech Lead
  • Syeta Inc
  • Irving, TX, USA
Big Data Lead
  • Maveric NXT Inc
  • Irving, TX, USA
Bigdata + Spark Developer
  • Virtusa Corporation
  • Irving, TX, USA
Sr Big Data Developer
  • Maveric NXT Inc
  • Irving, TX, USA
Bigdata developer
  • Syeta Inc
  • Irving, TX, USA
Pyspark developer
  • Syeta Inc
  • Irving, TX, USA
Big Data Developer
  • Larsen & Toubro Infotech Limited
  • Irving, TX, USA
Pyspark Lead
  • Maveric NXT Inc
  • Irving, TX, USA