Big Data Engineer Tech lead /Senior- full time -Dallas/ chicago/jacksonville

Big data, Apache Hadoop, Pyspark, Scala, Hive, Kafka, Spark-Scala
Full Time
Depends on Experience

Job Description

We are looking for Sr. Big Data Engineer, for our client- Leading Global IT provider

Base Location: Dallas, TX

Designation: Sr. Big Data Engineer (PySpark or Spark-Scala)

Position Status: Immediate

Education: Minimum Bachelor’s degree in Computer Science, Engineering, Business Information Systems, or related field. Masters in Computing related to scalable and distributed computing is a major plus

 

Key Responsibilities:

  • Develop Big Data applications using PySpark or Scala-Spark on Hadoop, Hive and/or Kafka, HBase, MongoDB

  • Build Feature Engineering, Scoring / Machine Learning models

  • Deployment on Cloud platforms

 

Experience & Skillset

MUST-HAVE

  • Total IT / development experience of 7+ years

  • Experience in PySpark or Spark-Scala developing Big Data applications on Hadoop, Hive and/or Kafka, HBase, MongoDB

  • Technical Design and Onsite-Offshore coordination

  • Deep knowledge of Spark libraries on Python or Scala to develop and debug complex data engineering challenges

  • Experience in developing sustainable data driven solutions with current new generation data technologies to drive our business and technology strategies

  • Exposure in deploying onCloud platforms

 

  • At least 3 years of development experience on designing and developing Data Pipelines for Data Ingestion or Transformation using PySpark or Spark-Scala

  • At least 4 years of development experience in the following Big Data frameworks: File Format (Parquet, AVRO, ORC), Resource Management, Distributed Processing and RDBMS

  • At least 4 years of developing applications in Agile with Monitoring, Build Tools, Version Control, Shell Scripting, Unit Test, TDD, CI/CD, Change Management to support DevOps

  • Prior experience on ETL or SQL or other Data technologies

 

GOOD-TO-HAVE

  • Banking domain knowledge

  • Hands-on experience in SAS toolset / statistical modelling migrating to Machine Learning models

  • Digital Marketing Machine Learning models and use cases

  • ETL / Data Warehousing and Data Modelling experience prior to Big Data experience

  • Deep knowledge on AWS stack for big data and machine learning

 

Dice Id : 91128037
Position Id : 7142885
Originally Posted : 2 months ago
Have a Job? Post it

Similar Positions

Big Data Engineer, Tech Lead
  • Syeta Inc
  • Irving, TX, USA
Big Data Engineer
  • NexGen IOT Solutions, LLC
  • Dallas, TX, USA
Big Data Lead
  • Maveric NXT Inc
  • Irving, TX, USA
Pyspark developer
  • Syeta Inc
  • Irving, TX, USA
Bigdata Engineer
  • Anblicks
  • Dallas, TX, USA
Big Data Developer
  • Larsen & Toubro Infotech Limited
  • Irving, TX, USA
Sr Big Data Developer
  • Maveric NXT Inc
  • Irving, TX, USA
Bigdata + Spark Developer
  • Virtusa Corporation
  • Irving, TX, USA
Bigdata developer
  • Syeta Inc
  • Irving, TX, USA
Big Data Engineer
  • Macrosoft
  • Plano, TX, USA