Big Data Engineer, Tech Lead

Bigdata engineer, Tech Lead, PySpark or Scala-Spark on Hadoop, Hive and/or Kafka, HBase, MongoDB, Data engineering, Machine learning, Digital marketing, Python
Full Time
$100,000 - $120,000

Job Description

Base Location: Dallas
Designation: Big Data Engineer, Tech Lead (PySpark or Spark-Scala)
Position Status: Immediate


Education: Minimum Bachelor’s degree in Computer Science, Engineering, Business Information
Systems, or related field. Masters in Computing related to scalable and distributed computing is
a major plus
Key Responsibilities:
Develop Big Data applications using PySpark or Scala-Spark on Hadoop, Hive and/or Kafka,
HBase, MongoDB
Build Feature Engineering, Scoring / Machine Learning models
Deployment on Cloud platforms
Experience & Skillset
MUST-HAVE
Total IT / development experience of 8+ years
Experience in PySpark or Spark-Scala developing Big Data applications on Hadoop, Hive and/or
Kafka, HBase, MongoDB
Technical leadership and Onsite-Offshore coordination
Deep knowledge of Spark libraries on Python or Scala to develop and debug complex data
engineering challenges
Experience in developing sustainable data driven solutions with current new generation data
technologies to drive our business and technology strategies
Exposure in deploying on Cloud platforms
At least 4 years of development experience on designing and developing Data Pipelines for Data
Ingestion or Transformation using PySpark or Spark-Scala
At least 5 years of development experience in the following Big Data frameworks: File Format
(Parquet, AVRO, ORC), Resource Management, Distributed Processing and RDBMS
At least 4 years of developing applications in Agile with Monitoring, Build Tools, Version Control,
Shell Scripting, Unit Test, TDD, CI/CD, Change Management to support DevOps
Prior experience on ETL or SQL or other Data technologies

GOOD-TO-HAVE
Banking domain knowledge
Hands-on experience in SAS toolset / statistical modelling migrating to Machine Learning
models
Digital Marketing Machine Learning models and use cases
ETL / Data Warehousing and Data Modelling experience prior to Big Data experience
Deep knowledge on AWS stack for big data and machine learning

Dice Id : 91121733
Position Id : 7115168
Originally Posted : 3 months ago
Have a Job? Post it

Similar Positions

Pyspark developer
  • Syeta Inc
  • Irving, TX, USA
Big Data Engineer
  • NexGen IOT Solutions, LLC
  • Dallas, TX, USA
Big Data Lead
  • Maveric NXT Inc
  • Irving, TX, USA
Big Data Developer
  • Larsen & Toubro Infotech Limited
  • Irving, TX, USA
Bigdata + Spark Developer
  • Virtusa Corporation
  • Irving, TX, USA
Sr Big Data Developer
  • Maveric NXT Inc
  • Irving, TX, USA
Bigdata developer
  • Syeta Inc
  • Irving, TX, USA
Pyspark Lead
  • Maveric NXT Inc
  • Irving, TX, USA
Bigdata Engineer-FTE
  • Impetus
  • Dallas, TX, USA