Sr.Big Data Cloud Engineer /Admin

Hadoop, HIVE, HBase, Spark, Kafka, Snowflake, Python, SQL, Java, Scala, Spark RDDs and Data Frames
Contract Corp-To-Corp, 12 Months
Depends on Experience
Work from home available Travel not required

Job Description

Able to translate functional and technical requirements into detail design. Responsible for analyzing large data sets to develop custom data pipelines to drive business solutions

  • Work with team leads for regular updates, requirement understanding and design discussions
  • Develops data pipeline for ingesting & cleansing data, applies data validation & transformation, stores data in various zones using Python, Hive & PySpark.
  • Hands on experience in design of Near Real time Data Processing solutions using various data streaming technologies in on-Prem and Azure cloud. Implements solution to stream data from various sources to Azure using Cloud native services
  • Hands on experience with Zaloni data management tool implementing data pipelines
  • Gathers the data from various third-party data in to cloud and on-Prem data platforms, optimizing it and joining with internal datasets to gather meaningful information. Stores data in various zones in right file format using appropriate compression techniques (ORC, Parquet and Avro)
  • Plan the closure of the solutions and see through the implementation of the solutions
  • Prepare and produce releases of data pipelines.
  • Good troubleshooting and application performance tuning experience.
  • Participates in day to day project and production delivery status meetings and provide technical support for faster resolution of issues.
  • Ability to co-ordinate work with geographically distributed team
  • Share knowledge, mentor and provide technical guidance to team members
  • Works with various scrum teams contribute to strengthening of DevOps and Agile methodologies to improve cycle times, consistency, and quality.
  • Excellent understanding / knowledge of Hadoop architecture and various components.
  • Solid understanding of Azure Data Platform and components

 

Dice Id : 10319875
Position Id : 6929310
Originally Posted : 4 weeks ago
Have a Job? Post it

Similar Positions

Lead Data Engineer (Cloud)
  • Judge Group, Inc.
  • Chicago, IL, USA
Google Cloud Data Engineer
  • Motion Recruitment
  • Chicago, IL, USA
Big Data Architect
  • VIVA USA INC
  • Chicago, IL, USA
Big Data Architect
  • Make Corporation
  • Chicago, IL, USA
Google Cloud Platform Data Engineer ( Architect )
  • FocuzMindz
  • Bolingbrook, IL, USA
Azure Data Scientist
  • Genesis NGN Inc.
  • Chicago, IL, USA
Manager- IT Data Engineering
  • Kraft Heinz
  • Chicago, IL, USA
Data Architect
  • Fusion IT Inc
  • Chicago, IL, USA
Principal Software Engineer
  • MRoads
  • Chicago, IL, USA
Senior Data Engineer with Scala, Spark, Big data
  • BizTech Solutions Inc
  • Chicago, IL, USA