10604 - Big Data Engineer ( Ecosystem)

Apache Hadoop, Apache Hive, Apache Kafka, Linux, Microsoft Power BI, Python, Unix, Tableau, NoSQL, Big data
Full Time
Depends on Experience
Work from home available

Job Description

Job Description

  • Hands on experience on Big Data ecosystem
  • Mastering/Leading in the development of applications/tools using Sqoop, Spark, Python, Scala packages
  • Good understanding of BI tools (Tableau/Power BI, MicroStrategy) for Big Data
  • Knowledge on Big suites Hive, Oozie, Kafka, good experience in Hadoop (Native) Big Data processing
  • Experience of writing Hive queries
  • Familiarity with development best practices such as code reviews and unit testing.
  • Experience in designing the automation framework using Python, Perl and Shell scripting.
  • Leading, guiding and helping the team members with the challenges faced during the project development process and problem solving.
  • Have flexibility and ability to learn and use new technologies and to work in team environment as wells as independently to get things done
  • Excellent communications skill
  • Experience in working with client and handle the pressure
  • Diagnosing software issues and engineering workarounds
  • Experience in working with users, user roles using Ranger
  • Good understanding on security and administration

Technology –

  • Big data (Native) Ecosystem
  • OS: CentOS/Linux/Unix
  • Tool/Programs: Sqoop, Spark, Impala, Python, Scala
  • Database: Hive, HBase, NoSQL

Qualification –

Bachelor’s Degree in Computer Science or equivalent


Primary Responsibilities

  • Act as a Big data subject/solution matter expert and liaison to Big Data users and IT
  • Serve as an escalation point for production and/or platform issues and acts as lead liaison with Technical Support to resolve outstanding issues within the Big Data environment
  • Execute actions necessary to achieve and maintain optimum Big Data platform performance, reliability and availability Identify bottlenecks in production systems and provide solutions
  • Provide Big Data Platform capacity planning, metrics, and performance tuning recommendations
  • Use Big Data Management tools (Ranger)
  • Creating Users/Groups/Roles
  • Creating and using Access Control Templates
  • Creating, deployment and scheduling jobs
  • Defining connections to data sources, libraries, metadata objects, and folder structures


  • 5+ years of experience working with and maintaining IT Application support.
  • 3+ years of experience with Big Data Engineer
  • Experience in production data loads and data/storage management for multiple data sources.
  • Knowledge of change management procedures and version control is required
  • Experience/Knowledge of application security, performance and tuning
  • Passion about technology and continuous learning and mastering computer-related technologies
  • Strong communication skills and ability to work as a team

Good to have

  • Strong knowledge of Big Data open-source communities.
  • Advanced analytics tools like R, SAS, Python
  • Experience in using ML, RPA
Dice Id : 10119475
Position Id : 10604
Originally Posted : 4 months ago
Have a Job? Post it

Similar Positions

  • Vertogic
  • Costa Mesa, CA, USA
Big Data Engineer
  • Everest Technologies
  • Burbank, CA, USA
Senior Big Data DevOps Engineer
  • Pyramid Technology Solutions, Inc.
Big Data Engineer (Remote)
  • Xntric Solutions
Big Data Engineer
  • IT Minds LLC
Big Data Engineer Lead
  • WinWire Technologies