Big Data Hadoop Engineer

Big Data Hadoop Engineer - MT 1780

Remote

Primary Skills: Scala, Java, Spark, Hive, Spark SQL, impala, Flask, Strong experience in Big Data, Cloudera Distribution 7.x,RDBMS development 4-5 years of programming experience in Python, and SQL is must. Strong experience building data pipelines using Hadoop components Sqoop, SOLR, MR, HBase. Strong experience with REST API development using Python frameworks (Django, Fast API etc.) and Java Springboot frameworks Project experience in AI/Machine Learning and NLP development.

Must Haves:

Strong experience in Big Data, Cloudera Distribution 7.x, RDBMS development

4-5 years of programming experience in Python, Java, Scala, and SQL is must.

Strong experience building data pipelines using Hadoop components Sqoop, Hive, SOLR, MR, Impala, Spark, Spark SQL, HBase.

Strong experience with REST API development using Python frameworks (Django, Flask, Fast API etc.) and Java Springboot frameworks

Project experience in AI/Machine Learning and NLP development.

TECHNICAL KNOWLEDGE AND SKILLS: Strong experience in Big Data, Cloudera Distribution 7.x, RDBMS
4-5 years of programming experience in Python, Java, Scala and SQL is must.
Strong experience building data pipelines using Hadoop components Sqoop, Hive, SOLR, MR, Impala, Spark, Spark SQL., HBase.
Strong experience with REST API development using Python frameworks (Django, Flask, Fast API etc.) and Java Springboot frameworks, Micro Services/Web service development experience using Spring framework.
Experience with Dask, Numpy, Pandas, Scikit-Learn
Proficient in Machine Learning Algorithms: Supervised Learning (Regression, Classification, SVM, Decision Trees etc.), Unsupervised Learning (Clustering) and Reinforcement Learning
Strong experience working in Real-Time analytics like Spark/Kafka/Storm
Experience with Gitlab, Jenkins, JIRA
Expertise in Unix/Linux environment in writing scripts and schedule/execute jobs.
Strong Experience with Data Science Notebooks like Jupyter, Zeppelin, RStudio. PyCharm etc.

PROFESSIONAL SKILLS:

Strong analytical skills with the ability to analyze information and identify and formulate solutions to problems.
Provide more in-depth analysis with a high-level view of goals and end deliverables.
Complete work within a reasonable time frame under the supervision of a manager or team lead.
Plan and manage all aspects of the support function.
Extensive knowledge of and proven experience with data processing systems, and methods of developing, testing and moving solutions to implementation.
Strong knowledge in project management practices and ability to document processes and procedures as needed.
Work collaboratively with other support team members and independently on assigned tasks and deliverables with minimum supervision
Communicate effectively with users at all levels, from data entry technicians up to senior management, verbally and in writing.
Self-motivated, working closely and actively communicating with team members to accomplish time critical tasks and deliverables
Ask questions and share information gained with other support team members, recording and documenting this knowledge, Elicit and gather user requirements and/or problem description information, and record this information accurately, Listen carefully and act upon user requirements
Convey and explain complex problems and solutions in an understandable language to both technical and non-technical persons
Present technical solutions to management and decision makers
Follow the lead of others on assigned projects as well as take the lead when deemed appropriate
Think creatively and critically, analyzing complex problems, weighing multiple solutions, and carefully selecting solutions appropriate to the business needs, project scope, and available resources
Take responsibility for the integrity of the solution.