BigData Engineer

Agile, Amazon Web Services, Analytical skill, Analytics, Apache Avro, Apache Hadoop, Apache Hive, Apache Kafka, Apache Parquet, Apache Spark, Apache Sqoop, Big data, Business analysis, Data flow, Data science, Communication skills, Python, SQL, Microsoft Windows Azure
Contract W2, Contract Independent, 12 Months
Depends on Experience
Travel not required

Job Description

We are looking for enthusiastic Big Data Engineer having minimum of 3 years’ experience in Big Data implementations and have 7 years of overall experience in IT and related Industries to join our team of analytics experts. Should be responsible for expanding and optimizing our data and data pipeline architecture, as well as optimizing data flow and collection for cross functional teams. Should also be experienced in data pipeline builder and data wrangler who enjoy optimizing data systems and building them from the ground up. The Data Engineer will support our software developers, database architects, Data analysts and data scientists on data initiatives and will ensure optimal data delivery architecture is consistent throughout ongoing projects.

Location: - Wilmington, DE 

Job Qualifications and Requirements:

  • Minimum 3 yrs development experience is required on Hadoop using HIVE, Python, Apache Spark, Sqoop, Shell scripting and SQL including working experience on building end-to-end data pipelines.
  • At least 1 years of verifiable experience with Real Time data stream platforms (Kafka, Spark Streaming)
  • At least 1 year of verifiable experience working with cloud data capabilities (Amazon AWS, Microsoft Azure)
  • Required proficiency in writing and optimizing complex hive queries and user defined functions to perform transformations in Spark and Hive
  • Working experience is required on both on-premise and cloud Hadoop platforms like Azure HDInsight, AWS EMR
  • Strong understanding of Avro, parquet, orc formats along with its advantages/disadvantages is required.
  • Experience on Job Scheduler like Airflow is plus
  • Work with data scientists, business analysts and product owners for requirements and business rules
  • Good analytical skills and problem solving skills
  • Experience of working in Agile Environment
  • Broad understanding and experience in real-time analytics
  • Finding ways to enhance data reliability, efficiency and quality over large data sets to address business issues
  • Excellent communication and interpersonal skills

Job Description:

Create and maintain optimal data pipeline architecture, Assemble large, complex data sets that meet functional / non-functional business requirements. Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.

Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL and AWS ‘big data’ technologies.

Build analytics tools that utilize the data pipeline to provide actionable insights into customer acquisition, operational efficiency and other key business performance metrics.

Work with stakeholders including the Executive, Product, Data and Design teams to assist with data-related technical issues and support their data infrastructure needs.

Keep our data separated and secure across national boundaries through multiple data centers and AWS regions. Create data tools for analytics and data scientist team members that assist them in building and optimizing our product into an innovative industry leader.

Work with data and analytics experts to strive for greater functionality in our data systems.

About the Company:

DataKare Solutions LLC is a data services and analytical company that helps customers to adapt to new approaches and technologies to manage, govern, secure and apply advance analytics on enterprise data to improve operations. We leverage latest cutting-edge tools and cloud computing to fulfil software initiatives, enabling our customers to transform their industries.

Thanks

Venkat

Technical Recruiting Specialist

Email:- venkatvn@datakaresolutions.com  Website: -www.datakaresolutions.com

 

Posted By

Alpharetta, GA, 30004

Dice Id : 91112812
Position Id : DKSBD0328
Originally Posted : 3 months ago
Have a Job? Post it

Similar Positions

Data Architect - AWS, Hadoop & Big Data
  • Computer Enterprises, Inc.
  • Philadelphia, PA
BigData Architect
  • Resource Informatics Group
  • Philadelphia, PA
Data Architect
  • Elite IT Professionals
  • Wilmington, DE
Big Data Engineer
  • Nedumaan Global Solutions
  • Wilmington, DE
Enterprise Data Architect - Cloud
  • The Robinson Group
  • Downingtown, PA
Data Engineer (Remote)
  • Wallero
  • Philadelphia, PA
Big Data Engineer - Java/J2EE and Spark
  • Photon Infotech
  • Wilmington, DE
Data Governance Analyst
  • TeamSoft
  • Wayne, PA
Big Data Developer
  • Scepter Technologies, Inc
  • Wilmington, DE
Spark/Big Data Engineer
  • Pyramid Consulting, Inc.
  • Philadelphia, PA
Big Data Developer
  • The Judge Group, Inc.
  • Wilmington, DE
Big Data Engineer
  • Renotek Group
  • Wilmington, DE
Big Data Developer with Java
  • STM Consulting, Inc.
  • Wilmington, DE