Senior Big Data Engineer (12+ Years only)

REQUIRED Skill Sets:

• 8+ years of experience with Big Data, Hadoop on Data Warehousing or Data Integration projects.
• Analysis, Design, development, support and Enhancements of ETL/ELT in data warehouse environment with Cloudera Bigdata Technologies (with a minimum of 7 or more years experience in Hadoop, MapReduce, Sqoop, PySpark, Spark, HDFS, Hive, Impala, StreamSets, Kudu, Oozie, Hue, Kafka, Yarn, Python, Flume, Zookeeper, Sentry, Cloudera Navigator) along with Oracle SQL/PL-SQL, Unix commands and shell scripting;
• Strong development experience (minimum of 7 or more years) in creating Sqoop scripts, PySpark programs, HDFS commands, HDFS file formats (Parquet, Avro, ORC etc.), StreamSets pipeline creation, jobs scheduling, hive/impala queries, Unix commands, scripting and shell scripting etc.
• Writing Hadoop/Hive/Impala scripts (minimum of 7 or more years experience) for gathering stats on table post data loads.
• Strong SQL experience (Oracle and Hadoop (Hive/Impala etc.)).
• Writing complex SQL queries and performed tuning based on the Hadoop/Hive/Impala explain plan results.
• Proven ability to write high quality code.
• Experience building datasets and familiarity with PHI and PIIdata.
• Expertise implementing complex ETL/ELT logic.
• Develop and enforce strong reconciliation process.
• Accountable for ETL/ELT design documentation.
• Good knowledge of Big Data, Hadoop, Hive, Impala database, data security and dimensional model design.
• Basic knowledge of UNIX/LINUX shell scripting.
• Utilize ETL/ELT standards and practices towards establishing and following centralized metadata repository.
• Good experience in working with Visio, Excel, PowerPoint, Word, etc.
• Effective communication, presentation and organizational skills.
• Familiar with Project Management methodologies like Waterfall and Agile
• Ability to establish priorities and follow through on projects, paying close attention to detail with minimal supervision.
• Required Education: BS/BA degree or combination of education & experience

DESIRED Skill Sets:

• Demonstrate effective leadership, analytical and problem-solving skills
• Required excellent written and oral communication skills with technical and business teams.
• Ability to work independently, as well as part of a team.
• Stay abreast of current technologies in area of IT assigned.
• Establish facts and draw valid conclusions.
• Recognize patterns and opportunities for improvement throughout the entire organization.
• Ability to discern critical from minor problems and innovate new solutions.