Big Data Engineer (Technical Specialist 4 (TS4)

Role: Big Data Engineer (Technical Specialist 4 (TS4)

Location: Columbus, Ohio

Experience: 12+ years

Duration: Long term

REQUIRED Skill Sets:
Proficiency in Data Warehousing, Data migration, and Snowflake is essential for this role.
Strong Experience in the implementation, execution, and maintenance of Data Integration technology solutions.
Minimum (4-6) years of hands-on experience with Cloud databases.
Minimum (2-3) years of hands-on data migration experience from the Big data environment to Snowflake environment.
Minimum (2-3) years of hands-on experience with the Snowflake platform along with Snowpipe and Snowpark.
Strong experience with Snow SQL, PL/SQL, and expertise in writing snowflake procedures using SQL/python/Java.
Experience with optimizing Snowflake database performance and real-time monitoring.
Strong database architecture, critical thinking, and problem-solving abilities.
Experience with the AWS platform Services.
Snowflake Certification is Highly desirable.
Snowpark with Python programming languages is preferred to be used to build data pipelines.
8+ years of experience with Big Data, Hadoop on Data Warehousing or Data Integration projects.
Analysis, Design, development, support and Enhancements of ETL/ELT in data warehouse environment with Cloudera Bigdata Technologies (with a minimum of 8-9 years' experience in Hadoop, MapReduce, Sqoop, PySpark, Spark, HDFS, Hive, Impala, StreamSets, Kudu, Oozie, Hue, Kafka, Yarn, Python, Flume, Zookeeper, Sentry, Cloudera Navigator) along with Oracle SQL/PL-SQL, Unix commands and shell scripting;
Strong development experience (minimum of 8-9 years) in creating Sqoop scripts, PySpark programs, HDFS commands, HDFS file formats (Parquet, Avro, ORC etc.), StreamSets pipeline creation, jobs scheduling, hive/impala queries, Unix commands, scripting and shell scripting etc.
Writing Hadoop/Hive/Impala scripts (minimum of 8-9 years' experience) for gathering stats on table post data loads.
Strong SQL experience (Oracle and Hadoop (Hive/Impala etc.)).
Writing complex SQL queries and performed tuning based on the Hadoop/Hive/Impala explain plan results.
Experience building data sets and familiarity with PHI and PII data.
Expertise implementing complex ETL/ELT logic.
Accountable for ETL/ELT design documentation.
Basic knowledge of UNIX/LINUX shell scripting.
Utilize ETL/ELT standards and practices towards establishing and following centralized metadata repository.
Good experience in working with Visio, Excel, PowerPoint, Word, etc.
Effective communication, presentation and organizational skills.
Familiar with Project Management methodologies like Waterfall and Agile.
Ability to establish priorities and follow through on projects, paying close attention to detail with minimal supervision.
Required Education: BS/BA degree or combination of education & experience.

DESIRED Skill Sets:
In addition, to overall Snowflake experience, a candidate should have experience in the development work in both the Snowpipe and Snowpark.
Experience with Data Migration from Big data environment to Snowflake environment.
Strong understanding of Snowflake capabilities like Snowpipe, STREAMS, TASKS etc.
Knowledge of security (SAML, SCIM, OAuth, OpenID, Kerberos, Policies, entitlements etc.).
Has experience with System DRP for Snowflake systems
Demonstrate effective leadership, analytical and problem-solving skills.
Required excellent written and oral communication skills with technical and business teams.
Ability to work independently, as well as part of a team.
Stay abreast of current technologies in area of IT assigned.
Establish facts and draw valid conclusions.
Recognize patterns and opportunities for improvement throughout the entire organization.
Ability to discern critical from minor problems and innovate new solutions.