Sr Data Engineer

Introduction

The Sr Data Engineer/AI Engineer will be responsible for developing ETL and data pipelines using AWS, Snowflake, Snowflake Cortex AI, and DBT. The ideal candidate should be experienced in building data pipelines using ELT methodology, self-directed, and comfortable supporting the data needs of multiple teams, systems, and products.

Responsibilities

• Building the infrastructure required for optimal extraction, transformation, and loading of data from various sources using Cloud Integration ETL Tools, Cloud Data Warehouse, SQL, and AWS technologies.
• Designing and developing ELT, ETL, event-driven data integration architecture solutions.
• Collaborating with Data Analysts, Data Architects, BI Architects, Data Scientists, and Data Product Owners to understand source data and determine data transformation and integration requirements.
• Troubleshooting and tuning complex SQL queries.
• Utilizing On-Prem and Cloud-based ETL platforms, Cloud Data Warehouse, AWS, GitHub, scripting languages, SQL, querying tools, data quality tools, and metadata management tools.
• Developing data validation processes to ensure data quality.
• Working individually and collaboratively as part of a team.

Requirements

Education and Experience

• Bachelor''s degree (or foreign equivalent) in Computer Science, Computer Engineering, or a related field.
• 8+ years of experience in Data Engineering, ETL, data warehouse/data mart development, and data lake development.
• Advanced working SQL knowledge and experience with relational databases.
• Experience with Cloud Data Warehouses like Snowflake, Google BigQuery, Amazon Redshift.
• Experience with AWS cloud services such as EC2, S3, Lambda, SQS, SNS, etc.
• Experience with Cloud Integration Tools like Matillion, Dell Boomi, Informatica Cloud, Talend, AWS Glue.
• Experience with GitHub and its integration with ETL tools for version control.
• Experience with Informatica PowerCenter, scripting languages, SQL, querying tools.
• Familiarity with modern data management tools and platforms including Spark, Hadoop/Hive, NoSQL, APIs, Streaming, and other analytic data platforms.
• Experience with object-oriented/object function scripting languages like Python, Java, Scala, etc.
• Experience with Agile/Scrum methodologies is valuable.

Other Knowledge, Skills, or Abilities Required

• Must be fluent in English.
• Ability to work with both business and IT stakeholders.
• Strong written and oral communication skills.
• Strong interpersonal skills.
• Ability to work independently and as part of a team.
• Highly motivated and a self-starter with strong problem-solving skills.
• Ability to multitask and meet aggressive deadlines efficiently.
• Extraordinary attention to detail and accuracy.