Sr Data Engineer

aws, S3, python, etl, sql, redshift, leadership, Lambda, data engineer, airflow, data mining, glue, EMR, Athena, Kinesis, Event bridge, BI reporting
Full Time
Depends on Experience
Work from home available Travel not required

Job Description

***Ideal candidate is local and able to work a hybrid schedule.  Will consider full remote for non-local candidates.  No sponsorship available***

MUST HAVE:

  • Strong SQL experience
  • Python scripting experience
  • AWS cloud stack experience

Job Summary
As a DataOps engineer, you will be responsible for managing our data platform and enabling our mission of improving Customer Average Value. You will work with cross functional partners and third-party vendors to support our customer data assets by maintain the data pipelines to acquire, organize, and aggregate customer data from various sources for use in direct/email marketing, targeted media campaigns and analytics.

Responsibilities

  • Provide production support, including production defect resolution within reporting, ETL, 3rd party integrations, EDW, and master data for customer platform
  • Maintain BI reliability standards and documentation
  • Ensure release packages are implemented with complete information and support the maintenance as part of the System Development Life Cycle.
  • Responsible for all change control, system compliance, and system audit reviews related to Customer platform.
  • Monitor Data Observability and ensure data integrity of the entire ecosystem include data pipelines, lakehouse and redshift environment
  • Enhance, Deploy and maintain data processing pipelines using cloud technology such as AWS Airflow, Redshift, and Python.
  • Enhance, Deploy, and maintain serverless data pipelines using Event Bridge, Kinesis, AWS Lambda, S3, and Glue.
  • Deploy and maintain Power BI, YellowFin, and AWS Quick Sight reports and dashboards.
  • Ensure high operational efficiency and quality of your solutions to meet SLAs and support commitment to our customers (Data Science, Data Analytics teams). 

Skills & Requirements

  • Bachelor’s degree in analytics, statistics, engineering, math, economics, computer science, information technology, or related discipline.
  • 7+ years of experience working with Business Intelligence / Data Warehouse concepts and principles.
  • 5+ years of experience in supporting BI Production support processes including ETL data loads.
  • 5+ years of experience to support production operational processes based on ETL data loads tools, performance monitoring, security access, troubleshooting, operational issues/processes, and implementations.
  • 3-5 years of experience managing large-scale, mission-critical data pipelines and features using modern big data architectures.
  • 3-5 years of hands-on experience in Strong coding skills with Python/Pyspark/Spark and SQL.
  • 3+ years of hands-on experience in ETL tools such as Informatica.
  • Experience in ETL tools, AWS Tech Stack – Redshift, Glue, EMR, Redshift Spectrum, and Athena.
  • Ability to run queries to troubleshoot, create reports and provide analysis. Demonstrated ability to analyze large data sets to identify gaps and inconsistencies, provide data insights, and advance effective solutions.
  • Expert knowledge in writing complex SQL and ETL development with experience processing extremely large datasets.
  • Experience developing Infrastructure as a Code by creating CloudFormation templates using YAML and JSON.
  • Experience integrating data from multiple data sources and file types such as JSON, Parquet, and Avro formats. 
  • Experience supporting and working with cross-functional teams in a dynamic environment 
  • Strong quantitative and communication skills. 
  • Experience with CI/CD tools like Gitlab and Terraform.
  • Experience with data quality processes, data quality checks, validations, data quality metrics definition, and measurement.
  • Experience in data quality tools such as Informatica DQ or Talend DQ tools.

 

Dice Id : 90933400
Position Id : 6762902
Originally Posted : 2 years ago
Have a Job? Post it

Similar Positions

Senior Data Engineer
  • Ledgent Technology
  • Irvine, CA, USA
Sr ETL Data Engineer
  • Kaygen Inc
  • Costa Mesa, CA, USA
Senior Software Engineer - Data Services
  • E-Business International, Inc.
  • Irvine, CA, USA
Senior Data Engineer
  • Motion Recruitment
  • Irvine, CA, USA
Senior Data Engineer (Python/Spark)
  • Platinum Enterprise Solutions
  • Irvine, CA, USA
Senior Data Engineer (Fintech)
  • Motion Recruitment
  • Irvine, CA, USA
Senior Data Engineer (Social Media/Streaming)
  • Motion Recruitment
  • Irvine, CA, USA
Data Architect/Senior Data Engineer (Remote, PDT or CDT Hours)
  • Irvine Technology Corporation (ITC)
  • Santa Ana, CA, USA
Sr. Data Engineer (Python, Flink)
  • Motion Recruitment
  • Irvine, CA, USA