DataOps/Cloud Data Engineer (32599)

Overview

Contract - W2

Skills

Software Design
Testing
Scheduling
Production Support
Reporting
Accountability
Data Management
Optimization
Workflow
Scalability
Data Quality
Data Integrity
Collaboration
Data Science
Machine Learning (ML)
Data Integration
Privacy
Data Migration
OLTP
SaaS
PaaS
IaaS
Python
Scala
Transact-SQL
Snow Flake Schema
OLAP
Agile
Continuous Delivery
Provisioning
Data Flow
Unstructured Data
DaaS
Database
Data Warehouse
Virtual Machines
Open Source
Performance Monitoring
Cloud Computing
Product Development
Data Analysis
Microsoft Exchange
High Availability
Data Governance
Data Security
Regulatory Compliance
Meta-data Management
Modeling
Dimensional Modeling
Normalization
Data Modeling
Data Engineering
SQL Azure
Analytics
Data Lake
Storage
Extract
Transform
Load
Orchestration
ADF
Microsoft Azure
DevOps
GitHub

Job Details

We are seeking a Data Engineering Specialist to conduct work on the full life cycle of data engineering including analysis, solution design, data pipeline engineering, testing, deployment, scheduling, and production support. Reporting to the Director, Data Strategy and Analytics, you will be Accountable for leading and providing data engineering expertise to analytics and modeling through the creation and maintenance of data pipelines, the design and development of data solutions, and performing data management and optimization to support Supply Ontario's current and future business needs.

Key Responsibilities

  • Design and develop scalable, efficient data pipelines using Azure Data Factory and Datapicks Workflows.
  • Optimize pipeline performance for scalability, throughput, and reliability with minimal latency.
  • Implement robust data quality, validation, and cleansing processes to ensure data integrity.
  • Collaborate with stakeholders to gather business and technical requirements for data solutions.
  • Troubleshoot and resolve data ingestion, transformation, and orchestration issues.
  • Support analytics, data science, and machine learning workloads through seamless data integration.
  • Support data governance initiatives, ensuring compliance with data security, privacy, and quality standards.
  • Contribute to data migration projects including OLTP/OLAP workloads and very large datasets (VLDs) to cloud platforms (SaaS, PaaS, IaaS).

Required Skills

  • +5 years of experience in data engineering, Strong proficiency in Python and familiarity with Azure Services is required.
  • Expertise with Azure Data Services: Azure SQL Database, Azure Data Lake, Azure Storage, Azure Datapicks.
  • Experience with data pipeline development, orchestration, deployment, and automation using ADF, Datapicks, Azure DevOps/GitHub Actions.
  • Proficiency in Python, Scala, and T-SQL.
  • Solid understanding of data warehousing and ETL concepts including star/snowflake schemas, fact/dimension modeling, and OLAP.
  • Familiarity with DataOps principles, Agile methodologies, and continuous delivery.
  • Proficient in data provisioning automation, data flow control, and platform integration.
  • Knowledge of both structured, semi-structured, and unstructured data ingestion, exchange, and transformation.
  • Experience with cloud-native data services such as DaaS (Data-as-a-Service), DBaaS (Database-as-a-Service), and DWaaS (Data Warehouse-as-a-Service), and infrastructure elements like Key Vault, VMs, and disks.
  • Experience with commercial and open-source data platforms, storage technologies (cloud and on-prem), and the movement of data across environments.
  • Experience in performance monitoring and tuning for cloud-based data solutions.
  • Experience supporting digital product development, data analysis, data security, and secure data exchange across platforms.
  • Proven experience designing enterprise-scale data architectures with high availability and security.
  • Understanding of data governance, data security, compliance, and metadata management.
  • Proficient in entity-relationship (ER) modeling and dimensional modeling.
  • Strong knowledge of normalization/denormalization techniques to support analytics-ready datasets.

Must Haves:

  • 5+ years of experience in data modelling and data engineering is required
  • 5+ Expertise with Azure Data Services: Azure SQL Database, Azure Synapse Analytics, Azure Data Lake, Azure Storage, Azure Datapicks
  • 5+ Experience with data pipeline development, orchestration, deployment, and automation using ADF, Datapicks, Azure DevOps/GitHub Actions
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.

About Myticas LLC