Lead SQL Developer - Azure Devops

  • Posted 12 hours ago | Updated 12 hours ago

Overview

Remote
$50 - $60
Accepts corp to corp applications
Contract - Independent
Contract - 12 Month(s)

Skills

Amazon Redshift
Amazon Web Services
Apache Oozie
Apache Spark
Cloud Computing
Collaboration
Continuous Delivery
Continuous Integration
Data Engineering
Data Processing
Data Quality
Data Warehouse
Databricks
DevOps
ELT
Extract
Transform
Load
Git
Good Clinical Practice
Google Cloud Platform
Microsoft Azure
Orchestration
Performance Tuning
PySpark
Python
Query Optimization
SQL

Job Details

Responsibilities:
Develop, optimize, and maintain ETL/ELT pipelines using PySpark and SQL.
Work with structured and unstructured data to build scalable data solutions.
Write efficient and scalable PySpark scripts for data transformation and processing.
Optimize SQL queries, stored procedures, and indexing strategies to enhance performance.
Design and implement data models, schemas, and partitioning strategies for large-scale datasets.
Collaborate with Data Scientists, Analysts, and other Engineers to integrate data workflows.
Ensure data quality, validation, and consistency in data pipelines.
Implement error handling, logging, and monitoring for data pipelines.
Work with cloud platforms (AWS, Azure, or Google Cloud Platform) for data processing and storage.
Optimize data pipelines for cost efficiency and performance.
Technical Skills Required:
Strong experience in Python for data engineering tasks.
Proficiency in PySpark for large-scale data processing.
Deep understanding of SQL (Joins, Window Functions, CTEs, Query Optimization).
Experience in ETL/ELT development using Spark and SQL.
Experience with cloud data services (AWS Glue, Databricks, Azure Synapse, Google Cloud Platform BigQuery).
Familiarity with orchestration tools (Airflow, Apache Oozie).
Experience with data warehousing (Snowflake, Redshift, BigQuery).
Understanding of performance tuning in PySpark and SQL.
Familiarity with version control (Git) and CI/CD pipelines.

Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.