Overview
Hybrid
Depends on Experience
Accepts corp to corp applications
Contract - Independent
Contract - W2
Contract - 2 Year(s)
Able to Provide Sponsorship
Skills
Continuous Integration
Apache Spark
Artificial Intelligence
Cloud Computing
Continuous Delivery
Amazon Redshift
Amazon Web Services
Apache Hadoop
Apache Kafka
Extract
Transform
Load
IT Architecture
IT Management
Legacy Systems
Meta-data Management
Data Governance
Data Lake
Databricks
DevOps
Documentation
ELT
Git
ADF
Microsoft Azure
Microsoft SSIS
Migration
Prompt Engineering
Python
SQL
Warehouse
Workflow
Job Details
Key Responsibilities
Design scalable data lake and data architectures using Databricks and cloud-native services.
Develop metadata-driven, parameterized ingestion frameworks and multi-layer data architectures.
Optimize data workloads and performance.
Define data governance frameworks for CHP.
Design and develop robust data pipelines.
Architect AI systems, including RAG workflows and prompt engineering.
Lead cloud migration initiatives from legacy systems to modern data platforms.
Provide architectural guidance, best practices, and technical leadership across teams.
Build documentation, reusable modules, and standardized patterns.
Required Skills and Experience
Strong expertise in cloud platforms, primarily Azure or AWS.
Hands-on experience with Databricks.
Deep proficiency in Python and SQL.
Expertise in building ETL/ELT pipelines and ADF workflows.
Experience architecting data lakes and implementing data governance frameworks.
Hands-on experience with CI/CD, DevOps, and Git-based development.
Ability to translate business requirements into technical architecture.
Technical Expertise
Programming: Python, SQL, R
Big Data: Hadoop, Spark, Kafka, Hive
Cloud Platforms: Azure (ADF, Databricks, Azure OpenAI), AWS
Data Warehousing: Redshift, SQL Server
ETL/ELT Tools: SSIS
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.