Overview
On Site
Accepts corp to corp applications
Contract - W2
Contract - 14 day((s))
Skills
Python
Apache Spark
Azure Data Factory
Azure Data Lake
ETL/ELT
Azure SQL
azure synapse
Azure Databricks
Azure App Service
Job Details
Role: Azure Databricks Engineer with Python & Spark (Senior / Lead)
Location: Weehawken, NJ (Onsite from Day 1)
Job Type: W2 Contract
Note: 2 positions are available - Senior & Lead
Experience Level: Senior OR Lead
Must have skills (Mandatory):
- Apache Spark
- Python
- Azure Data Factory
- Azure SQL Microsoft
- Azure Synapse Analytics
- ETL/ELT Solutions
Nice to have skills:
- Apache Kafka
- CI/CD
Experience & Qualifications:
- Expert level understanding of Azure Data Factory, Azure Synapse, Azure SQL, Azure Data Lake, and Azure App Service
- Working knowledge of Python
- Expert level skills writing and optimizing complex SQL
- Experience with complex data modeling, ETL design, and using large databases in a business environment
- Experience with building data pipelines and applications to stream and process datasets at low latencies
- Fluent with Big Data technologies like Spark, Kafka and Hive
- Designing and building data pipelines using API ingestion and Streaming ingestion methods
- Knowledge of Dev-Ops processes (including CI/CD) and Infrastructure as code
- Experience in developing NoSQL solutions using Azure Cosmos DB
- Thorough understanding of Azure and AWS Cloud Infrastructure offerings
Key Responsibilities:
- Design and implement scalable and secure data processing pipelines using Azure Data Factory, Azure Databricks, and other Azure services
- Manage and optimize data storage using Azure Data Lake Storage, Azure SQL Data Warehouse, and Azure Cosmos DB
- Monitor and troubleshoot data-related issues within the Azure environment to maintain high availability and performance
- Automate data pipelines and workflows to streamline data ingestion, processing, and distribution tasks
- Utilize Azure's analytics services, such as Azure Synapse Analytics, to provide insights and support data-driven decision-making
- Provide guidance and support for data governance, including metadata management, data lineage, and data cataloging
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.