Data Engineer
SmallArc, Inc
Dice Job Match Score™
🔢 Crunching numbers...
Job Details
Skills
- Data Engineering
- Microsoft Azure
- Computerized System Validation
- Data Modeling
Summary
Job Overview
We are seeking a hands-on Data Engineer with strong Microsoft Azure experience to design, develop, and maintain reliable, scalable data pipelines and datasets that support analytics and operational insights. The ideal candidate will have extensive experience with Azure Data Factory (ADF), Azure Blob Storage, Azure SQL, data modeling, and ETL/ELT processes.
Key Responsibilities
Design, develop, schedule, and monitor batch and near-real-time data pipelines using Azure Data Factory (ADF).
Manage Azure Blob Storage, including containers, access tiers, lifecycle policies, and data partitioning.
Design and optimize Azure SQL databases, including schema design, indexing, query optimization, and performance tuning.
Develop dimensional data models using star and snowflake schemas, 3NF, and Data Vault 2.0 concepts.
Implement data quality, validation, governance, lineage, and schema evolution standards.
Optimize pipeline performance, implement monitoring and alerting, and troubleshoot production issues.
Collaborate with analytics, product, and operations teams to deliver secure, reliable, and well-documented data products.
Apply security best practices, including RBAC, encryption, data masking, and key management.
Develop reusable components and follow best practices for data engineering and CI/CD.
Required Qualifications
Bachelor's degree with 5 years of experience, or 4 years of experience in lieu of a degree.
Ability to obtain and maintain a Public Trust clearance, requiring U.S. citizenship.
Strong hands-on experience with Azure Data Factory, Azure Blob Storage, and Azure SQL.
Knowledge of star/snowflake schemas, 3NF, Data Vault, and Kimball vs. Inmon data warehousing methodologies.
Proficiency in SQL and data formats such as JSON, Parquet, CSV, and Avro.
Experience with Python (preferred) or Scala for data transformations and automation.
Experience with Git-based workflows and Azure DevOps or equivalent CI/CD tools.
Experience monitoring, troubleshooting, and maintaining production data pipelines.
Preferred Qualifications
Experience with Azure Synapse Analytics and large-scale ETL/ELT processing.
Familiarity with Microsoft Fabric, Lakehouse, OneLake, and data pipelines.
Power BI semantic modeling, DAX, and incremental refresh experience.
Knowledge of Event Hubs, Azure Functions, Databricks, or Spark Structured Streaming.
Experience with Terraform, Bicep, or ARM templates.
Familiarity with Microsoft Purview or similar data cataloging and lineage tools.
I look forward to your response. Please feel free to reach out if you have any questions.
- Dice Id: 10115873
- Position Id: 9123333
- Posted 6 hours ago
Company Info
Our firm was established back in 2000 by a group of consultants with a strong focus on delivering quality services to our clients from the perspective of those who worked in the field. Because of the huge amount of industry experience behind us, we knew that we could provide our clients with a no gimmicks, truly valuable set of solutions that exactly matched all of their IT needs.
In today’s digital world, we’re surrounded by information at all times. The purpose of SmallArc is to provide a cohesive set of information management solutions that organize, optimize and present information in an efficient and informative way.
SmallArc is a premier consulting firm specializing in Business Intelligence, Data Warehousing, and Data Integration. We provide end-to-end services from Data Architecture, Implementation, Project Management, Long Term Support, and Training.
We provide you with a competitive advantage which is derived from our extensive team of business intelligence experts. We recruit and employ only the best and brightest to ensure the highest level of knowledge and competence. We provide you with the most innovative, reliable, and consistent results on every project — no matter the scope or scale. We have earned a solid reputation for client success due to our unique ability to deliver business intelligence solutions on time, and under budget.
