Overview
On Site
BASED ON EXPERIENCE
Contract - W2
Contract - Independent
Contract - 10+ mo(s)
Skills
Collaboration
Data Masking
Auditing
Meta-data Management
Dimensional Modeling
Visualization
Reporting
Business Intelligence
Dashboard
Workflow
Performance Tuning
Computer Science
Data Engineering
Optimization
Databricks
Apache Spark
Data Flow
SQL
Python
Pandas
PySpark
Microsoft SQL Server
Data Modeling
Data Governance
Data Security
Extract
Transform
Load
ELT
Data Quality
Microsoft Power BI
Semantics
Modeling
Unity
Real-time
Data Processing
Streaming
Regulatory Compliance
Microsoft Azure
DevOps
Terraform
Continuous Integration
Continuous Delivery
Microsoft
Analytics
Soft Skills
Analytical Skill
Problem Solving
Conflict Resolution
Communication
IT Consulting
Staff Augmentation
Information Technology
Insurance
Life Insurance
SANS
Job Details
Analytics Engineer
Hybrid
About the Job
Data Engineering & Pipeline Development
Preferred Skills:
Certifications (preferred):
Soft Skills:
About our Company
DataSoft Technologies is a highly recognized provider of professional IT Consulting services in the US. Founded in 1994, DataSoft Technologies, Inc. provides staff augmentation services for Information Technology and Automotive Services. Our team member benefits include:
Hybrid
About the Job
- Duration: 12 Month Contract (possibility of extension)
- Location: - Hybrid: Atlanta, GA
- Pay rate: Hourly, depending on experience
- Job ID: 768025
Data Engineering & Pipeline Development
- Build and maintain ETL/ELT pipelines using Azure Databricks and Microsoft Fabric.
- Implement medallion architecture (Bronze, Silver, Gold layers) to support data lifecycle and quality.
- Develop real-time and batch ingestion processes from IES Gateway and other source systems.
- Ensure data quality, validation, and transformation logic is consistently applied.
- Use Python, Spark, and SQL in Databricks and Fabric notebooks for data transformation.
- Delta Lake: Implementing Delta Lake for data versioning, ACID transactions, and schema enforcement.
- Integration with Azure Services: Integrating Databricks with other Azure services like Azure One Lake, Azure ADLS Gen2, and Microsoft fabric.
- Collaborate with the Domain Owners to design dimensional and real-time data models.
- Create analytics-ready datasets for Power BI and other reporting tools.
- Standardize field naming conventions and schema definitions across datasets.
- Apply data classification and tagging based on data governance framework.
- Implement row-level security, data masking, and audit logging as per compliance requirements.
- Support integration with Microsoft Purview for lineage and metadata management.
- Dimensional modeling
- Real-time data modeling patterns
- Partner with BI developers to ensure data models are optimized for Power BI.
- Provide curated datasets that align with reporting requirements and business logic.
- Create BI dashboards and train users.
- Support CI/CD pipelines for data workflows using Azure DevOps.
- Assist in monitoring, logging, and performance tuning of data jobs and clusters.
- Bachelor's degree in computer science, Data Engineering, or related field.
- 3+ years of experience in data engineering or analytics engineering roles.
- Advanced SQL: Proficiency in advanced SQL techniques for data transformation, querying, and optimization.
- Hands-on experience with:
- Azure Databricks (Spark, Delta Lake)
- Microsoft Fabric (Dataflows, Pipelines, OneLake)
- SQL and Python (Pandas, PySpark)
- SQL Server 2019+
- Familiarity with data modeling, data governance, and data security best practices.
- Strong understanding of ETL/ELT processes, data quality, and schema design.
Preferred Skills:
- Experience with Power BI datasets and semantic modeling.
- Knowledge of Microsoft Purview, Unity Catalog, or similar governance tools.
- Exposure to real-time data processing and streaming architectures.
- Knowledge of federal/state compliance requirements for data handling
- Familiarity with Azure DevOps, Terraform, or CI/CD for data pipelines.
Certifications (preferred):
- Microsoft Fabric Analytics Engineer.
Soft Skills:
- Strong analytical and problem-solving abilities.
- Excellent communication skills for technical and non-technical audiences.
- Experience working with government stakeholders.
About our Company
DataSoft Technologies is a highly recognized provider of professional IT Consulting services in the US. Founded in 1994, DataSoft Technologies, Inc. provides staff augmentation services for Information Technology and Automotive Services. Our team member benefits include:
- Paid Holidays/Paid Time Off (PTO)
- Medical/Dental Insurance
- Vision Insurance
- Short Term/Long Term Disability
- Life Insurance
- 401 (K)
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.