Senior Data Engineer

Role: Senior Data Engineer Company: Arch Systems Client: US Federal Govt. Location: Remote Type: Full-time

Senior Data Engineer Azure Power Platform & Dataverse Modernization

Role Overview
The Senior Data Engineer is a critical technical role responsible for architecting, engineering, and operating enterprise-grade data pipelines and integrations supporting the modernization of legacy systems (e.g., AMS) to the Microsoft Azure Power Platform and Dataverse ecosystem. This role ensures secure, scalable, compliant, and high-performance data movement, synchronization, and governance across academic, operational, and DoD-connected systems.
The position directly supports mission-critical student, cadet, and institutional data, requiring strict adherence to DoD 8140, NIST, STIG, and RMF requirements.

Minimum Experience
15+ years of progressive experience in data engineering, enterprise data integration, and platform modernization
Demonstrated experience supporting large-scale government or DoD modernization programs
Proven experience migrating legacy relational systems to cloud-native architectures

Core Responsibilities Data Architecture & Engineering
Design and implement end-to-end data architectures supporting Microsoft Dataverse as the system of record.
Engineer high-availability, fault-tolerant, and scalable data pipelines across Azure services.
Lead data migration strategies from legacy SQL-based systems to Dataverse with minimal downtime.
Implement bi-directional and near-real-time synchronization where required during phased cutovers.

Azure Platform & Connectivity
Design and manage Azure platform connectivity, including:
Secure VNET integrations
Private endpoints
Service endpoints
Configure and manage Azure Active Directory (Entra ID) access groups, role assignments, and least-privilege access controls.
Integrate Dataverse with Azure services while maintaining IL-2 / DoD enclave compliance.

Data Integration & Pipelines
Build and maintain enterprise data pipelines using:
Azure Data Factory
Azure Synapse Analytics
Develop ingestion, transformation, validation, and publishing workflows.
Ensure data pipelines support operational systems, reporting, analytics, and downstream DoD integrations.

SQL & Dataverse Engineering
Deep expertise in Azure SQL Database and SQL optimization.
Translate complex relational schemas into Dataverse tables, relationships, and business rules.
Optimize Dataverse performance through proper indexing, partitioning, and data modeling.
Support Power Apps (Model-Driven and Canvas) with performant, secure data structures.

Data Migration & Validation
Lead data migration planning and execution, including:
Data profiling and cleansing
Field mapping and transformation
Reconciliation and validation
Produce detailed data migration reports, error logs, and reconciliation artifacts.
Support multiple migration waves aligned with the program rollout timeline.

Security, Compliance & Governance
Implement data security controls aligned with:
DoDM 8140
NIST SP 800-53
DISA STIGs
Ensure encryption at rest and in transit.
Support audit readiness and compliance documentation.
Partner with cybersecurity teams to remediate findings and vulnerabilities.

Operational Support & Optimization
Monitor data pipelines and integrations for performance, reliability, and failures.
Implement logging, alerting, and telemetry for proactive issue resolution.
Support production operations, troubleshooting, and continuous improvement.

Mandatory Certifications (Required No Exceptions)
DoD / Cybersecurity
DoD 8140: IAT Level II
CompTIA Security+ CE (or DoD-approved equivalent)
Microsoft / Azure
Azure platform connectivity (hands-on enterprise experience required)
Azure SQL Database (design, performance tuning, migration)
Azure Data Factory
Azure Synapse Analytics
Microsoft Dataverse development
Azure Active Directory (Entra ID) groups and access management
All certifications and qualifications must be current and verifiable at time of contract award.
No grace period applies to contractor personnel.

Key Technical Skills
Azure networking and secure service integration
Enterprise SQL design and optimization
Data migration and transformation at scale
Dataverse schema design and performance tuning
Power Platform integration patterns
Identity and access management (IAM)
DevSecOps-aligned data engineering practices
Government data compliance and audit readiness