Senior Lead Data Engineer (R4680)

San Francisco, CA, US • Posted 1 day ago • Updated 12 hours ago
Full Time
On-site
USD $140,000.00 - 210,000.00 per year
Fitment

Dice Job Match Score™

📋 Comparing job requirements...

Job Details

Skills

  • Apache HiveMind
  • LinkedIn
  • YouTube
  • Research and Development
  • Accounting
  • Program Management
  • Cloud Architecture
  • Data Modeling
  • Advanced Analytics
  • Lean Methodology
  • Data Science
  • Backbone.js
  • Decision-making
  • Reporting
  • Cost Control
  • ELT
  • Extract
  • Transform
  • Load
  • Semantics
  • Orchestration
  • Business Intelligence
  • Statistical Models
  • Collaboration
  • GL
  • Forecasting
  • Data Quality
  • Mentorship
  • Code Review
  • Documentation
  • Data Engineering
  • Data Architecture
  • SQL
  • Python
  • PySpark
  • Microsoft
  • Databricks
  • Snow Flake Schema
  • Dimensional Modeling
  • Analytics
  • Regulatory Compliance
  • Statistics
  • Evaluation
  • Microsoft Azure
  • Amazon Web Services
  • Cloud Computing
  • SAN
  • Security Controls
  • Machine Learning (ML)
  • Workflow
  • Finance
  • Enterprise Resource Planning
  • FP&A
  • Project Accounting
  • FAR
  • Auditing
  • Continuous Integration
  • Continuous Delivery
  • GitHub
  • SAP LE
  • Military
  • Artificial Intelligence
  • Recruiting

Summary

Founded in 2015, Shield AI is a venture-backed deep-tech company with the mission of protecting service members and civilians with intelligent systems. Its products include the V-BAT and X-BAT aircraft, Hivemind Enterprise, and the Hivemind Vision product lines. With offices and facilities across the U.S., Europe, the Middle East, and the Asia-Pacific, Shield AI's technology actively supports operations worldwide. For more information, visit Follow Shield AI on LinkedIn, X, Instagram, and YouTube.

Job Description:

Shield AI is building a modern financial enterprise reporting and analytics platform to support a complex, R&D-intensive, multi-billion-dollar organization. As a Senior Data Engineer, you will architect and implement secure, scalable, and cost-efficient data systems in Microsoft Fabric that power Finance, FP&A, Accounting, Program Management, and Executive reporting.

This role goes beyond traditional ETL development. You will own cloud architecture decisions, drive data modeling standards, embed advanced analytics readiness into the platform, and ensure that our systems are built for auditability, governance, and scale. In a lean team, you will operate across engineering, analytics, and applied data science enablement.

You will design the backbone of a mission-critical analytics platform that supports executive decision-making, forecast accuracy, compliance reporting, and scalable AI adoption. Your work will directly influence cost control, capital allocation, and operational insight across the enterprise.

What you'll do:

  • Architect and implement end-to-end ELT/ETL pipelines (Python, SQL, PySpark) in Microsoft Fabric Lakehouse environments, with prior experience in Azure Data Factory, Synapse, or comparable Azure-native data platforms.
  • Design and optimize medallion architectures (raw, curated, semantic) for enterprise-scale financial and corporate data.
  • Build secure, automated ingestion frameworks integrating ERP, FP&A, project accounting, and operational systems.
  • Design orchestration and monitoring frameworks to ensure reliability, observability, and audit readiness.
  • Optimize Fabric compute unit (CU) usage, query folding strategies, partitioning, indexing, and workload isolation.
  • Develop production-grade data models that support BI, self-service analytics, and advanced statistical modeling.
  • Collaborate on productionizing ML-enabled workflows such as GL classification, anomaly detection, and forecasting.
  • Implement data quality validation, reconciliation automation, and governance controls aligned with compliance standards.
  • Partner with Finance and Product stakeholders to translate business needs into scalable technical solutions.
  • Mentor engineers and contribute to engineering standards, CI/CD, code review practices, and documentation.

Required qualifications:

  • 7+ years of experience in data engineering or cloud data architecture roles.
  • Expert-level proficiency in SQL and Python; strong hands-on experience with PySpark.
  • Production experience in Microsoft Fabric, Databricks, Snowflake, or comparable cloud data platforms.
  • Strong understanding of dimensional modeling, star schemas, and analytics engineering best practices.
  • Experience designing secure, compliance-aware data platforms in regulated environments.
  • Demonstrated ability to independently own architectural decisions and deliver enterprise-grade systems.
  • Working knowledge of statistics, model evaluation concepts, and ML production workflows.

Preferred qualifications:

  • Experience with Azure or AWS cloud-native services (IAM, storage, networking, security controls).
  • Hands-on exposure to ML model deployment pipelines and feature engineering workflows.
  • Experience with finance systems (ERP, FP&A platforms, project accounting systems).
  • Familiarity with governance frameworks (DCAA, FAR/DFARS, CUI) and audit support processes.
  • Experience building CI/CD pipelines for data systems using GitHub or equivalent tools.

$140,000 - $210,000 a year

#LI-TF1

#LE

Full-time regular employee offer package:

Pay within range listed + Bonus + Benefits + Equity

Temporary employee offer package:

Pay within range listed above + temporary benefits package (applicable after 60 days of employment)

Salary compensation is influenced by a wide array of factors including but not limited to skill set, level of experience, licenses and certifications, and specific work location. All offers are contingent on a cleared background and possible reference check. Military fellows and part-time employees are not eligible for benefits. Please speak to your talent acquisition representative for more information.

Shield AI is proud to be an equal opportunity workplace and is an affirmative action employer. We are committed to equal employment opportunity regardless of race, color, ancestry, religion, sex, national origin, sexual orientation, age, marital status, disability, gender identity or Veteran status. If you have a disability or special need that requires accommodation, please let us know.

We may use artificial intelligence (AI) tools to support parts of the hiring process, such as reviewing applications, analyzing resumes, or assessing responses. These tools assist our recruitment team but do not replace human judgment. Final hiring decisions are ultimately made by humans. If you would like more information about how your data is processed, please contact us.
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.
  • Dice Id: 91093533
  • Position Id: 666e8496e5c32d1b4c5754f3d252d76c
  • Posted 1 day ago
Create job alert
Set job alertNever miss an opportunity! Create an alert based on the job you applied for.

Similar Jobs

San Francisco, California

Today

Full-time

USD 120,000.00 - 180,000.00 per year

Oakland, California

Today

Full-time

San Francisco, California

Today

Full-time

USD 210,800.00 - 341,000.00 per year

San Francisco, California

24d ago

Full-time

USD 158,400.00 - 198,000.00 per year

Search all similar jobs