Job ID: 2511326
Location: REMOTE WORK, DC, US
Date Posted: 2025-11-04
Category: Engineering and Sciences
Subcategory: Solutions Archt
Schedule: Full-time
Shift: Day Job
Travel: No
Minimum Clearance Required: None
Clearance Level Must Be Able to Obtain: Public Trust
Potential for Remote Work: Yes
Description We are seeking an experienced and versatile
Databricks Architect to lead the architectural design and implementation of Databricks-based solutions within a broader AI/ML platform environment. This role is part of the core platform team enabling scalable, secure, and reusable data science and GenAI workflows across federal mission use cases. The ideal candidate will have strong data architecture skills, a deep understanding of the AI/ML lifecycle, and familiarity with both traditional and emerging GenAI technologies.
This architect plays a vital role in enabling the
full lifecycle of AI/ML model development for IRS data scientists-from data ingestion and feature engineering to model training, deployment, and monitoring. As the platform team serves data science users directly, the architect must ensure Databricks capabilities are accessible, extensible, and aligned with user workflows.
You will collaborate with infrastructure, security, compliance, and AI engineering teams to ensure Databricks capabilities are integrated seamlessly with supporting services such as S3, Amazon SageMaker, Bedrock, and shared pipeline frameworks. You will also work closely with the
Product Manager and Chief Architect to inform roadmap evolution and ensure technical feasibility, performance, and alignment with broader architectural direction. As a hands-on architect, you will guide the design of workspaces, cluster policies, CI/CD integration, and governance structures that enable secure and efficient AI/ML model development and deployment.
Key Responsibilities: - Lead the architecture and design of Databricks environments (multi-tenant, secure, scalable) to support diverse AI/ML and GenAI workloads.
- Define and maintain workspace patterns, access policies, cluster configurations, and notebook orchestration aligned to platform governance.
- Collaborate with AI/ML engineers, data scientists, and platform SMEs to optimize data pipelines, feature stores, and model training processes.
- Drive integration of Databricks with enterprise data sources and services including AWS S3, IAM, SageMaker, Bedrock, and Terraform-managed infrastructure.
- Support development of reusable accelerators, templates, and onboarding patterns to reduce friction for mission teams.
- Provide architectural input into CI/CD pipelines for notebook deployment, model packaging, and automation of data workflows.
- Work with architects from adjacent platform areas (infrastructure, security, LLM, CSx) to ensure end-to-end solution alignment.
- Participate in planning and estimation activities with service delivery and product leadership.
Qualifications Required Qualifications: - Bachelor's or master's degree in computer science, Data Engineering, or a related technical field.
- 8+ years of experience in data architecture, platform engineering, or advanced analytics.
- 5+ years of hands-on experience designing and implementing Databricks environments in AWS.
- Strong working knowledge of Spark, Delta Lake, MLflow, Unity Catalog, and Databricks Runtime for ML.
- Solid foundation in data modeling, ETL pipeline design, and modern data lake architecture.
- Familiarity with MLOps practices, model lifecycle management, and compliance-aligned deployment patterns.
Desired Skills: - Active IRS Security Clearance is highly desired.
- Awareness of GenAI integration patterns and experience supporting LLM experimentation or prompt-based workflows.
- Experience working with infrastructure-as-code and CI/CD tools (Terraform, Bitbucket, Bamboo).
- Understanding of government compliance standards (FedRAMP, NIST, SOC2) and secure boundary design.
- Ability to collaborate across multiple engineering teams to support reusable, well-governed platform architecture.
- Excellent communication skills and technical documentation ability.
Target salary range: $200,001 - $240,000. The estimate displayed represents the typical salary range for this position based on experience and other factors.
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.
- Dice Id: 10111346
- Position Id: 2511326
- Posted 5 hours ago
Never miss an opportunity! Create an alert based on the job you applied for.
