Databricks Architect Expert

Job ID: 2507298

Location: REMOTE WORK, VA, US

Date Posted: 2025-07-07

Category: Information Technology

Subcategory: Platform Engr

Schedule: Full-time

Shift: Day Job

Travel: No

Minimum Clearance Required: None

Clearance Level Must Be Able to Obtain: Public Trust

Potential for Remote Work: Yes

Description

Summary:

We are seeking an experienced Databricks Architect Expert with a strong data engineering background to support the design and delivery of secure, monitored, cloud-hosted environments. The ideal candidate will possess hands-on experience with data science and analytics, extensive use of AI/ML and LLM models, AI/MLOps, DevSecOps, and GenAI tools and technologies, with expertise in deploying solutions to high-stakes production environments.

The successful candidate will join our platform team, which provides capabilities, tools, guardrails, and various patterns for our customers within a federal agency. Our goal is to support customers as they bring their applications/models into our environment, facilitating the training, building, tweaking, and productionalization of their data and models.

Key Responsibilities:

• Design and implement event-driven architecture to support scalable, resilient, and dynamic GenAI applications.
• Refactor and rehost existing applications onto cloud platforms, optimizing for performance and scalability.
• Collaborate on the development of Platform Architecture, ensuring seamless integration and efficient operation of cloud services.
• Develop and optimize Databricks Solution Architectures to handle large-scale data processing and analytics workloads.
• Engineer robust Bedrock Solution Architectures to support infrastructure provisioning and management at scale.
• Define and execute Deployment Architectures that allow for automated and reliable deployments of applications and services.
• Establish and enforce Security Architectures to protect data and maintain compliance with industry and regulatory standards.
• Create Use Case Solution Architectures, transforming business requirements into technical solutions.
• Proactively identify and address technical issues and challenges to improve the efficiency and reliability of cloud-based systems.
• Work with cross-functional teams to facilitate the smooth deployment of cloud services across various operational environments.
• Document architectural decisions and create best practice guidelines for cloud development and deployment.
• Stay abreast of emerging cloud technologies and recommend adaptations to keep ahead of industry trends.
• Utilize Agile and DevSecOps methodologies and practices.
• Coach and provide direction to junior engineers in the Databricks and data engineering areas.

Lead efforts to integrate various data sources, ensuring seamless data flow and transformation. Develop strategies for data ingestion, cleansing, normalization, and integration.

• Leverage experience with MLFlow and general CI/CD principles for automating model deployment.
• Collaborate with customers and critical resources in an SME program to come up with solutions that align with long-term roadmaps and strategy.
• Execute Databricks upgrades, particularly in the federal space.

Qualifications

Qualifications:

• Bachelor's or master's degree in computer science, Data Science, Engineering, or a related field with 8 years of experience.
• Proven experience in cloud engineering and architecture, with a focus on event-driven solutions.
• In-depth knowledge of public cloud platforms (e.g., AWS, Azure, Google Cloud, Oracle).
• Proficiency with Databricks and understanding of big data ecosystems.
• Experience with Infrastructure as Code (IaC) using tools like Terraform, AWS CloudFormation, etc.
• Strong understanding of deployment pipelines and automated testing frameworks.
• Demonstrated experience in implementing security practices in cloud architectures.
• Clear understanding of microservices architecture and containerization technologies (e.g., Docker, Kubernetes).
• Excellent verbal and written communication skills, with the ability to document and explain complex architectures to non-technical stakeholders.
• Databricks Certification and any other AI/ML or data engineering certifications (AWS, Azure, Google Cloud Platform).

Desired Skills:

• Previous experience working in a regulated environment, particularly within government agencies like the IRS.
• Certifications related to AWS or cloud architecture.
• Experience with AI/ML platforms and understanding of their integration with cloud environments.
• Prior experience with event-driven architectures and related technologies (e.g., Apache Kafka, RabbitMQ).
• Prior experience in a high-stakes production environment where uptime is critical.

Target salary range: $160,001 - $200,000. The estimate displayed represents the typical salary range for this position based on experience and other factors.