Databricks Architect

Job#: 2075713

Job Description:

Senior Lead Consultant - Databricks Platform Architect

Overview:

Apex Systems is seeking a Senior Lead Consultant to serve as a Databricks Architect, leading greenfield data platform modernization initiatives, helping enterprise clients design and deliver Databricks-based solutions from inception to production. This role is part of Apex Systems' Solutions Delivery Organization and will be responsible for building scalable, cloud-based data and analytics platforms that integrate seamlessly with enterprise environments.

As a technical leader, you will drive solution architecture, data migration, and integration strategies, ensuring flexibility, scalability, and adherence to best practices. Additionally, you will provide subject matter expertise to sales and account teams during the scoping of new opportunities.

Key Responsibilities:

• Serve as the technical lead for project teams, ensuring high-quality execution and alignment with best practices.
• Listen and seek to understand the client and meet their needs, providing consultative guidance.
• Stay attuned to the future needs of the client and work with internal resources to identify opportunities.
• Build and foster relationships with clients, stakeholders, and technical communities, representing Apex Systems at industry events.
• Stay current with technological trends, proactively identifying innovative solutions to improve client outcomes.
• Act as a mentor to junior engineers and technical team members, fostering skills development and knowledge sharing.
• Collaborate with internal and external stakeholders to ensure project alignment with business goals and success metrics.
• Contribute to the evaluation and recruitment of technical talent through professional networking and technical screenings.

Lead the end-to-end development of greenfield Databricks-based solutions, from architecture and design to implementation and delivery, ensuring alignment with business and technical requirements.

Develop scalable solution architectures, including data platforms, and integrations with enterprise applications and analytics platforms.

Advise clients on best practices for Databricks architecture, governance, and security to ensure long-term flexibility and scalability.

Drive cloud data platform modernization, leveraging Azure services (Azure Data Factory, Azure Functions, etc.) and complementary technologies.

Architect and implement streaming/batch data pipelines that optimize performance, cost, and operational efficiency.

Integrate Databricks solutions with enterprise data sources.

Implement enterprise-grade governance and security measures, including role-based access control (RBAC), data masking, and compliance frameworks.

Optimize Databricks environments, including storage and compute performance, partitioning strategies, and cluster configurations.

Act as a trusted advisor to clients and internal teams, guiding them through technical decisions and trade-offs.

Support pre-sales efforts, including solution scoping, architecture discussions, and client presentations.

Mentor and guide teams of developers and data engineers working within Agile delivery frameworks.

Required Skills & Experience:

7+ years of experience in enterprise cloud data platforms and architecture, with at least 3+ years of hands-on experience designing and deploying Databricks solutions.

Proven experience leading greenfield Databricks implementations, building platforms from the ground up in enterprise settings.

Deep experience in Infrastructure-as-Code technologies like Terraform, Bicep, and ARM Templates.

Deep expertise in Azure services that complement Databricks (Azure Data Factory, Azure Functions, Azure Synapse, etc.).

Deep expertise in Azure Cloud Networking, Security, Storage.

Proficiency in governance and security best practices, including Databricks Unity Catalog, RBAC, data encryption, and compliance controls.

Hands-on experience optimizing Databricks environments, including partitioning strategies, cluster sizing, and performance tuning.

Familiarity with streaming architectures and real-time data processing (Delta Live Tables, Kafka, etc.) is a plus.

Experience working in Agile delivery environments, leading teams through solution design, implementation, and deployment.

Strong consultative skills, with the ability to engage with stakeholders, define technical roadmaps, and deliver high-impact solutions.

• BA/BS required, preferably in Management Information Systems, Computer Information Systems, Computer Science, or related fields (applicable education and experience may be substituted).

Preferred Qualifications:

Prior experience at Databricks or working directly with Databricks professional services.

Databricks technical certifications.

• Exposure to IDMC (Informatica Intelligent Data Management Cloud) or other data integration platforms.

Experience with DataOps concepts and CI/CD pipelines for Databricks deployments.

Experience designing and implementing data pipelines using Python, SQL, and Spark (PySpark, Scala, or Java).

EEO Employer

Apex Systems is an equal opportunity employer. We do not discriminate or allow discrimination on the basis of race, color, religion, creed, sex (including pregnancy, childbirth, breastfeeding, or related medical conditions), age, sexual orientation, gender identity, national origin, ancestry, citizenship, genetic information, registered domestic partner status, marital status, disability, status as a crime victim, protected veteran status, political affiliation, union membership, or any other characteristic protected by law. Apex will consider qualified applicants with criminal histories in a manner consistent with the requirements of applicable law. If you have visited our website in search of information on employment opportunities or to apply for a position, and you require an accommodation in using our website for a search or application, please contact our Employee Services Department at or .

Apex Systems is a world-class IT services company that serves thousands of clients across the globe. When you join Apex, you become part of a team that values innovation, collaboration, and continuous learning. We offer quality career resources, training, certifications, development opportunities, and a comprehensive benefits package. Our commitment to excellence is reflected in many awards, including ClearlyRated's Best of Staffing in Talent Satisfaction in the United States and Great Place to Work in the United Kingdom and Mexico.

Apex Benefits Overview: Apex offers a range of supplemental benefits, including medical, dental, vision, life, disability, and other insurance plans that offer an optional layer of financial protection. We offer an ESPP (employee stock purchase program) and a 401K program which allows you to contribute typically within 30 days of starting, with a company match after 12 months of tenure. Apex also offers a HSA (Health Savings Account on the HDHP plan), a SupportLinc Employee Assistance Program (EAP) with up to 8 free counseling sessions, a corporate discount savings program and other discounts. In terms of professional development, Apex hosts an on-demand training program, provides access to certification prep and a library of technical and leadership courses/books/seminars once you have 6+ months of tenure, and certification discounts and other perks to associations that include CompTIA and IIBA. Apex has a dedicated customer service team for our Consultants that can address questions around benefits and other resources, as well as a certified Career Coach. You can access a full list of our benefits, programs, support teams and resources within our 'Welcome Packet' as well, which an Apex team member can provide.