Tech Lead with Microsoft Fabric, Databricks
Huston, TX (Onsite)
Must have: Fabrick Lakehouse
Position Summary
The Technical Lead – Data Platforms & Analytics will provide hands-on technical leadership for the design, implementation, administration, security, and continuous improvement of the enterprise data platform. The role will lead solutions across Microsoft Fabric, OneLake, Azure Databricks, Unity Catalog, and Microsoft SQL database technologies, enabling reliable and governed data products for oil and gas operations, finance, asset management, supply chain, and corporate functions. This position combines architecture, engineering, platform operations, database administration, data governance, and technical team leadership.
Key Responsibilities
- Technical leadership: Lead solution design, engineering standards, code and architecture reviews, delivery planning, troubleshooting, and technical mentoring for data engineering and platform teams.
- Microsoft Fabric and OneLake: Design and operate Fabric workspaces, Warehouses, pipelines, notebooks, semantic models, and data products; establish OneLake organization, naming, security, lifecycle, and performance standards.
- Azure Databricks: Architect and optimize Databricks workspaces, clusters, jobs, notebooks, Delta Lake solutions, medallion architectures, and batch or streaming pipelines using SQL, Python, PySpark, and Spark.
- Unity Catalog: Implement centralized governance for catalogs, schemas, tables, views, volumes, models, and lineage; administer permissions, service principals, managed identities, external locations, and least-privilege access.
- SQL database administration: Administer SQL Server and Azure SQL environments, including installation and configuration, upgrades, patching, backup and recovery, high availability, disaster recovery, security, capacity planning, performance tuning, indexing, and query optimization.
- Data integration: Build resilient ingestion and transformation patterns for operational applications, historians, ERP, field systems, APIs, files, and third-party data sources.
- Security and compliance: Apply identity, encryption, network isolation, secrets management, data classification, retention, auditing, and segregation-of-duties controls in partnership with cybersecurity and risk teams.
- DevOps and automation: Establish source control, branching, testing, CI/CD, infrastructure-as-code, release management, monitoring, alerting, and automated environment promotion.
- Reliability and support: Define service levels, observability, incident response, problem management, runbooks, capacity thresholds, and platform cost optimization practices.
- Stakeholder partnership: Translate business requirements into scalable technical solutions and communicate architecture decisions, risks, dependencies, and trade-offs to technical and executive stakeholders.
Oil & Gas Domain Responsibilities
- Develop governed data solutions supporting upstream, midstream, or oilfield services use cases such as drilling, production, well performance, equipment reliability, maintenance, HSE, logistics, and cost management.
- Integrate and contextualize time-series, telemetry, sensor, and transactional data while maintaining assets, well, field, basin, customer, and equipment hierarchies.
- Partner with operations and engineering teams to improve data quality, timeliness, traceability, and availability for field and corporate decision-making.
- Design for remote-site connectivity constraints, intermittent data transmission, high-volume telemetry, operational continuity, and recovery requirements.
- Apply awareness of relevant industry data standards and regulatory, records-management, cybersecurity, and audit obligations.
Required Qualifications
- Bachelor’s degree in computer science, information systems, engineering, data science, or related discipline, or equivalent practical experience.
- Eight or more years of progressive experience in data engineering, analytics platforms, database administration, or cloud data solutions, including at least three years in a technical lead or solution architecture capacity.
- Production experience with Microsoft Fabric and OneLake, including Lakehouse, Warehouse, Data Factory capabilities, notebooks, security, monitoring, and workload administration.
- Strong Azure Databricks experience with Delta Lake, Spark, SQL, Python or PySpark, job orchestration, performance tuning, and workspace administration.
- Hands-on experience implementing and administering Unity Catalog, including access controls, lineage, external locations, credential patterns, and governance operating models.
- Advanced SQL Server and Azure SQL database administration experience, including high availability, disaster recovery, backup and restore, security, patching, performance diagnostics, and query tuning.
- Strong knowledge of dimensional modeling, data lakehouse patterns, metadata management, master and reference data, data quality, and data lifecycle management.
- Experience with Azure identity, networking, Key Vault, private endpoints, monitoring, and cloud security controls.
- Experience with Git-based development, automated testing, CI/CD, and infrastructure-as-code tools.
- Demonstrated experience delivering data solutions in the oil and gas industry and working with operational and engineering stakeholders.
- Strong written, verbal, facilitation, and technical documentation skills.
Preferred Qualifications
- Microsoft certifications related to Fabric, Azure Data Engineering, Azure Database Administration, or Azure Solutions Architecture.
- Databricks Data Engineer, Data Analyst, or platform administration certification.
- Experience with Power BI semantic models, Direct Lake, data gateways, row-level security, and enterprise reporting governance.
- Experience with Purview, data cataloging, lineage, classification, and data loss prevention capabilities.
- Familiarity with operational technology data, SCADA, historians, IoT platforms, and common oil and gas applications or data exchange standards.
- Experience leading platform migrations, legacy modernization, cloud cost management, vendor engagement, and managed service delivery.
Leadership Competencies
- Leads through technical credibility, sound judgment, accountability, and a strong service mindset.
- Balances strategic architecture with pragmatic, hands-on delivery.
- Creates clear standards while enabling teams to deliver with appropriate autonomy.
- Explains complex technical concepts, risks, and trade-offs in business terms.
- Builds productive partnerships across data, applications, infrastructure, cybersecurity, operations, engineering, and business teams.
- Coaches team members, develops reusable practices, and promotes continuous learning.
Measures of Success
- Reliable, secure, scalable, and cost-effective operation of Fabric, OneLake, Databricks, Unity Catalog, and SQL platforms.
- Improved platform availability, pipeline success rates, recovery readiness, query performance, and incident resolution times.
- Consistent adoption of architecture, engineering, security, governance, and DevOps standards.
- Faster delivery of trusted data products with measurable improvements in data quality, lineage, and user adoption.
- Effective knowledge transfer, team capability growth, and reduced dependency on individual subject-matter experts.
Work Environment
This role may require collaboration across multiple time zones, participation in an on-call or escalation rotation, and occasional travel to office, data center, operational, or field locations. The successful candidate must be able to work effectively with both corporate technology teams and field-based operational stakeholders.
