Data Platform Engineer (Databricks/Pyspark) - W2

Houston, TX, US • Posted 1 hour ago • Updated 57 minutes ago
Contract Corp To Corp
Contract Independent
Contract W2
On-site
$DOE
Company Branding Image
Fitment

Dice Job Match Score™

🔗 Matching skills to job...

Job Details

Skills

  • Python
  • SQL
  • Spark
  • Databricks
  • rbac
  • Delta Lake
  • Unity Catalog
  • ABAC

Summary

Job Title: IT Data Engineering Platform Engineer
Location: Dairy Ashford, Houston, TX (Hybrid)
Employment Type: Contract (Flex) 8 months with potential for extension
Job Summary
Shell is seeking an experienced Platform Engineer Data Engineering to support and govern a shared Databricks analytics platform. This role focuses on platform administration, data governance, security, cost management, and operational stability, enabling multiple analytics and AI teams to build solutions safely, efficiently, and consistently.
The ideal candidate will act as the platform owner for guardrails and standards, ensuring scalable, compliant, and cost effective use of Databricks across business teams.
Key Responsibilities
  • Platform Administration & Governance
  • Administer Databricks workspaces, clusters, jobs, compute policies, Unity Catalog, and environment configurations.
  • Define, implement, and enforce platform guardrails and engineering standards.
  • Establish and maintain RBAC and ABAC access controls to ensure secure and compliant data access.
  • Enforce data ingestion standards, naming conventions, schema rules, Delta Lake design patterns, and data quality expectations.
Data Quality & Ingestion Standards
  • Define platform-wide standards for ingestion pipelines, Delta architecture, lineage, versioning, and validation.
  • Review and approve data pipelines to ensure compliance with platform requirements.
  • Partner with data engineering teams to promote best practices and consistency.
Security, Compliance & Access Controls
  • Manage workspace and catalog permissions, including row- and column-level security and attribute-based access policies.
  • Collaborate with security and compliance teams to enforce enterprise data protection standards.
Cost Management & Monitoring
  • Implement cost controls, thresholds, alerts, and compute policies to prevent overspend.
  • Monitor job and cluster usage, identify anomalies, and recommend optimization strategies.
  • Provide transparency into workspace-level and SKU-level cost trends.
Operational Stability & Observability
  • Ensure platform reliability using automated testing, CI/CD templates, and code governance practices.
  • Build dashboards to monitor data access, pipeline health, schema drift, compliance, and cost thresholds.
  • Resolve platform incidents and implement preventative controls.
Enablement & Best Practices
  • Define and document standardized patterns ("handrails") for ingestion, Delta Lake usage, CI/CD, observability, and AI/ML workloads.
  • Support and coach analytics and data engineering teams on compliant onboarding and optimal platform usage.
  • Maintain internal documentation, reusable templates, and engineering guidelines.
Required Qualifications
  • 5+ years of experience in data engineering or platform engineering, including 2 3+ years administering Databricks.
  • Strong expertise in Databricks platform administration, including Unity Catalog, cluster policies, jobs, Spark, and Delta Lake.
  • Solid understanding of data governance, ingestion frameworks, schema enforcement, versioning, and data lineage.
  • Proven experience implementing RBAC and ABAC access controls.
  • Experience with cost optimization, usage monitoring, and compute governance.
  • Proficiency in Python/PySpark and SQL.
  • Familiarity with Databricks Workflows, Delta Live Tables (DLT), Airflow, or similar orchestration tools.
  • Strong communication skills with the ability to influence and guide cross functional teams.
Preferred Qualifications
  • Experience supporting large-scale enterprise data platforms in Azure, AWS, or Google Cloud Platform.
  • Exposure to trading, supply chain, or other high impact analytical environments.
  • Experience building dashboards for governance, cost management, compliance, and pipeline health.
  • Experience with CI/CD tools such as GitHub Actions, Azure DevOps, or similar.
Success Metrics
  • Consistent adoption of ingestion and data standards across teams.
  • Predictable and controlled platform costs.
  • Strong utilization of platform guardrails, templates, and dashboards.
  • Reduced incidents related to access control, data quality, or cost overruns.

Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.
  • Dice Id: 10515154
  • Position Id: 2026-6325/35038
  • Posted 1 hour ago

Company Info

About Aroha Technologies

Michael Arrington, founder and co-editor of Tech Crunch once famously said that ‘The best startups generally come from somebody needing to scratch an itch’.

 Quite true to the thought, Aroha Technologies Inc was launched in 2008 in Florida. While we began our journey with a powerful Staffing Specialist & Software Development, Today our procedure has amplified into a more robust and simplified solution for Marketers looking for consistent and long term.

In our quest to do better, we resolve to simplify user acquisition, retention, engagement and large Business without blinking an eyelid! A semi-remote team spread across the globe, we are here to help you one step at a time.

Unmatchable growing excellence When AROHA was formed, we interviewed many potential clients and business colleagues and asked what a single important question seemed: “If you could design a firm to meet your needs, what would it look like?” The answers to that question have guided our thinking and planning on an ongoing basis, and form the core of our thinking about how we treat our clients and our employees, whom we call associates.

At a most basic level, we have stated our mission in a promise to our clients and our associates: “Unmatchable growing excellence.” This alignment governs superior service to our customers and supports our associates who perform that work, which cannot be matched by our competitors.

In short, we believe that care of our associates, disciplined financial management, and constant attention to the fundamentals of good delivery adds up to a successful, scalable business.

About_Company_OneAbout_Company_Two
Create job alert
Set job alertNever miss an opportunity! Create an alert based on the job you applied for.

Similar Jobs

Houston, Texas

Today

Easy Apply

Third Party, Contract

$DOE

Culver City, California

Today

Easy Apply

Contract, Third Party

$DOE

Search all similar jobs