Location: Elk Grove, CA
Salary: $180,000.00 USD Annually - $210,000.00 USD Annually
Description: We are seeking a high-level
Senior Data Platform Engineer to serve as a technical authority in the design, deployment, and optimization of enterprise-scale data ecosystems. This role is focused on building resilient data pipelines and modernizing platforms to support advanced analytics, reporting, and machine learning initiatives. You will act as a bridge between data architecture and business intelligence, ensuring all production systems are high-performing, secure, and scalable.
This is a
hybrid role requiring three days per week on-site on the south side of
Sacramento, CA, with no fully remote options available.
Key Responsibilities- Pipeline Architecture: Build and manage sophisticated ETL/ELT workflows to ingest and transform massive datasets across structured and unstructured formats.
- Platform Management: Oversee the stability and performance of big data environments (such as Databricks, Snowflake, or Google Cloud Platform), ensuring high availability for global data operations.
- AI & Search Optimization: Develop and refine Retrieval-Augmented Generation (RAG) systems, utilizing advanced chunking and semantic search strategies to enhance LLM integrations and enterprise knowledge retrieval.
- Performance Engineering: Identify and resolve bottlenecks in SQL queries, Spark jobs, and cluster configurations to maximize cost-efficiency and processing speed.
- Governance & Security: Execute rigorous data masking, access controls, and audit logging to maintain compliance and protect sensitive information.
- DevOps Integration: Embed data workflows into CI/CD pipelines, utilizing version control and infrastructure-as-code to automate deployments.
- Collaborative Problem Solving: Partner with cross-functional teams to translate complex business needs into technical requirements within an Agile framework.
Technical Requirements- Experience: 7+ years of hands-on data engineering experience, specifically supporting production-grade big data pipelines.
- Core Skills: Mastery of PySpark, Python, and SQL, with deep expertise in the Spark ecosystem and Data Lakehouse architectures (Delta Lake).
- AI/ML Infrastructure: Proven experience with vector databases, messaging/streaming (Kafka, Pub/Sub), and model tracking tools like MLflow.
- Cloud & Data Platforms: Advanced proficiency in modern cloud-native tools (BigQuery, Cloud Storage) and enterprise platforms like Databricks or Snowflake.
- Automation: Practical experience with CI/CD tools (Jenkins, Git) and infrastructure-as-code (Terraform).
- Communication: Strong ability to document operational procedures and present technical solutions to diverse stakeholders.
Tools & Preferred Qualifications- Data Governance: Experience with data cataloging and enterprise access control solutions.
- Analytics: Familiarity with BI tools such as Looker Studio or Power BI.
- Preferred Certifications: * Databricks, Snowflake, or Google Cloud Platform Data Engineer certifications.
- Google Professional Cloud Architect or Azure Solutions Architect Expert is a significant plus.
- Experience with Apache Airflow for orchestration.
By providing your phone number, you consent to: (1) receive automated text messages and calls from the Judge Group, Inc. and its affiliates (collectively "Judge") to such phone number regarding job opportunities, your job application, and for other related purposes. Message & data rates apply and message frequency may vary. Consistent with Judge's Privacy Policy, information obtained from your consent will not be shared with third parties for marketing/promotional purposes. Reply STOP to opt out of receiving telephone calls and text messages from Judge and HELP for help.
Contact: This job and many more are available through The Judge Group. Please apply with us today! 
Never miss an opportunity! Create an alert based on the job you applied for.