Lead Data Engineer

Santa Clara, CA, US • Posted 1 day ago • Updated 1 day ago

Contract Corp To Corp

Contract Independent

Contract W2

No Travel Required

On-site

Depends on Experience

HPTech Inc.

Fitment

Dice Job Match Score™

📋 Comparing job requirements...

Job Details

Skills

Data Engineer

Summary

Role: Lead Data Engineer
Location: Santa Clara, CA (Day 1 Onsite)

We are looking for a highly skilled and motivated Data Engineer to join our team. The ideal candidate will be responsible for designing, building, and maintaining scalable data infrastructure that drives business intelligence, advanced analytics, and machine learning initiatives. You must be comfortable working autonomously, navigating complex challenges, and driving projects to successful completion in a dynamic cloud environment.

Core Responsibilities
• Design and Optimization: Design, implement, and optimize clean, well-structured, and performant analytical datasets to support high-volume reporting, business analysis, and data science model development.
• Pipeline Development: Architect, build, and maintain scalable and robust data pipelines for diverse applications, including business intelligence and advanced analytics.
• Big Data & Streaming: Implement and support Big Data solutions for both batch (scheduled) and real-time/streaming analytics.
• Collaboration: Work closely with product managers and business teams to understand data requirements and translate them into technical solutions.
• DevOps Integration: Collaborate with DevOps teams to ensure smooth deployment, monitoring, and maintenance of data pipelines and infrastructure in cloud environments.

Required Skills & Experience
• Cloud Platform Expertise (Google Cloud Platform Focus): Extensive hands-on experience working in dynamic cloud environments, with a strong preference for Google Cloud Platform (Google Cloud Platform) services, specifically:
o BigQuery: Expert-level skills in data ingestion, performance optimization, and data modeling within a petabyte-scale environment.
o Experience with other relevant Google Cloud Platform services like Cloud Storage, Cloud Dataflow/Beam, or Pub/Sub.
• Programming & Querying:
o Python: Expert-level programming proficiency in Python, including experience with relevant data engineering libraries.
o SQL: A solid command of advanced SQL for complex querying, data processing, and performance tuning.
• Data Pipeline Orchestration: Prior experience using workflow management and orchestration tools (e.g., Apache Airflow, Cloud Composer, Dagster, or similar).

DevOps/CI/CD:
o Strong understanding of DevOps principles and practices.
o Experience with CI/CD pipelines, automation tools, and deployment strategies.
o Familiarity with version control systems (Git) and tools like GitLab CI/CD, GitHub Actions, or Jenkins.
o Knowledge of containerization (Docker) and orchestration tools (Kubernetes) is a plus.
• Monitoring & Automation: Ability to implement monitoring solutions and automate operational tasks to ensure reliability and scalability.

Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.

Dice Id: 91162806
Position Id: 8858092
Posted 1 day ago

Company Info

About HPTech Inc.

HPTech Inc has grown to become a premier provider of IT consulting services for technology. With a specialized niche in transaction and data warehousing applications, HPTech Inc helps clients implement mission-critical, enterprise solutions, speed conversion from legacy environments, and optimize the benefits inherent in their systems. Our consulting solutions range from project strategy and planning, software assessment and selection, to implementation and integration, hosting and change management, support and maintenance translating to higher productivity, lower costs.

Go to company profile

Create job alert

Never miss an opportunity! Create an alert based on the job you applied for.