Role : Senior Data Engineer (Platform, ML & Analytics)
Employment type : FullTime
Location : Remote
Experience: 7-9 years in data engineering or ML data engineering, in production Google Cloud Platform
Location: Remote (US)
Stack: Google Cloud Platform, BigQuery, Kafka, Dataflow, Spark (DataSpark), Vertex Al, Cloud Storage, Adobe Analytics, Python/Java/Node.js
Key asks
- Ingestion: Stream health, benefits, news and click-through files (CV/Parquet) through Kafka and Dataflow into Cloud Storage and BigQuery-
- Transformation: Run ETL/ELT jobs on Dataflow and Datapros Spark to produce clean, normalized BigQuery datasets.
- Curated marts: Build 360° denormalized views that combine prescription (Rx), benefits, behavioral and clinical signals.
- Feature Store and ML datasets: Load the Google Cloud Platform Feature Store, build batch and online feature paths, create train/eval splits and version datasets.
Monitor data drift and feature freshness with Vertex Al.
- Adobe Analytics: Bring in behavioral event data and line it up with clinical and benefits data.
- Data quality and BigQuery operations: Set up SLAs, alerting, schema validation and anomaly detection. Manage partitioning, schema changes and cost.
- Disaster recovery: Set recovery time and recovery point targets (RTO/RPO), replicate BigQuery across regions, maintain DR buckets and Feature Store snapshots, and test failover runbooks.
- Archival: Set BigQuery table expiry, move older data to cheaper Cloud Storage tiers (Nearline, Coldline, Archive) and meet HIPAA retention
rules.
- HIPAA and patient health information (PHI): Apply field-level encryption and masking and de-identify data (Safe Harbor or Expert
Determination) before training. Enforce access controls and audit logging.
- Documentation: Write up data contracts, data lineage, Feature Store definitions, DR runbooks and archival policies.
Must-have skill
- Core data skills: BigQuery (advanced), Kafka, Dataflow/Beam, Spark (PySpark or Scala), Pub/Sub, Composer/Airflow, plus strong SQL and data modeling (Lakehouse).
- ML data work: Feature Stores, Vertex Al Pipelines or similar MLOps tools, and the ML lifecycle.
- Proven experience with: DR design, archival and retention for regulated data, and hands-on HIPAA/PHI controls.
Nice to have
Adobe Experience Platform, Looker, Google Cloud Platform customer-managed encryption keys (CMEK), HIPAA business associate agreement (BAA) awareness,
NIST/HITRUST, and healthcare data experience (Rx, clinical or benefits).
Shivani | Sr. Talent Acquisition Specialist
Amaze Systems Inc
E: |