Data Engineer

Data Engineer

Location: Remote

Duration: Contract

Job Summary

Must complete the Data Engineer Glider

Job Description

As a Data Engineer, you will be responsible for designing and implementing a cloud-native data processing and API integration system. This role involves ingesting identity data from upstream sources, detecting record-level changes, and synchronizing user metadata to a downstream system via API. This is a position that focuses on scalable data handling, automation, and fault-tolerant service deployment within Google Cloud Platform (Google Cloud Platform).

What You'll Do

• Solution Design & Development: Build modular Python applications that process identity data files or APIs and sync them to target platforms
• Data Staging & Processing: Stage identity metadata in BigQuery using defined schemas and implement change detection logic (create/update/delete)
• API Integration: Design and implement logic to call RESTful APIs to maintain target user repositories (e.g., user attributes, roles)
• Workflow Orchestration: Use Google Cloud Platform Pub/Sub, Composer, and/or Cloud Run to manage asynchronous workflows and ensure event-driven processing
• Infrastructure as Code: Deploy and manage services using Terraform with a focus on security, idempotency, and configuration as code
• Observability & Resilience: Implement logging, retry logic, and incident handling to ensure system reliability and traceability
• Testing & Validation: Build automated test coverage for critical processing logic and API interactions

What You'll Need

Required:

• Bachelor's or master's degree in computer science, data engineering, or equivalent work experience
• 6+ years in backend development or data engineering roles focused on identity, security, or metadata systems
• Strong Python engineering for data processing and backend development
• Advanced experience with Google Cloud Platform services: BigQuery, Cloud Run, Cloud Functions, Cloud Composer, Pub/Sub, Cloud Storage, Secret Manager, Cloud Scheduler
• Interacting with REST APIs, including OAuth2 or token-based authentication
• Terraform for cloud infrastructure automation
• Proficiency with SQL for data transformation and validation
• Strong understanding of CI/CD, containers (Docker), Git workflows
• Comfortable working with structured metadata, user roles, and directory-style data
• Able to work independently and meet delivery milestones
• Strong documentation and debugging skills
• Must adhere to enterprise security and change control practices

Preferred:

• Experience integrating with IAM or identity systems (e.g., LDAP, Okta, custom directories)
• Background working in regulated or high-security environments
• Experience handling large-scale user datasets (millions of records)
• Familiarity with hybrid data processing (batch + streaming)
• Google Cloud Platform Certifications

Physical Demands:

• Ability to safely and successfully perform the essential job functions consistent with the ADA and other federal, state, and local standards
• Sedentary work that involves sitting or remaining stationary most of the time with occasional need to move around the office to attend meetings, etc.
• Ability to conduct repetitive tasks on a computer, utilizing a mouse, keyboard, and monitor