![]()
Backend Software Engineer (Platform)
W2 Contract
Pay Rate: $55 - $65 per hour
Location: Cupertino, CA - Hybrid Role
Job Summary:
We are seeking strong Software Engineers to support onboarding and metadata ingestion into our Data Registry, a central data catalog platform. In this role, you will work on the systems and connectors that enable organizations across our organization to register datasets into the Data Registry through scalable, reliable ingestion pipelines. This position is ideal for an engineer who is comfortable working across data ingestion, connector development, operational troubleshooting, and platform support in a distributed data ecosystem.
Duties and Responsibilities:
- Build, enhance, and support metadata ingestion pipelines for onboarding datasets into a large Data Registry
- Develop and maintain connector-based ingestion workflows using open source DataHub connector patterns
- Troubleshoot failed ingestions, analyze logs, identify root causes, and restore pipeline health
- Work with internal customers to support both pull-based and push-based onboarding models
- Modify existing connectors and build custom Python and Scala-based ingestion components when required
- Support operational monitoring and ongoing reliability of ingestion services
- Investigate connectivity, deployment, and interoperability issues across distributed environments
- Collaborate with partner teams to onboard new data sources and improve onboarding efficiency
- Contribute to evolving ingestion architecture, including batch and selected real-time ingestion patterns
- Support platform transitions and deployment improvements across the onboarding ecosystem
Requirements and Qualifications:
- 5+ years of software engineering experience, or 3+ years with a master's degree
- Strong programming experience in Python and Scala
- Experience designing, building, or supporting ETL, ingestion, or metadata pipeline workflows
- Hands-on experience debugging production issues in distributed systems
- Strong understanding of logging, monitoring, and operational troubleshooting
- Experience working with data systems, connectors, or pipeline orchestration in cloud or hybrid environments
- Ability to write clean, maintainable, well-tested code and operate effectively in a fast-paced engineering environment
- Strong communication and collaboration skills
Preferred Qualifications:
- Experience with open source DataHub, especially connectors or metadata ingestion frameworks
- Working knowledge of Scala
- Experience with batch and streaming data technologies such as Spark, Kafka, or Flink
- Familiarity with monitoring and observability platforms such as Datadog
- Understanding of network connectivity concepts, including ingress, egress, and cross-system integration
- Experience building custom connectors or integration components for niche source systems
- Familiarity with deployment and onboarding patterns in modern data platforms
- Exposure to AWS services and data governance tooling, such as Lake Formation
- Computer Science degree or equivalent, strong foundational background in software engineering
Bayside Solutions, Inc. is not able to sponsor any candidates at this time. Additionally, candidates for this position must qualify as Inc.'s candidate.
Bayside Solutions, Inc. may collect your personal information during the position application process. Please reference Bayside Solutions, Inc.'s CCPA Privacy Policy at
Never miss an opportunity! Create an alert based on the job you applied for.