We're looking for a hands-on Data Engineer Lead to support an important Phase 0 discovery effort for contact identity, golden record, and marketing data model readiness. This role is ideal for someone who enjoys digging into complex data landscapes, partnering with cross-functional teams, and turning discovery findings into clear engineering handoff materials for Databricks build work.
Responsibilities
- Inventory priority source systems, data entities, identifiers, fields, relationships, and known data quality concerns.
- Review source documentation, sample data structures, field inventories, and SME input to understand the current data landscape.
- Assess data completeness, duplicates, freshness, conflicts, availability, lineage, and source reliability.
- Support source-to-target mapping for a canonical marketing data model.
- Translate identity rules, survivorship logic, source hierarchy, field ownership, account matching, and exception handling into engineering-ready inputs.
- Document data gaps, dependencies, assumptions, risks, and open questions for the internal engineering team.
- Help define acceptance criteria, data quality thresholds, and match-confidence considerations for the final handoff package.
- Partner with strategy and architecture stakeholders to ensure outputs are actionable and ready for Databricks implementation.
Skills
- 5+ years of experience in data engineering, data platforms, or a related field.
- Strong background in data discovery, profiling, mapping, and engineering handoff work.
- Hands-on experience with Python and PySpark.
- Strong Databricks experience.
- Experience with SQL and NoSQL databases.
- Experience working in enterprise data environments.
- Knowledge of ETL/ELT processes, data integration, and data modeling.
- Hands-on experience with AWS or another major cloud platform.
- Strong analytical, problem-solving, and communication skills.
Preferred Skills
- Experience with cloud data tools such as AWS Glue or Azure Data Factory.
- Familiarity with modern data architecture principles, including data lakes or data mesh.
- Exposure to governance, security, and compliance practices in data environments.
- Experience with performance optimization and scalable data solutions.
- Certifications in cloud data technologies are a plus.
- Ability to work collaboratively in a hybrid, fast-moving environment.
Horizontal Talent is committed to creating an inclusive environment where diverse backgrounds, perspectives, and experiences are valued. We encourage candidates from all communities to apply and bring their unique strengths to the team.
By applying for this position, you acknowledge and agree that Horizontal Talent may contact you regarding your application using automated technology, including phone calls, SMS/text messages, or email, which may be delivered by our virtual AI recruiter, Alex.
Please apply through this online posting or by visiting our Job Board at Applications will be accepted for 4 weeks. For those that join the team, we offer competitive compensation and benefits including medical, dental, vision, and retirement. Check out all we have to offer and how you can become part of the Horizontal Talent Team. The pay range for this role is $51 - $89 per hour based on qualifications and experience.
Never miss an opportunity! Create an alert based on the job you applied for.