Senior Data Engineer

Role Summary

We re seeking an onshore Data Engineer/ Senior Systems Analyst to lead the productionalization of data ingestion and curation on Databricks. You will design the frameworks, controls, and processes that make data movement reliable and repeatable, create clear documentation, and guide an offshore team responsible for daytoday build and run. You ll also help assess when to use Fivetran vs. native Databricks/Unity Catalog patterns for specific use cases.

What You ll Do (Key Responsibilities)

• Design data movement frameworks: Define reusable patterns for source onboarding, change data capture (as applicable), schema evolution, error handling, and operational runbooks.
• Implement data quality & reconciliation controls: Define sourcetotarget parity checks, row/columnlevel validations, sampling strategies, reconciliation dashboards, and exception workflows; partner with engineers to embed controls in pipelines.
• Create clear documentation: Author process docs, onboarding guides, data flow diagrams, mapping specifications, and operational SOPs to ensure consistent understanding across teams.
• Evaluate tooling choices: Recommend where Fivetran connectors make sense (e.g., rapid replication/archival) vs. when to leverage Databricks ingestion, Delta, and Catalog capabilities.
• Coordinate offshore delivery: Provide requirements, review designs, groom tasks, and ensure adherence to standards; mentor engineers and unblock execution.
• Stakeholder alignment: Work directly with product/architecture leads to confirm scope, prioritize sources, and communicate status, risks, and tradeoffs.

MustHave Qualifications

• 8+ years in data engineering / data platforms with strong ETL/ELT fundamentals and data movement patterns (batch/nearreal time).
• Handson Databricks experience (workspace organization, notebooks/jobs, Delta Lake fundamentals, performance/optimization basics).
• Proven experience designing validation and reconciliation controls (sourcetotarget checks, DQ rules, automated reporting).
• Demonstrated ability to author highquality documentation (frameworks, standards, SOPs, mapping/specs).
• Experience leading or enabling offshore teams (backlog grooming, code/design reviews, quality gates, knowledge transfer).
• Strong communication and stakeholder management; able to translate business needs into actionable technical work for engineering teams.

NicetoHave

• Exposure to Fivetran (connector configuration, load strategies, monitoring) not required but helpful.
• Familiarity with Unity Catalog/Databricks Catalog, Delta Live Tables, or other Databricks-native ingestion/orchestration approaches.
• Experience with Azure data services (e.g., ADLS, ADF/Synapse triggers/pipelines) in support of Databricks.
• Insurance data domain familiarity (e.g., policy, claims, billing) and legacy source onboarding.
• SQL proficiency for profiling, validation queries, and investigative analysis.