About the Role:
We're looking for a Senior Data Engineer to design, build, and maintain reliable data pipelines that power analytics and AI-driven insights. You'll work with Spark and Snowflake to move and transform data at scale, and use AI tools to build visualizations and dashboards that make that data useful to the business.
Key Responsibilities:
- Pipeline Development: Design, build, and maintain scalable ETL/ELT pipelines for batch and, where needed, near-real-time data.
- Data Processing: Use Spark to transform and process large datasets efficiently.
- Data Warehousing: Model, load, and optimize data in Snowflake for performance and cost.
- Data Quality & Reliability: Implement validation, monitoring, and error handling to keep pipelines accurate and dependable.
- AI-Powered Visualization: Use AI tools to build dashboards and reports that communicate insights clearly.
- AI Enablement: Apply AI agents, MCP (Model Context Protocol), and LLM-based tools to automate workflows and speed up data tasks.
- Documentation: Document pipelines, data lineage, and transformations.
- Collaboration: Work with analysts, product managers, and business stakeholders to turn requirements into data solutions.
Required Qualifications:
- 5–8 years of experience in data engineering, including ETL and data pipeline development
- Working knowledge of Apache Spark
- Hands-on experience with Snowflake
- Strong SQL skills
- Basic Python for data processing and scripting
- Experience using AI tools to create visualizations and dashboards
- Awareness of AI concepts, including agents, MCP, and LLM-based tools
Preferred:
- Orchestration tools (e.g., Airflow)
- Cloud platform experience (AWS, Azure, or Google Cloud Platform)
- Git and CI/CD practices
- Familiarity with data governance and cataloging
Tech Stack:
Spark · Snowflake · SQL · Python · AI visualization tools · AI agents/MCP
