Job Title: Sr. Data Integration Engineer - Consultant
Duration - 6+ months
Location: Sunnyvlae, CA (3 Days onsite)
Top Skills:
Snowflake, Python data integrations, Apache Airflow, Snowflake Stremilit Apps, Snowflake Cortex AI.
Detailed Job description:
Need results-oriented Data Integration Engineer with a strong background in Apache Airflow, Snowflake, Python, and SAP replication using HVR
The ideal candidate will have a strong foundation in Python programming, experience with Snowflake for data warehousing, proficiency in AWS and Kubernetes (EKS) for cloud services management, and expertise in CI/CD practices, Apache Airflow, DBT, and API development. Experience with SAP data replication using HVR is highly desirable, as it plays a critical role in our enterprise data strategy. This role is essential to enhancing our data integration capabilities and supporting our data-driven initiatives.
Role and Responsibilities
As the Technical Data Integration Engineer, you will play a pivotal role in shaping the future of our data integration engineering initiatives. You will be part of a talented team of data integration engineers while remaining actively involved in the technical aspects of the projects. Your responsibilities will include:
- Hands-On Contribution: Continue to be hands-on with data integration engineering tasks, including data pipeline development, EL processes, and data integration. Be the go-to expert for complex technical challenges.
- Integrations Architecture: Design and implement scalable and efficient data integration architectures that meet business requirements. Ensure data integrity, quality, scalability, and security throughout the pipeline.
- Tool Proficiency: Leverage your expertise in Snowflake, SQL, Apache Airflow, AWS, API, Python, and HVR for SAP replication to architect, develop, and optimize data solutions. Stay current with emerging technologies and industry best practices.
- Data Quality: Monitor data quality and integrity, implementing data governance policies as needed.
- Cross-Functional Collaboration: Collaborate with data science, data warehousing, analytics, and other cross-functional teams to understand data requirements and deliver actionable insights.
- Performance Optimization: Identify and address performance bottlenecks within the data infrastructure. Optimize data pipelines for speed, reliability, and efficiency.
Qualifications
- Minimum bachelor’s degree in computer science, Engineering, or related field. Advanced degree is a plus.
- 5+ years of hands-on experience in data engineering.
- Familiarity with cloud platforms, such as AWS or Azure.
- Expertise in Apache Airflow, Snowflake, SQL, Python, Shell scripting, API gateways, web services setup.
- Experience with SAP data replication using HVR or similar tools.
- Strong experience in full-stack development, AWS, Linux administration, data lake construction, data quality assurance, and integration metrics.
- Excellent analytical, problem-solving, and decision-making abilities.
- Strong communication skills, with the ability to articulate technical concepts to non-technical stakeholders.
- A collaborative mindset, with a focus on team success.
