Overview
On Site
Depends on Experience
Contract - W2
Contract - 24 Month(s)
No Travel Required
Skills
Data Engineer
azure
Job Details
Role: Senior Data Engineer
Location: Mississauga ON Canada Client: Synechron
Major Responsibilities
- Building and maintaining data pipelines to ingest data from source systems into Databricks
- Designing, Building and maintaining generic frameworks to ingest data through the medallion layers
- Working with the core data platform team to integrate the framework with reference architecture
- Assisting the data mesh teams with best practices and onboarding to the new framework
Requirements
- 10+ years of data engineering experience in building OLTP and OLAP databases, both on-prem and in the cloud
- 5+ years of experience with large scale data processing using SQL, Python, Pandas and Pyspark
- 5+ years of experience building robust and scalable data pipelines in Databricks Lakehouse platform on any key cloud platform provider, preferably Azure cloud
- Experience with implementing Medallion architecture for data processing
- Experience with implementing Streaming data ingestion using Kafka
- Test driven development experience
- Proven experience in solution design and applying best design practices
- Data modelling and ER/UML diagram skills
- Experience with orchestration/enterprise scheduler platforms like Airflow, Tidal etc..
- Experience with implementing API based data integration pipelines
- Experience with source control and DevOps for CI/CD
- Experience working in Agile/scrum methodologies
- Experience with Jira, confluence for project work and documentation
- Experience with Service now for incident management
- Finance domain knowledge in Investment Banking (Securities, Capital markets) and Security reference data knowledge
- Nice to have ADF and SSIS experience
- College degree required.
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.