Overview
On Site
$120,000 - $140,000
Full Time
Skills
Data Engineering
pyspark
python
azure
spark
Job Details
Job Title: Data Engineer
Missisauga,Canada
Fulltime
Need to complete Certification before onboarding
Mandatory Skills: Python, Spark
Pyspark Must
Responsibilities:
- Design, develop, and maintain software solutions using Python.
- Developed Python and pyspark programs for data analysis.. Good working experience with python to develop Custom Framework for generating of rules (just like rules engine).
- Developed Python code to gather the data from HBase and designs the solution to implement using Pyspark. Apache Spark DataFrames/RDD's were used to apply business transformations and utilized Hive Context objects to perform read/write operations..
- Develop back-end services, APIs, and integrate databases.
- Optimize applications for performance, security, and maintainability.
- Troubleshoot and resolve software defects and issues.
Required Skills:
- Strong proficiency in Python and Spark.
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.