Sr. Python/Pyspark Developer

The Role:

Our Client is seeking a Sr. Python/Pyspark Developer to join our team in Mississauga, ON (Need Onsite day 1, hybrid 3 days from office).

JR ID: JR1027328

C2C Pay Rate: CAD 640 CAD 680

Python, Pyspark, SQL, Data Engineering.

Note: This role is heavy on data engineering and not app dev

Job Description:

As a senior Python and PySpark developer, your challenge is to architect and deliver scalable, high-performance data processing solutions that empower data-driven decision-making. You will tackle complex data integration, transformation, and analytics workloads by designing robust applications, optimizing SQL queries, and streamlining workflows. Your expertise will enable the organization to leverage cloud-scale data platforms effectively, maintain high code quality through rigorous testing, and foster collaboration within agile teams to innovate and maintain cutting-edge data solutions.

Responsibilities:

• Design, develop, and maintain robust Python and PySpark applications to support large-scale data processing, transformation, and analytics initiatives.
• Implement and execute unit testing and code quality checks using tools like Pytest and Pylint to ensure reliable and maintainable codebases.
• Design, optimize, and troubleshoot complex SQL queries, primarily in tSQL, to support data extraction, transformation, and loading processes.
• Develop and manage data integration workflows utilizing ETL/ELT frameworks, enterprise schedulers, and scripting in Python and other relevant tools.
• Collaborate with cross-functional teams to gather requirements, create high-level system specifications, and produce detailed design documents for data solutions.
• Implement RESTful API integrations to connect various data sources and applications, ensuring seamless data flow and accessibility.
• Lead and contribute to agile development processes, participating in sprint planning, stand-ups, and code reviews to ensure timely delivery of features.
• Leverage frameworks such as Django, Flask, Pyramid, or Tornado to build scalable web applications or APIs as needed.
• Utilize version control systems like Git, and manage development pipelines to support continuous integration and deployment.
• Apply best practices in software architecture, data structures, and algorithms to optimize application performance and scalability.
• Coordinate with enterprise teams on scheduling and automation tasks using enterprise schedulers and SSIS, enhancing operational efficiency.
• Lead technical discussions and influence stakeholders to adopt best practices and innovative solutions for data processing and application development.
• Maintain thorough documentation of system architecture, design specifications, and coding standards to support ongoing maintenance and knowledge sharing.
• Stay updated on emerging technologies and industry trends to recommend improvements and ensure the solutions remain cutting-edge.

Requirements:

• At least 10 years of Experience in Python development
• Must have hands-on experience in Python and Pyspark
• Experience in Python testing and code analysis tools like Pytest and Pylint
• At least 5 years of fulltime development experience using Python.
• At least 5 years working with tSQL, XML and enterprise schedulers
• Proficient understanding of algorithms, data structures, architectural design patterns and best practices
• Extensive experience in software development and the entire SDLC.
• Solid understanding of a variety of programming tools and development platforms.
• Excellent organizational and leadership abilities.
• Highly analytical mindset, with an ability to see both the big picture and the details.
• Experience in creating high level product specifications and design documents.
• Proven collaboration and influencing skills
• Experience in writing Python applications with the benefit of frameworks like Django, Flask,
• Pyramid, or Tornado.
• Integration experience (ETL, ELT) with Python
• Familiarity with SSIS would be helpful
• General development expertise, use of version control, ticketing, and continuous integration systems.
• Experience in an Agile Development environment
• Experience in consuming API's (i.e. Rest, etc.)

Preferred, but not required:

• Detail-oriented, ability to turn deliverables around quickly with a high degree of accuracy
• Strong analytical skills, ability to interpret business requirements and produce functional and technical design documents.
• Good time management skills Ability to prioritize and multi-task, handling multiple efforts at once.
• Strong desire to understand and learn domain.
• Experience in a financial services/banking industry
• Ability to work in a fast-paced environment; to be flexible and learn quickly.
• Ability to multi-task with attention to detail/ prioritize tasks.