ClifyX group is an award winning IT Consultancy formed in 1998. Our Mission is to provide our clients with Optimal Technology solutions that are effective and within budgets. We specialize in helping Organizations to review their strategic SOW Projects/Talent needs and implement high value and cost effective solutions to increase profitability and efficiency. Our consulting capabilities include expertise in Cloud, Artificial Intelligence, Data Analytics and compliance aspects of Cyber Security development.
Work With Us : What makes working at ClifyX so great? We give tons of flexibilty, listen to you, give opportunity to work in different technologies, give feedback on your profile and present you to global fortune companies.
Job Title: Engagement Manager (PySpark Data and Solution Architect)
Location: Flexible within North East Region, preferable within Baltimore, MD region
Job Description:
Job Description:
PySpark Lead with 7+ years of experience, to drive the architecture, design development and optimization of big data processing solutions in the Asset and Wealth Management Industry Segment. The ideal candidate will have deep expertise in Apache Spark, Python, and distributed data processing while leading a project team of data analysts, business analysts/testers, domain consultants and engineers to deliver scalable data solutions.
Lead the end to end lifecycle covering architecture design, development, and deployment PySpark-based big data solutions.
Collaborate with and manage client project leadership, and key workstream leads to drive the solutions, strategize and navigate delivery and manage delivery outcomes
Architect and optimize ETL pipelines for structured and unstructured data.
Collaborate with Client, data engineers, data scientists, and business teams to understand requirements and provide scalable solutions.
Optimize Spark performance through partitioning, caching, and tuning.
Implement best practices in data engineering (CI/CD, version control, unit testing).
Work with cloud platforms like AWS, Azure
Ensure data security, governance, and compliance.
Mentor junior team members and developers and review code for best practices and efficiency.
Qualifications:
7+ years of experience in big data and distributed computing.
Strong hands-on experience with PySpark, Apache Spark, and Python.
Experience with SQL and NoSQL databases (DB2, PostgreSQL, Snowflake, etc.).
Proficiency in data modeling and ETL workflows.
Familiarity with workflow schedulers like Airflow
Experience with cloud-based data platforms (AWS, Azure, Google Cloud Platform).
Knowledge of DevOps, CI/CD pipelines, and containerization (Docker, Kubernetes) is a plus.
Strong problem-solving skills and ability to lead a team.
Industry experience in Wealth and Asset Management
We are an Equal Opportunity/Affirmative Action employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, national origin, gender identity, sexual orientation, disability status, protected veteran status, or any other characteristic protected by law.
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.