Overview
On Site
$60 - $65
Accepts corp to corp applications
Contract - W2
Contract - Independent
Contract - 12 Month(s)
Skills
pyspark
docker
azure
sql
azure data factory
Job Details
Role: PySpark Developer
Location: Iselin NJ
Need Passport Number
Face to Face Interview Required
Responsibilities:
- Develop, maintain, and optimize data processing pipelines using PySpark on Azure Cloud.
- Collaborate with cross-functional teams to understand data requirements and translate business needs into technical specifications.
- Design and implement scalable data models and data architecture to support analytical and reporting needs.
- Perform data cleansing, transformation, and enrichment processes to enhance data quality and usability.
- Monitor and troubleshoot data processing jobs, ensuring high availability and performance.
- Implement best practices for data governance and security within Azure Cloud.
- Stay up-to-date with emerging technologies in big data, cloud computing, and the financial services industry.
Requirements:
- Minimum of 9 years of experience in PySpark development.
- Proven experience working with Azure Cloud services (Azure Data Factory, Azure Databricks, etc.).
- Prior experience in the banking, investment banking, or financial services sector is mandatory.
- Proficient in PySpark for data processing and ETL tasks.
- Experience with SQL and NoSQL databases.
- Familiarity with data warehousing concepts and tools.
- Knowledge of Azure services related to big data and analytics.
Preferred, but not required:
- Experience with containerization technologies (e.g., Docker, Kubernetes).
- Familiarity with machine learning frameworks (e.g., MLlib, TensorFlow).
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.