Overview
On Site
Accepts corp to corp applications
Contract - 12th Month(s)
Skills
pyspark
AWS Data Engineer
Job Details
Job Title:- AWS Engineer with Databricks
Location:- Malvern, PA
Job Type:- Contract
On-Site
Job Description:-
Key Responsibilities:
- Design, build, and optimize scalable data pipelines and ETL workflows using PySpark.
- Develop and maintain data architectures on AWS, leveraging services like S3, Redshift, Glue, EMR, Lambda, and Athena.
- Collaborate with data analysts and stakeholders to translate business requirements into technical solutions.
- Ensure the quality, security, and reliability of data pipelines and storage.
- Implement data governance practices, including cataloging and metadata management.
- Monitor and troubleshoot production pipelines to ensure high availability and performance.
- Participate in code reviews, testing, and deployment processes to ensure robust solutions.
Qualifications and Skills:
- Bachelor's or Master's degree in Computer Science, Information Systems, or a related field.
- 4+ years of experience as a Data Engineer, with a strong focus on AWS cloud services.
- Proficiency in PySpark for large-scale data processing.
- Experience with AWS services such as Glue, EMR, Redshift, Lambda, and S3.
- Strong knowledge of SQL and relational databases.
- Familiarity with data warehousing concepts and design.
- Hands-on experience with CI/CD tools and practices.
- Knowledge of data lake architectures and data governance.
- Strong problem-solving skills and ability to work in a fast-paced environment.
Preferred Skills:
- Experience with containerization and orchestration tools like Docker and Kubernetes.
- Familiarity with streaming data platforms like Kafka or Kinesis.
- Knowledge of Python and Java for building data solutions.
- AWS certifications such as AWS Certified Data Analytics - Specialty or AWS Certified Solutions Architect.
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.