AWS Data Engineer - PySpark / Data AutomationLocation: Remote within the U.S.; Eastern or Central time zones preferred
U.S. Citizenship Required - Ability to obtain Public Trust Clearance Pay: $55.00-$60.00 per hour
26-08455
Job SummaryThe AWS Data Engineer designs, develops, and supports scalable cloud-based data pipelines and automation solutions using Python, PySpark, SQL, and AWS data services. This role focuses on building and troubleshooting production data workflows, integrating batch and streaming data sources, and optimizing data processing performance. The position also supports secure cloud environments, CI/CD practices, production operations, and emerging AI-enabled data solutions.
Job Responsibilities - Design, develop, test, and support scalable Python and PySpark data pipelines.
- Build ETL/ELT workflows for large-scale structured and unstructured datasets.
- Develop batch and near-real-time data ingestion pipelines using Apache Spark and PySpark.
- Integrate streaming technologies such as Kafka, Flume, or similar platforms.
- Develop solutions using AWS services such as S3, Glue, Lambda, EMR, Step Functions, and AWS database technologies.
- Develop and optimize complex SQL-based data transformations and queries.
- Integrate REST APIs and external data sources into automated data workflows.
- Troubleshoot and optimize Spark jobs, SQL queries, data pipelines, and production workloads.
- Support data reconciliation, performance monitoring, incident resolution, and root-cause analysis.
- Apply source-control and CI/CD practices using GitHub, Azure DevOps, Jenkins, or similar tools.
- Support secure cloud environments using IAM, encryption, VPC controls, and role-based access.
- Participate in Agile and DevOps delivery practices and production support activities.
- Support AI and GenAI-enabled data solutions using platforms such as Amazon Bedrock, Azure OpenAI, or other LLM services.
- Collaborate with technical and business teams to develop reliable, scalable, and maintainable data solutions.
Job Requirements - Minimum of 5 years of experience in data engineering, data automation, or a related field.
- Demonstrated hands-on experience building and supporting production Apache Spark/PySpark data pipelines.
- Strong hands-on Python and SQL skills.
- Experience building and supporting production data pipelines.
- Experience with AWS-based data engineering environments.
- Experience with multiple AWS services, including S3, Glue, Lambda, EMR, Step Functions, or AWS databases.
- Experience with ETL/ELT and data pipeline development.
- Experience with Kafka, Flume, or similar streaming and ingestion technologies.
- Experience integrating REST APIs and external data sources.
- Experience troubleshooting production data platforms, including performance tuning, incident resolution, reconciliation, and root-cause analysis.
- Experience with source control and CI/CD tools such as GitHub, Azure DevOps, Jenkins, or similar platforms.
- Bachelor\'s degree in Computer Science or a related technical field.
- U.S. citizenship required.
- Ability to obtain and maintain a Federal Public Trust.
- Ability to work remotely within the U.S.
- Ability to travel occasionally or quarterly to the Gaithersburg, MD/Washington, DC area as required.
Preferred Qualifications - Experience working in AWS and Azure multi-cloud environments.
- Experience with Amazon Bedrock, Azure OpenAI, LangChain, or other GenAI/LLM technologies.
- Experience with RAG, embeddings, vector search, Solr, or OpenSearch.
- Experience with Docker and Kubernetes.
- Experience with Terraform or other infrastructure-as-code tools.
- AWS or Azure certifications.
Additional Job Details (Commute, Attendance, etc.) - Fully remote position within the U.S.
- Eastern or Central time zone candidates preferred.
- Occasional or quarterly travel to the Gaithersburg, MD/Washington, DC area may be required.
- Federal Public Trust eligibility is required before starting the assignment.
#ED
Future Communication Consent:
By applying, you are providing consent for Equiliem to engage with you via phone calls, AI-generated calls, AI-generated text messages, standard text messages, and/or emails to share job opportunities. Consistent with our Privacy Policy, information obtained will not be shared with third parties for marketing or promotional purposes. Message frequency may vary for text messages. Message and data rates may apply. Carriers are not liable for delayed or undelivered messages. You may reply STOP to cancel or HELP for assistance.
EEO Employer:
Equiliem is an equal opportunity employer. We do not discriminate or allow discrimination based on race, color, religion, creed, sex, age, sexual orientation, gender identity, national origin, ancestry, citizenship, genetic information, registered domestic partner status, marital status, disability, status as a crime victim, protected veteran status, political affiliation, union membership, or any other characteristic protected by law. Equiliem will consider qualified applicants with criminal histories in a manner consistent with the requirements of applicable law. If you have visited our website in search of information on employment opportunities or to apply for a position, and you require an accommodation in using our website for a search or application, please contact our Employee Services Department at or .
Pandologic. Keywords: Data Automation Engineer, Location: Washington, DC - 20402
.png?format=webp)
Never miss an opportunity! Create an alert based on the job you applied for.
