Overview
Remote
$100,000+
Full Time
Able to Provide Sponsorship
Skills
Ansible
Data Engineering
Java
Information Technology
Python
PySpark
Shell Scripting
Software Development
Data Processing
Job Details
Key Responsibilities:
Required Skills and Qualifications:
- Design, develop, and maintain system components and data pipelines using Java, Python, and PySpark.
- Collaborate with cross-functional teams to implement scalable and resilient solutions in cloud or on-premise environments.
- Develop and maintain ETL processes for data ingestion, transformation, and loading across multiple data sources.
- Optimize and troubleshoot distributed applications for performance and reliability.
- Implement system monitoring, logging, and alerting to ensure high availability and system integrity.
- Automate deployment and configuration management using tools such as Ansible, Jenkins, or Airflow.
- Participate in code reviews, contribute to technical documentation, and follow DevOps and CI/CD best practices.
- Work with Big Data ecosystems (Hadoop, Spark, Hive, Kafka, etc.) to handle large-scale data processing.
- Analyze and resolve complex technical issues across software, infrastructure, and data layers.
Required Skills and Qualifications:
- Bachelor s or Master s degree in Computer Science, Information Technology, or a related field.
- 10+ years of experience in system engineering, software development, or data engineering.
- Strong programming experience in Java and Python.
- Expertise in PySpark for distributed data processing and transformation.
- Hands-on experience with Hadoop ecosystem components such as Spark, Hive, HDFS, and Kafka.
- Solid understanding of Linux/Unix systems, shell scripting, and system-level debugging.
- Experience with version control systems (Git, Bitbucket) and CI/CD pipelines (Jenkins, GitLab CI).
- Familiarity with cloud platforms (AWS, Azure, or Google Cloud Platform) and data orchestration tools (Airflow, Oozie)
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.