Senior System Engineer (Java, Python, and PySpark)

Overview

Remote
Depends on Experience
Full Time
Able to Provide Sponsorship

Skills

Amazon Web Services
Ansible
Apache Hadoop
Apache Hive
Apache Kafka
Apache Spark
Big Data
Cloud Computing
Data Engineering
Data Processing
Extract
Transform
Load
HDFS
PySpark
Linux
Python
Java
System Monitoring
Software Development
Systems Architecture
Systems Engineering
Unix

Job Details

We are seeking a highly skilled System Engineer with strong experience in Java, Python, and PySpark to design, develop, and optimize large-scale data and application systems. The ideal candidate will have a solid background in system architecture, software development, and data engineering, along with hands-on experience integrating distributed systems and ensuring performance and reliability.

Key Responsibilities:

  • Design, develop, and maintain system components and data pipelines using Java, Python, and PySpark.

  • Collaborate with cross-functional teams to implement scalable and resilient solutions in cloud or on-premise environments.

  • Develop and maintain ETL processes for data ingestion, transformation, and loading across multiple data sources.

  • Optimize and troubleshoot distributed applications for performance and reliability.

  • Implement system monitoring, logging, and alerting to ensure high availability and system integrity.

  • Automate deployment and configuration management using tools such as Ansible, Jenkins, or Airflow.

  • Participate in code reviews, contribute to technical documentation, and follow DevOps and CI/CD best practices.

  • Work with Big Data ecosystems (Hadoop, Spark, Hive, Kafka, etc.) to handle large-scale data processing.

  • Analyze and resolve complex technical issues across software, infrastructure, and data layers.

Required Skills and Qualifications:

  • Bachelor s or Master s degree in Computer Science, Information Technology, or a related field.

  • 10+ years of experience in system engineering, software development, or data engineering.

  • Strong programming experience in Java and Python.

  • Expertise in PySpark for distributed data processing and transformation.

  • Hands-on experience with Hadoop ecosystem components such as Spark, Hive, HDFS, and Kafka.

  • Solid understanding of Linux/Unix systems, shell scripting, and system-level debugging.

  • Experience with version control systems (Git, Bitbucket) and CI/CD pipelines (Jenkins, GitLab CI).

  • Familiarity with cloud platforms (AWS, Azure, or Google Cloud Platform) and data orchestration tools (Airflow, Oozie).

  • Strong analytical and problem-solving skills, with a focus on scalability and performance tuning.

Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.

About Divit Technologies, Inc.