Spark/Snowflake Developer

Job Title: Spark/Kafka Lead Developer

Location: Houston, Texas

Job Type: Permanent

Job Description

Required Qualifications:

At least 4 years of Information Technology experience.

Experience in Hadoop ecosystem & Apache Spark.

Experience in Kafka streaming.

Solid understanding and hands-on experience in SQL, Unix shell scripting

Preferred Qualifications:

Proficiency in programming languages such as Scala, Java, or Python

At least 4 years of experience in Apache Spark, including Spark Core, Spark SQL, Spark Streaming, and Spark MLlib

Lead the development and optimization of batch and real-time data pipelines, ensuring scalability, reliability, and performance.

Architect, design, and deploy data integration, streaming, and analytics solutions leveraging Spark, Kafka, and Snowflake.

Extensive hands-on experience with Apache Kafka, including topics, partitions, producers, consumers, and stream processing

Solid understanding of distributed computing concepts, data partitioning, fault tolerance, and parallel processing

Experience with cluster management tools like Apache Mesos or Apache YARN is desirable

Good understanding of data integration, data quality and data architecture

Experience in Relational Modeling, Dimensional Modeling and Modeling of Unstructured Data

Good understanding of Agile software development frameworks

Strong communication and Analytical skills

Ability to work in teams in a diverse, multi-stakeholder environment comprising of Business and Technology teams

Experience and desire to work in a global delivery environment