Spark Technical Engineer

spark, pyspark, kafka, aws, azure, java, scala
Full Time
$140,000 - $160,000
Work from home available Travel required to 10%.

Job Description

Spark Technical Solutions Engineer

Fulltime role


Either San Francisco or Dallas, TX.

To relocate after the pandemic. Until then remote work is fine


Key skillset - Spark Development and troubleshooting



• Troubleshoot and resolve complex customer issues related to spark core internals, spark sql, structured streaming, delta and other data engineering and product areas.

• Provide best practices guidance for custom-built solutions developed by customers

• Provide suggestions for improving efficiency in customer-specific environments

• Help with issues around spark/third-party integrations with environment

• Demonstrate true and coordinate with engineering and escalation teams to achieve resolution of customer issues and requests

• Help create company documentation and knowledge articles

• Be a true proponent of customer advocacy

• Strengthen your AWS/Azure and Databricks platform expertise through learning and internal training programs

• Value collaboration and the importance of supporting your team and coach other team members

• Participate in weekend and weekday on call rotation


• Minimum 5 years’ experience developing, testing, and sustaining Python or Java or Scala-based applications. Expert level knowledge in python/scala is desired.

• Experience in Big Data/Hadoop/Spark/Kafka/Elasticsearch data pipelines.

• Solid experience with SQL-based database systems is preferred.

• Linux/Unix administration skills is preferred.

• Skills in Linux JVM, Thread dump-based troubleshooting preferred.

• Experience with AWS or Azure related services is preferred.

• You have excellent written and oral communication skills.

• Good technical skills in Programming, data structures and algorithms is preferred.








• Medical, dental, vision

• 401k Retirement Plan

• Unlimited Paid Time Off

• Catered lunch (everyday), snacks, and drinks

• Gym reimbursement

• Employee referral bonus program

• Awesome coworkers

• Maternity and paternity plans

Dice Id : swapps
Position Id : 7265498
Originally Posted : 2 weeks ago
Have a Job? Post it