Position Title: System Development Engineer
Location: Seattle WA 98101 – Onsite
Job Description:
As a member of the Region Flexibility Migration (RFM) team, you will be responsible for driving Amazon's DRAM memory optimization initiative focused on reducing the memory footprint of EMR on EC2 clusters through infrastructure-level tuning, cluster configuration optimization, and instance family right-sizing.
You will analyze cluster metrics, evaluate Spark runtime configurations, and recommend optimal cluster topologies that minimize DRAM consumption while preserving job performance and reliability.
This is a high-impact, independently driven role where you will assess EMR cluster utilization patterns, identify over-provisioned resources, and implement configuration-level optimizations, including instance type migrations (e.g., R-family to C-family), Spark executor tuning, YARN resource allocation adjustments, and dynamic scaling configurations. You will collaborate with service teams and RFM engineering leadership to validate changes and establish reusable frameworks for fleet-wide EMR optimization.
Key Responsibilities
• Analyze EMR cluster metrics, Spark application telemetry, and YARN resource utilization to identify over-provisioned memory allocations, underutilized executors, and suboptimal cluster configurations that contribute to excessive DRAM consumption.
• Recommend and implement cluster-level optimizations, including instance family right-sizing (e.g., migrating from memory-optimized R-type to compute-optimized C-type instances), node count adjustments, EBS volume configurations, and spot/on-demand fleet composition changes.
• Tune Spark runtime configurations at the cluster level, including executor memory/core ratios, YARN container sizing, dynamic resource allocation settings, memory overhead parameters, and shuffle service configurations, to achieve optimal memory utilization without impacting job SLAs.
• Perform custom operations and iterative experiments using Amazon internal tooling to validate optimization impact: own end-to-end deployment, test execution, metric validation, and derive actionable insights from results.
• Collaborate with service teams to review cluster architectures, discuss findings, propose optimization plans, and align resolution strategies while communicating effectively across engineering leadership and technical stakeholders.
• Monitor service health metrics and troubleshoot operational issues during and after optimization activities, ensuring zero degradation to job completion times, data processing throughput, and downstream SLAs.
• Develop comprehensive operational runbooks, SOPs, documentation, and technical specifications that capture cluster optimization patterns and can be consumed by both human engineers and AI agents to orchestrate optimization workflows at scale.
• Extract scalable learnings from optimization engagements and develop programmatic frameworks that enable the initiative to scale across hundreds of EMR clusters, including training and enabling other vendor engineers to execute optimization playbooks.
Required Qualifications:
• Bachelor's degree in computer science, Engineering, or equivalent technical field.
• 5-7+ years of hands-on experience operating, configuring, and tuning EMR clusters in large-scale production environments, with demonstrable results in cost or resource optimization.
• Deep expertise in the Spark/YARN ecosystem operations, including cluster re-sizing and instance type selection for memory optimization, YARN resource manager configuration, Spark memory model (executor memory, memory overhead, off heap), and dynamic allocation tuning.
• Strong knowledge of AWS EC2 instance families and their performance characteristics, with the ability to evaluate trade-offs between memory-optimized, compute-optimized, and general-purpose instances for data processing workloads.
• Proficiency in interpreting CloudWatch metrics, EMR cluster logs, Spark History Server data, and YARN resource manager metrics to perform data-driven capacity analysis and make right-sizing recommendations.
• Proficiency in using generative AI tools and assistants as part of daily engineering workflows to accelerate problem-solving, analysis, and technical documentation.
Preferred Qualifications:
• Direct experience with EMR fleet management at scale (50+ clusters), including automated scaling policies, instance fleet configurations, and cost optimization strategies.
• Experience with Amazon internal tools including Amazon Profiler, CloudWatch, X-Ray, and load/stress testing frameworks for validating cluster optimization impact under production-like conditions.
• Understanding Spark workload characteristics (batch, streaming, interactive) and ability to recommend cluster configurations tailored to each workload type for optimal memory utilization.
• Experience developing AI agents or automated workflows that can orchestrate tasks, extract information from services, and coordinate optimization activities across multiple systems.
• Strong attention to detail and effective communication abilities: able to present findings, propose strategies, and influence service team stakeholders and engineering leadership.
• Familiarity with CI/CD pipelines, deployment automation, and Amazon deployment technologies and best practices for infrastructure changes.
Never miss an opportunity! Create an alert based on the job you applied for.
