Senior Real-Time Linux Kernel Engineer (C++, Kernel testing)

  • San Diego, CA
  • Posted 3 days ago | Updated 3 days ago

Overview

On Site
$55 - $70
Contract - W2
Contract - 12 Month(s)

Skills

Linux kernel
C++ programming
CPU

Job Details

Position: Senior Real-Time Linux Kernel Engineer

Position Overview

We are seeking a highly skilled Senior Engineer to architect and build a comprehensive testing framework for our real-time kernel on x86 server platforms. This critical role requires deep expertise in real-time operating systems, x86 server architecture, and the unique challenges of validating real-time constraints in high-performance server environments. You will create testing infrastructure that can verify deterministic behavior, timing guarantees, and system reliability while also contributing to kernel debugging and optimization of critical system resources.

Key Responsibilities

Primary Responsibility - Real-Time Kernel Testing Framework Development

  • Design and implement a specialized testing framework for real-time kernel validation on x86 server platforms
  • Develop test methodologies to verify real-time constraints, scheduling deadlines, and deterministic behavior under high load
  • Build automated tests for kernel subsystems including scheduler, memory management, interrupt handling, and IPC mechanisms
  • Create stress testing frameworks to validate kernel stability under extreme throughput conditions
  • Implement latency measurement and timing analysis tools with microsecond/nanosecond precision
  • Design framework to test CPU isolation, real-time throttling, and scheduling latency
  • Implement performance regression testing for throughput and latency metrics

Kernel Debugging and Issue Analysis

  • Debug complex kernel-level issues including race conditions, memory corruption, and timing violations using debugging tools such as ftrace, perf, eBPF
  • Analyze kernel panic dumps, trace logs, and system lockups in multi-core environments
  • Investigate interrupt latency problems, scheduling anomalies, and CPU stalls
  • Debug issues related to high-speed I/O and network packet processing paths

System Resource Optimization

  • Optimize kernel memory management, including huge pages and NUMA optimization
  • Profile and optimize interrupt handling routines, IRQ distribution, and CPU affinity
  • Analyze and improve CPU cache utilization, memory bandwidth, and PCIe throughput
  • Tune kernel parameters for optimal real-time performance on x86 servers

Required Qualifications

  • 10+ years of systems programming experience with at least 5 years in kernel, systems, or performance-critical development
  • Strong programming skills in C/C++ with understanding of low-level system concepts
  • Experience with Linux kernel development, real-time systems, or high-performance computing
  • Familiarity with Linux kernel debugging tools (at least some of: crash, ftrace, perf, eBPF)
  • Experience measuring and analyzing system performance metrics

Strong Plus

  • Experience with real-time Linux variants (PREEMPT_RT patches)
  • Knowledge of kernel internals and ability to read/understand kernel code
  • Familiarity with CPU isolation and IRQ affinity concepts

Bonus Qualifications

  • Experience with DPDK or similar kernel bypass technologies
  • Experience with SR-IOV, VF drivers, or NIC offload features

Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.