Senior Observability Engineer Telemetry & Tooling

Senior Observability Engineer – Telemetry & Tooling 

Employment Type: W2 Only (No C2C)
Location: Remote

We are hiring two Senior Observability Engineers to design and build the telemetry foundation powering SRE across 40+ healthcare applications. In this role, you will define instrumentation standards, architect ingestion pipelines, implement alerting logic, and create operational dashboards that turn fragmented signals into actionable observability for engineering and operations teams.

Key Responsibilities

• Design and deploy metrics, logs, traces, and event ingestion pipelines from healthcare applications into observability platforms.
• Establish and enforce logging, tracing, and instrumentation standards (structured logs, correlation IDs) in alignment with PHI/HIPAA guidelines.
• Configure, tune, and optimize alerting rules, noise-reduction strategies, and signal-to-noise improvements for SRE and operations teams.
• Build and maintain operational dashboards using Splunk, Grafana, Dynatrace, Datadog, or similar tools.
• Develop runbooks, onboarding patterns, and reusable documentation to support scalable application onboarding.
• Collaborate closely with SRE, APM, infrastructure, and application teams to improve end-to-end telemetry coverage.

Required Experience & Skills

• 7+ years of experience in Observability, SRE, DevOps, Production Engineering, or similar roles.
• Strong hands-on expertise with log and metrics platforms such as Splunk, Elastic, Datadog, Prometheus, or equivalent.
• Experience working with at least one enterprise APM tool (Dynatrace, AppDynamics, New Relic, Datadog).
• Ability to design and build dashboards in Grafana or similar visualization tools.
• Proficiency in scripting (Python, Bash, or PowerShell) for automation, parsing, and integrations.
• Strong understanding of distributed systems, microservices, APIs, containers, and cloud environments.

Preferred / Nice to Have

• Experience in healthcare IT (EHR systems, patient portals, integration engines, payer platforms).
• Understanding of HIPAA/PHI logging controls, redaction patterns, and data-segmentation best practices.
• Exposure to OpenTelemetry, event streaming platforms (Kafka, Kinesis), and modern telemetry pipelines.