Splunk Administrator (LOCAL ONLY)

Splunk Administrator Operational Technology (OT)

Summary:
The OT Splunk Administrator is responsible for administering, maintaining, and optimizing the Splunk Enterprise platform supporting Operational Technology (OT) environments, including substations, grid management systems, telecom networks, data centers, and critical OT cyber-security infrastructure.

This role supports both security operations and regulatory compliance, enabling accurate log ingestion, detection use-case development, dashboarding, and reporting aligned with NERC CIP, internal controls, and the OT Compliance Center of Excellence.
The Splunk Administrator will partner closely with OT Network Security Analysts, Firewall & Network teams, Enterprise Cyber Security, and Compliance teams to ensure log integrity, visibility, and platform reliability across critical OT assets.
Key Responsibilities:
Splunk Platform Administration
Maintain, administer, and troubleshoot Splunk Enterprise (indexers, search heads, forwarders, deployment server, cluster management).
Ensure system uptime, performance tuning, capacity planning, and scaling for OT-critical workloads.
Install, configure, and maintain Splunk Universal Forwarders on OT systems (Windows, Linux, appliances where applicable).
Manage Splunk apps, add-ons, data models, and knowledge objects.
OT-Specific Log Ingestion & Content Development
Onboard OT data sources such as firewalls, switches/routers, SCADA-adjacent systems, VPN concentrators, RSA SecureID, Tripwire Enterprise, endpoint security, and network monitoring tools.
Maintain and validate NERC CIP-relevant log retention and log integrity requirements.
Build and maintain dashboards, alerts, correlation searches, and reports for OT security and compliance use cases.
Create OT-specific detection content in collaboration with OT Network Security Analysts.
Security Operations & Monitoring Support
Partner with security operations teams to troubleshoot missing logs, ingestion errors, and detection gaps.
Conduct root-cause analysis for log issues impacting OT security visibility.
Support incident response by providing Splunk queries, timelines, and data exports.
Regulatory Compliance Enablement
Support internal and external audits with evidence extracts, dashboards, and log integrity verification.
Ensure Splunk configurations meet compliance expectations (e.g., NERC CIP-007, CIP-010, CIP-003 monitoring controls).
Maintain documentation for logging architectures, data flows, and procedures aligned with the Compliance Center of Excellence.
Automation & Tooling Integration
Integrate Splunk with ServiceNow for alerting, ticket creation, and automated workflows.
Collaborate with Firewall Governance, PKI, RSA, and VPN lifecycle stakeholders for log enhancement opportunities.
Maintain scripting (Python, PowerShell, Bash) for automation of ingestion and monitoring tasks.
Operational Support & Governance
Maintain work registers, operational runbooks, and architectural documents
Provide knowledge transfer
Minimum Qualifications
3 5+ years' experience administering Splunk Enterprise, preferably in an industrial, utility, or OT environment.
Strong understanding of:
Splunk configuration, tuning, and troubleshooting
Log ingestion pipelines
Windows & Linux server administration
Network security concepts (firewalls, VPN, routing, segmentation)
OT/ICS protocols (preferred, not required)
Ability to obtain and maintain NERC CIP access requirements.
Preferred Qualifications
Bachelor s degree in Cybersecurity, Information Systems, Engineering, or related field or equivalent experience.
Experience in utility OT environments (telecom, substations, control centers, power generation, or pipeline operations).
Familiarity with:
Tripwire, RSA SecureID, SCADA systems, firewall governance
NERC CIP requirements
Splunk ES or Splunk ITSI
Scripting automation (Python, PowerShell, Bash)
Experience developing dashboards, correlation searches, and detection content.
Key Competencies
Strong analytical and troubleshooting capability
Clear written documentation and evidence-generation skills
Stakeholder partnership and cross-functional communication
Ability to work in ambiguous, high-stakes OT environments
Accountability, follow-through, and consistency