Penn Medicine is dedicated to our tripartite mission of providing the highest level of care to patients, conducting innovative research, and educating future leaders in the field of medicine. Working for this leading academic medical center means collaboration with top clinical, technical and business professionals across all disciplines.
Today at Penn Medicine, someone will make a breakthrough. Someone will heal a heart, deliver hopeful news, and give comfort and reassurance. Our employees shape our future each day. Are you living your life's work?
Penn Medicine is looking for a Senior Site Reliability Engineer to help revolutionize health care by safely and responsibly applying Machine Learning and Artificial Intelligence to clinical care: from prototype to operations; through problem discovery, user-oriented design, and scaled experimentation. Applied data science at Penn Medicine has AI programs helping patients in every care setting. We continue to extend the capabilities and flexibility of the infrastructure that runs predictive models and is essential to the data science processes to improve patient care.
The Senior Site Reliability Engineer (SRE) is responsible for production systems enabling custom software development work in support of areas such as Application Development, Informatics, Predictive Healthcare, and Translational Research. The SRE applies software engineering, systems engineering, and dev-ops principles to operations; designs and implements cohesive end-to-end systems for comprehensive solutions with measurable patient and clinician outcomes; and builds resilient, self-healing systems. The Senior SRE values proactive automation, expert tool-smithing, and adherence to design and engineering principles over reactive systems management, or traditionally siloed systems administration.
• Designs, builds, and maintains our core infrastructure, while retaining the flexibility to integrate next-generation systems for Predictive Medicine.
• Applies Systems Engineering and Software Engineering skills to advance core infrastructure, systems design, recurring microservices, tooling, automation, and libraries to "lift all the boats" instead of fragmented support for individual applications.
• Improves and automates system infrastructure and application deployment processes to be as boring as possible.
• Implements proactive monitoring and alerting of symptoms, instead of reactive alerting of outages.
• Establishes best practices for securing automated systems infrastructure and microservices.
• Advances the maturity of SRE discipline across Penn Medicine.
• Applies dev ops process to monitor and stabilize core infrastructure.
• Prevents incidents, e.g., reduce baseline noise, streamline metrics, characterize expected latency, tune alert thresholds, ticket applications without effective health checks, improve playbooks for issue resolution.
• Uses playbooks to document actions alongside code in source control to turn initial problem discovery and resolution into automated processes.
• Participates in on-call rotation for systems infrastructure and to provide subject matter expert support for Software Engineers, Data Engineers, and Data Scientists developing, building, testing, deploying, and monitoring their microservices.
• Collaborates to define service level agreements (SLAs), objectives (SLOs), and automates measurement of service level indicators (SLIs)
• Triages production issues across all products and services, and at all levels of the stack.
• Performs duties in accordance with Penn Medicine and entity values, policies, and procedures
• Other duties as assigned to support the unit, department, entity, and health system organization
Required Education and Experience
• Bachelor's Degree required, in a relevant field, including Computer Science, Systems Engineering, Data Science, Mathematics, Statistics. Master's Degree preferred.
• 3+ years of Software engineering experience required.
• Current Internal Penn Medicine Information Services division employees may be considered with proof of active and continued enrollment in an approved bachelor degree program
• 1+ years of Infrastructure as code with a cloud provider or Systems Engineering required.
• Exceptional design and programming skills in a language such as golang, python, C, C++ is required.
• Exceptional coding skill in ANSI SQL or PL/PGSQL is required.
• Competency in Linux and the Unix shell, or with equivalent operating systems is required.
• Demonstrated interpersonal/verbal communication skills is required.
• Ability to communicate effectively with all levels of staff is required.
• Demonstrated customer service skills is required.
• Production experience with microservice orchestration (e.g., Hashicorp, Kubernetes), logging, metrics, and alerting (e.g., Loki, Grafana, Prometheus, Kibana, Fluentd).
• Production experience with infrastructure as code using a cloud provider (e.g., Azure, AWS, Google Cloud Platform) using tools like Terraform and python.
• Production experience with dev ops automation directly from source control, semantic versioning, and CI/CD (e.g., GitHub actions, Circle CI, Travis CI).
• Development of open-source products.
• Effective asynchronous communication, documentation of process and code in source control toward automation.
• Practical understanding of Agile software development process with code review and retrospectives.
As part of our COVID-19 response, this position may currently be offering partial or full remote work. However, in the near future this position will require full or partial on-site work.
Be a part of the exciting and ground-breaking upcoming years for the Penn Medicine Information Services department!
Because growth is essential to continuing to meet the current and future needs of patients, Penn Medicine continues to expand its capabilities.
Penn Medicine's Information Services (IS) Department focuses its efforts on the clinical and financial systems that support the day-to-day operations of six (6) hospitals, several satellite practices, and more than 8,923 physicians.
We believe that the best care for our patients starts with the best care for our employees. Our employee benefits programs help our employees get healthy and stay healthy. We offer a comprehensive compensation and benefits program that includes one of the finest prepaid tuition assistance programs in the region. Penn Medicine employees are actively engaged and committed to our mission. Together we will continue to make medical advances that help people live longer, healthier lives.
Penn Medicine http://www.pennmedicine.org/careers/
Live Your Life's Work
We are an Equal Opportunity and Affirmative Action employer. Candidates are considered for employment without regard to race, ethnicity, color, sex, sexual orientation, gender identity, religion, national origin, ancestry, age, disability, marital status, familial status, genetic information, domestic or sexual violence victim status, citizenship status, military status, status as a protected veteran or any other status protected by applicable law.