Health Data Engineer

A Health Data Engineer is sought for a growing data center which serves the schools of the health sciences. This person will manage the data lifecycle from ETL to data destruction in SQL Server on-prem or Cloud environments. On-prem work will be done in SQL Server with T-SQL and SSDT. When working in the cloud, this role uses tools such as Azure Synapse, Azure Data Factory, and Microsoft Fabric, along with Apache Spark.

The Data Engineer will receive data and load into SQL Server. The data will arrive in various formats including SAS, CSV, Parquet and fixed width. The format will change based on the data owner. All actions must be performed in our secure HIPAA compliant environment according to data center policies and procedures and thoroughly documented. The data engineer will monitor the SQL Server execution plans and make modifications to improve performance.

The incumbent will work with Principal Investigators and their teams to create analytic datasets in SQL server. This will require gathering specifications and listening to requirements from various teams. The position must be responsive to different requirements from different groups. In some cases, you will serve as an Honest Broker. Must be able to consider various options for data collection and recommend solution that works for customer. The candidate must be able to evaluate different performance metrics and recommend ETL solutions. Must be able to work as part of a team with strong communication skills.

Required skills include strong SQL Server skills with the ability to write advanced queries and excellent organizational, documentation and communication skills with proficiency in Microsoft Office. Experience with cloud services such as Azure Synapse, Azure Data Factory, and Microsoft Fabric, along with an understanding of Spark, is desired. Experience with tools such as with Python, C#, Powershell, Power BI, SAS, Stata, and Web technologies are needed. Experience with health datasets is ideal but not required. Familiarity with Execution Plans and optimization is a plus.