Cloud Platforms Data Manager

Job#: 2066261

Job Description:

TITLE: Cloud Platforms Data Manager JOB CODE:

JOB FAMILY: FLSA: Non-Exempt/Exempt DATE: December 2024

SUMMARY

This role will oversee the design, implementation, and ongoing management of our cloud-based data platform, ensuring it is scalable, reliable, secure, and meets business needs by leading a team of engineers, collaborating with stakeholders, and staying abreast of emerging cloud technologies to optimize data storage, processing, and analysis capabilities across the organization.

ESSENTIAL FUNCTIONS

1. Must understand existing enterprise technologies and their standard applications, as well as the application of new technologies, when needed. Build an AI portfolio of capabilities, products and services.

2. Support the delivery of an cloud data architecture for a given initiative complying with enterprise reference architecture. .

3. Understand the role of information technology as a stakeholder and enabler within the governance and oversight processes and committees across the institution.

4. Participate in and demonstrate acumen for advisory, compliance and audit activities. Includes approaches, like responsible AI; best practices, like bias detection; and adherence, like regulatory or legislative requirements assurance.

5. Demonstrate proficiency in aggregation of use case, workload, performance, capacity, throughput, preciseness, availability, orchestration components, and other discreet considerations sufficient to reflect an informed view ready for design or implementation.

6. Embrace a structure of demo, development, test, staging and production AI environments. Define the expectations for vendors, programmers, users, and support personnel who interact in multiple personas and roles.

7. Advise leadership and other stakeholders on configurations (implicit or explicit), guardrails, fitness, and limitations of AI technologies in a conversational manner tailored to healthcare, research, and academic aims.

8. Works with cross-functional business and IT teams to ensure proper performance of developed or acquired assets.

Foster a culture of collaboration, innovation, automation, and continuous improvement across IT and the business.

Research and share emerging technologies, leading practices, and industry trends.

Collaborate with business to identify appropriate solutions aligned with appropriate standards, use cases, and leading practices.

MINIMUM QUALIFICATIONS

EDUCATION/EXPERIENCE

Required

• Bachelor's degree in computer science, Data, Engineering or a related field (preferred).

10+ years of experience in data, Cloud, and ML, with a proven track record of delivering enterprise-level Cloud Data Platforms

Healthcare (clinical), research, academic and leadership experience is desired.

• Experience with SQL and NoSQL databases and their integration with AI models and RAG systems.
• Familiarity with enterprise data governance frameworks and the ability to integrate AI solutions within these frameworks.
• Familiarity with Azure cloud services and other cloud-based data platforms.
• Experience with data science and ML operations practices, including data pipelines, model training and evaluation, and CI/CD processes.
• Experience with enterprise architecture frameworks (e.g., TOGAF, Zachman) for capturing, maintaining, and aligning data architectures with broader enterprise architecture standards and business objectives.
• Strong business acumen with the ability to translate technical concepts into business value.
• Demonstrated experience working in high-performance computing and Cloud-based environments.
• Experience combining artificial intelligence technologies with other technologies like automation.

KNOWLEDGE, SKILLS, & ABILITIES

Strong proficiency in Microsoft Azure services, particularly Azure Data Factory, Azure SQL Database, Azure Databricks, Azure Cosmos DB, and Azure Synapse Analytics (formerly SQL Data Warehouse).

Experience with data pipeline orchestration tools like Apache Airflow, Azure Data Factory, and AWS Glue.

Develops data management strategies for cloud data pipeline development, training and deployment

Work requires knowledge of data, data science and how D&A can be used to support business decision making

Work requires knowledge of data architecture, data management and the overlap with privacy, security and compliance.

Work requires the ability to work with internal and external individuals from different disciplines and different levels of training.

Work requires self-motivation with the ability to work effectively in a team environment.

Work requires strong organizational, technical and communication skills.

Work requires flexibility and adaptability in dynamic environment.

Demonstrates a good understanding of product management, agile principles and development methodologies

Delivers presentation skills to relevant stakeholders and technical audiences

Working knowledge of consumption-based models driven by processing needs and model or feature performance.

Understands, support, and adheres with appropriate data use agreements.

The following is the acronym, "PACT", and is fundamental to all non-clinical positions at UT Southwestern Medical Center:

P-Problem Solving: Employees take ownership in solving problems effectively, efficiently, and to the satisfaction of customers, or managers. They show initiative in addressing areas of concern before they become problems.

A-Ability, Attitude and Accountability: Employees exhibit ability to perform their job and conduct themselves in a professional and positive manner reflecting a professional environment readily assuming obligations in a dependable and reliable manner.

C-Communication, Contribution, and Collaboration: Who are our Customers? Anyone who requests our help, needs our work product, or receives our services. Employees focus on customer service with creative solutions while improving the customer experience through clear, courteous, and timely delivery and communication. Sharing ideas with others helps expand our contribution to department goals.

T-Teamwork: Employees work to contribute to the department's success by supporting co-workers, promoting excellence in work product and customer service, and in maintaining a satisfying, caring environment for each other.

Overview: The role is focused on managing cloud data platforms, with a strong emphasis on Azure. The goal is to find a Data Engineering Manager with expertise in cloud platforms, who can build, maintain, and optimize data solutions. The role requires experience with Azure, Databricks, and other cloud data services. A critical component is the ability to collaborate with infrastructure and shared services teams while speaking the same technical language, especially given the organization's relative newness in the Azure space.

Key Responsibilities:

Cloud Data Platform Strategy & Execution:

• Focus on building and optimizing cloud-based data solutions.
• Manage the Azure data platform, and work with data ingestion and data pipelines in the cloud.
• Implement DevOps processes, including CI/CD pipeline setup (to some extent).
• Evaluate and enable the right Azure services, including:
• Azure Cognitive Services
• Azure Cosmos DB
• Azure Data Factory
• Other relevant data services for AI/ML, image processing, etc.

Collaboration with Infrastructure Teams:

• Engage with infrastructure and shared services teams, particularly in helping them understand and enable the right Azure services.
• Must be able to speak the language of infrastructure teams, ensuring they are on the same page regarding services like Cosmos DB, Data Factory, and more.
• Provide input on cloud architecture, such as decisions on tools like Terraform vs. Bicep.

Evaluating New Requests:

• Evaluate and recommend Azure services for new requests (e.g., AI image processing needs).
• Advise others on what services and solutions to use based on the use case.

Data Platform Architecture:

• Design and advise on data platform architecture (including medallion architecture).
• Transition from on-prem data infrastructure (e.g., SSIS, Oracle-based warehouses) to cloud architecture, ensuring the ability to bring in both structured and unstructured data.
• Assist in understanding how to move data to the cloud while enabling research teams to access the data.

AI/ML & Data Processing:

• Assist in determining the right architecture for AI/ML services.
• Experience in AI-related services in Azure (e.g., image processing, NLP, etc.), especially evaluating them from a data volume, type, and processing need perspective.

CI/CD & DevOps:

• Understand CI/CD processes and build basic pipelines, or at least direct the team on what needs to be done to implement them. The focus is on strategic guidance for DevOps practices.

Experience Needed:

• Azure Cloud Expertise:
• Strong knowledge of Azure Data Services like Data Factory, Cosmos DB, Cognitive Services, etc.
• Familiarity with cloud-based data architecture, especially with services related to big data processing and AI.
• Experience with Data Ingestion & Pipelines in Azure, as well as Databricks.

Data Platform Management:

Background in data engineering or data management with the ability to understand or advise on data architecture.

Experience working with structured and unstructured data environments.

Understanding of data mesh and its implementation.

DevOps Experience:

Understanding of DevOps practices, particularly in the context of data pipelines.

Familiarity with CI/CD pipeline creation (hands-on experience not necessary but a strong understanding of the process is required).

Collaboration & Leadership:

Ability to communicate effectively with infrastructure teams to help them understand and enable relevant services.

Ability to evaluate and drive the enabling of specific Azure services as needed.

Ability to assess AI/ML-related requests and provide guidance on the appropriate services.

Current Tech Landscape:

Current Data Architecture:

Primarily using SQL-based data warehouses (Microsoft Data Warehouse, Oracle-based).

Aiming to build cloud data infrastructure with Azure landing zones and medallion architecture.

Synthetic data is being used for clinical research, with a partner involved in synthetic data generation for the first use case.

Existing data pipelines are built with SSIS and are on-premise.

Future Vision:

Transition to the cloud, focusing on building a robust data platform.

Migrate unstructured data into Azure, enabling pipelines and making this data available for research teams.

Work with synthetic data as the organization moves towards scaling AI-driven initiatives and incorporating new types of data.

Challenges:

The infrastructure team has limited visibility into the enabled Azure services.

The organization is still working to define and understand its cloud capabilities, and the person in this role will be pivotal in pushing for service enablement and cloud architecture best practices.

Final Thoughts:

This role requires someone who can bridge the gap between data engineering and infrastructure teams in a growing cloud environment.

The ideal candidate should be comfortable strategically guiding the organization's cloud journey while also having hands-on experience to advise on technical decisions (especially Azure services).

The person must be able to speak the language of both technical data engineering and infrastructure teams, ensuring alignment and proper service enablement across the board.

EEO Employer

Apex Systems is an equal opportunity employer. We do not discriminate or allow discrimination on the basis of race, color, religion, creed, sex (including pregnancy, childbirth, breastfeeding, or related medical conditions), age, sexual orientation, gender identity, national origin, ancestry, citizenship, genetic information, registered domestic partner status, marital status, disability, status as a crime victim, protected veteran status, political affiliation, union membership, or any other characteristic protected by law. Apex will consider qualified applicants with criminal histories in a manner consistent with the requirements of applicable law. If you have visited our website in search of information on employment opportunities or to apply for a position, and you require an accommodation in using our website for a search or application, please contact our Employee Services Department at or .

Apex Systems is a world-class IT services company that serves thousands of clients across the globe. When you join Apex, you become part of a team that values innovation, collaboration, and continuous learning. We offer quality career resources, training, certifications, development opportunities, and a comprehensive benefits package. Our commitment to excellence is reflected in many awards, including ClearlyRated's Best of Staffing in Talent Satisfaction in the United States and Great Place to Work in the United Kingdom and Mexico.

Apex Benefits Overview: Apex offers a range of supplemental benefits, including medical, dental, vision, life, disability, and other insurance plans that offer an optional layer of financial protection. We offer an ESPP (employee stock purchase program) and a 401K program which allows you to contribute typically within 30 days of starting, with a company match after 12 months of tenure. Apex also offers a HSA (Health Savings Account on the HDHP plan), a SupportLinc Employee Assistance Program (EAP) with up to 8 free counseling sessions, a corporate discount savings program and other discounts. In terms of professional development, Apex hosts an on-demand training program, provides access to certification prep and a library of technical and leadership courses/books/seminars once you have 6+ months of tenure, and certification discounts and other perks to associations that include CompTIA and IIBA. Apex has a dedicated customer service team for our Consultants that can address questions around benefits and other resources, as well as a certified Career Coach. You can access a full list of our benefits, programs, support teams and resources within our 'Welcome Packet' as well, which an Apex team member can provide.