Job Description:Role Overview We are seeking a Senior Cloud GenAI Governance Engineer to manage enterprise-scale GenAI platform engineering. This role focuses on the critical gateway and security layers, managing the architecture flow from User API requests, through Model Armor, to the inference endpoint, and finally into telemetry and Arize AI. The core of this position is building responsible AI infrastructure and AI runtime governance.
Key ResponsibilitiesEnterprise AI Security: Act as the primary owner of AI security guardrails. Implement and manage Model Armor to handle prompt filtering, data leakage prevention, request/response inspection, and enterprise-safe inference.
Advanced API Ecosystem Debugging: Manage complex API lifecycle tracing rather than simple API integration. Troubleshoot endpoint bottlenecks, container orchestration failure analysis, and trace the exact origin of 429 errors across ingress, gateway, load balancing, and GPU saturation.
AI Observability & Telemetry: Utilize Arize AI for runtime monitoring, inference tracing, and model behavior analysis.
Traffic & Gateway Operations: Manage distributed inference debugging, token queueing, and upstream/downstream API flows.
Cloud Infrastructure: Maintain Google Cloud Platform/Azure landing zones, Terraform deployments, and Kubernetes clusters to support these governance pipelines.
Required Qualifications- 8+ years experience in API lifecycle tracing, rate limiting*** and token throughput management.
- 8+ years hands-on expertise with AI security tools, specifically Model Armor.
- Deep understanding of enterprise AI observability platforms, particularly Arize AI.
- Strong background in cloud networking (Google Cloud Platform/Azure) and Kubernetes infrastructure.
Note: This role is focused entirely on generative LLMs; traditional predictive ML or data science backgrounds are not a fit.
Must be onsite at client in Charlotte, NC at least 3 days/week
About NTT DATA:
NTT DATA is a $30 billion trusted global innovator of business and technology services. We serve 75% of the Fortune Global 100 and are committed to helping clients innovate, optimize and transform for long term success. As a Global Top Employer, we have diverse experts in more than 50 countries and a robust partner ecosystem of established and start-up companies. Our services include business and technology consulting, data and artificial intelligence, industry solutions, as well as the development, implementation and management of applications, infrastructure and connectivity. We are one of the leading providers of digital and AI infrastructure in the world. NTT DATA is a part of NTT Group, which invests over $3.6 billion each year in R&D to help organizations and society move confidently and sustainably into the digital future. Visit us at us.nttdata.com
NTT DATA endeavors to make accessible to any and all users. If you would like to contact us regarding the accessibility of our website or need assistance completing the application process, please contact us at This contact information is for accommodation requests only and cannot be used to inquire about the status of applications. NTT DATA is an equal opportunity employer. Qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability or protected veteran status. For our EEO Policy Statement, please click here. If you''d like more information on your EEO rights under the law, please click here. For Pay Transparency information, please click here.
Never miss an opportunity! Create an alert based on the job you applied for.