Search Jobs | Dice.com

Job Details

Skills

Backend Engineer
AI Gateway
LLM
network programming
cloud infrastructure
AsyncPython
aiohttp
anyio
TCP
UDP
WebSocket
SSE servers
TLS
gRPC
HTTP/2
FastAPI
Pydantic
OpenAPI
Docker
AKS
EKS
GKE
Redis Streams
RabbitMQ
Kafka
PostgreSQL
Vault
load balancing
IaC
CI/CD

Summary

Backend Engineer _ AI Gateway

Location: Englewood Cliffs, NJ, 07632 & Plano, TX, 75023-(5 days Onsite/week)

Duration: 1 year

Work Hours: 9AM-6PM local time

We are looking for candidates who are willing to relocate to Texas starting in October and are available to work onsite in New Jersey through September

Job Summary:

We are building an AI Gateway a high-performance, secure intermediary layer that routes, inspects, and governs all traffic between enterprise clients and LLM/AI service providers. This role requires deep expertise in network programming, async Python, and cloud infrastructure to design and operate a latency-sensitive, policy-driven gateway that handles massive concurrent streaming connections at scale.

What we are looking for:

AsyncPython & Network Programming (3+yrs) Strong hands on experience with asyncio, aiohttp/anyio, low level TCP/UDP sockets, WebSocket & SSE servers, TLS handshake, connection pooling, keep alive, and zero copy buffering.
Streaming & Large Payload Handling Ability to manage chunked transfer, bidirectional streaming, back pressure and high throughput data paths for real time LLM inference.
Protocol Knowledge Working familiarity with HTTP/2&HTTP/3 (multiplexing, HPACK), gRPC, and occasional custom protocol parsers.
FastAPI Based Backend (3+yrs) Design, develop and test production grade REST/Streaming APIs; pragmatic use of Pydantic, dependency injection and OpenAPI docs.
Containerization & Kubernetes Docker, AKS/EKS/GKE deployment experience; HPA/VPA scaling, rolling updates, and service mesh (e.g., Istio/Envoy) basics.
Async Messaging & Event Driven Design Production use of Redis Streams, RabbitMQ or Kafka for decoupled task queues, retries and back off logic.
PostgreSQL & Vault Advanced query tuning, partitioning, PgBouncer pooling, plus HashiCorp Vault (dynamic secrets, transit encryption) for key management.
Cloud Networking & Security L4/L7 load balancing, TLS termination, reverse proxy (NGINX/Envoy), mTLS between services, and automated cert lifecycle (ACME/Let sEncrypt).
IaC & CI/CD Terraform modules for multi env infra, plus GitHub Actions/GitLab CI pipelines with container image scanning and canary/blue green deployments.

Work experience desired:

Enterprise Security Gateways Integration with CASB/DLP/WSS or similar edge security platforms; centralized logging & audit trails.
IAM & SSO Configuring Azure AD/Entra ID, Okta, SAML2.0 or OIDC SSO flows; managing Conditional Access & SCIM provisioning.
AI/LLM Pipelines Building prompt engineering workflows, PII anonymisation, and monitoring Azure OpenAI (or comparable) usage, rate limits & cost.
API Gateway Operations Custom webhook/REST integrations, circuit breaker patterns, rate limiting, auto retry and health checking.
Observability & FinOps End to end monitoring with Datadog, PrometheGrafana or Azure Monitor; proactive cost optimisation and 24/7 incident response (SLA/SLO).

Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.

Dice Id: cybrthnk
Position Id: JPC - 93589
Posted 9 hours ago

Fitment

Dice Job Match Score™

🤯 Applying directly to the forehead...

Contact the job poster

Vijaykumar Thadaka

Recruiter @ cyberThink, Inc.

View Profile

Create job alert

Never miss an opportunity! Create an alert based on the job you applied for.

Englewood Cliffs, New Jersey

•

Today

Job Title: Backend Engineer (Python + Networking) Location: Englewood Cliffs, NJ Duration: 1 Year of Contract What we are looking for AsyncPython & Network Programming (3+yrs) Strong hands on experience with asyncio, aiohttp/anyio, low level TCP/UDP sockets, WebSocket & SSE servers, TLS handshake, connection pooling, keep alive, and zero copy buffering. Streaming & Large Payload Handling Ability to manage chunked transfer, bidirectional streaming, back pressure and high throughput data paths f

Easy Apply

Contract

Depends on Experience

Senior Azure Infrastructure Automation Engineer

Newark, New Jersey

•

Today

Senior Azure Infrastructure Automation Engineer Location: Newark NJ (Hybrid) Experience 12+ Years (Cloud Infrastructure, Automation, Platform Engineering) Overview We are seeking a highly skilled Senior Azure Infrastructure Automation Engineer to help scale and enhance our Azure Infrastructure-as-Code (IaC) ecosystem. This role focuses on expanding and standardizing Terraform-based infrastructure automation, strengthening existing Azure platform foundations, and developing an AI-enabled sel

Easy Apply

Third Party

Software Engineer, Gateway & Networking Infrastructure

New York, New York

•

4d ago

CoreWeave is The Essential Cloud for AI . Built for pioneers by pioneers, CoreWeave delivers a platform of technology, tools, and teams that enables innovators to build and scale AI with confidence. Trusted by leading AI labs, startups, and global enterprises, CoreWeave combines superior infrastructure performance with deep technical expertise to accelerate breakthroughs and turn compute into capability. Founded in 2017, CoreWeave became a publicly traded company (Nasdaq: CRWV) in March 2025. Le

Full-time

USD 182,000.00 - 242,000.00 per year

Senior Engineer Core Infrastructure

Queens, New York

•

Today

Position Summary: The Senior Core Infrastructure Engineer - is focused on ensuring the creation and management of Infrastructure including managing operational stability, performance, and cost-efficiency of our cloud-hosted data infrastructures, serving both internal and external users. The Senior Core Infrastructure Engineer excels when working on complex projects, is motivated to deliver results, and exhibits the JetBlue values of Safety, Caring, Integrity, Passion, and Fun. Essential Respons

Full-time

USD 105,600.00 - 150,400.00 per year

Filter Results

Job post features

Posted date

Work settings

Employment type

Distance

Employer type

Work authorization

Backend Engineer _ AI Gateway

Skills

Summary

Dice Job Match Score™

Vijaykumar Thadaka

Similar Jobs

Backend Engineer _ AI Gateway

Job Details

Skills

Summary

Dice Job Match Score™

Vijaykumar Thadaka

Similar Jobs