Apply Now

Backend Engineer _ AI Gateway

Englewood Cliffs, NJ, US • Posted 11 days ago • Updated 2 hours ago

Full Time

On-site

Fitment

Dice Job Match Score™

✨ Finding the perfect fit...

Job Details

Skills

Leadership
Health Insurance
Insurance
IaaS
Python
Network Programming
TCP
UDP
Sockets
WebSocket
Servers
Real-time
HTTP
Multiplexing
Dependency Injection
Kubernetes
Docker
Messaging
Redis
Streaming
RabbitMQ
Apache Kafka
PostgreSQL
Query Optimization
Encryption
Cloud Computing
Computer Networking
Load Balancing
TLS
Proxies
Nginx
Continuous Delivery
Terraform
GitHub
GitLab
Continuous Integration
Jersey
DLP
Microsoft SharePoint
Auditing
SAML
OIDC
SSO
Management
Provisioning
Artificial Intelligence
Prompt Engineering
Workflow
API
Microsoft Azure
Incident Management
SLA

Leadership
Health Insurance
Insurance
IaaS
Python
Network Programming
TCP
UDP
Sockets
WebSocket
Servers
Real-time
HTTP
Multiplexing
Dependency Injection
Kubernetes
Docker
Messaging
Redis
Streaming
RabbitMQ
Apache Kafka
PostgreSQL
Query Optimization
Encryption
Cloud Computing
Computer Networking
Load Balancing
TLS
Proxies
Nginx
Continuous Delivery
Terraform
GitHub
GitLab
Continuous Integration
Jersey
DLP
Microsoft SharePoint
Auditing
SAML
OIDC
SSO
Management
Provisioning
Artificial Intelligence
Prompt Engineering
Workflow
API
Microsoft Azure
Incident Management
SLA

Summary

Company Description

For More Open Positions Visit us at:
;/strong>

Our Mission

WOONGJIN INC. is a rapidly growing team who provides a range of unique, exceptional, and enhanced services to our clients. We have a strong moral code that includes the service of goodness without expectations of reward. We are motivated by the sense of responsibility and servant leadership.

Benefits
Medical Insurance
Vision Insurance
Dental Insurance
401(k)
Paid Sick hours

Job Description

We are building an AI Gateway - a high-performance, secure intermediary layer that routes, inspects, and governs all traffic between enterprise clients and LLM/AI service providers. This role requires deep expertise in network programming, async Python, and cloud infrastructure to design and operate a latency-sensitive, policy-driven gateway that handles massive concurrent streaming connections at scale.

What we are looking for
Async Python & Network Programming (3 + yrs) - Strong hands-on experience with asyncio, aiohttp/anyio, low-level TCP/UDP sockets, WebSocket & SSE servers, TLS handshake, connection-pooling, keep-alive, and zero-copy buffering.
Streaming & Large-Payload Handling - Ability to manage chunked transfer, bidirectional streaming, back-pressure and high-throughput data paths for real-time LLM inference.
Protocol Knowledge - Working familiarity with HTTP/2 & HTTP/3 (multiplexing, HPACK), gRPC, and occasional custom protocol parsers.
FastAPI-Based Backend (3 + yrs) - Design, develop and test production-grade REST/Streaming APIs; pragmatic use of Pydantic, dependency injection and OpenAPI docs.
Containerization & Kubernetes - Docker, AKS/EKS/GKE deployment experience; HPA/VPA scaling, rolling updates, and service-mesh (e.g., Istio/Envoy) basics.
Async Messaging & Event-Driven Design - Production use of Redis Streams, RabbitMQ or Kafka for decoupled task queues, retries and back-off logic.
PostgreSQL & Vault - Advanced query tuning, partitioning, PgBouncer pooling, plus HashiCorp Vault (dynamic secrets, transit encryption) for key management.
Cloud Networking & Security - L4/L7 load balancing, TLS termination, reverse-proxy (NGINX/Envoy), mTLS between services, and automated cert lifecycle (ACME/Let's Encrypt).
IaC & CI/CD - Terraform modules for multi-env infra, plus GitHub Actions/GitLab CI pipelines with container image scanning and canary/blue-green deployments.
Availability to work on-site in New Jersey for the initial assignment and relocate to Dallas by October 2026

Qualifications
Enterprise Security Gateways - Integration with CASB/DLP/WSS or similar edge-security platforms; centralized logging & audit trails.
IAM & SSO - Configuring Azure AD/Entra ID, Okta, SAML 2.0 or OIDC SSO flows; managing Conditional Access & SCIM provisioning.
AI/LLM Pipelines - Building prompt-engineering workflows, PII anonymisation, and monitoring Azure OpenAI (or comparable) usage, rate limits & cost.
API-Gateway Operations - Custom webhook/REST integrations, circuit-breaker patterns, rate-limiting, auto-retry and health-checking.
Observability & FinOps - End-to-end monitoring with Datadog, PrometheGrafana or Azure Monitor; proactive cost-optimisation and 24/7 incident response (SLA/SLO).

Additional Information

All your information will be kept confidential according to EEO guidelines.

*****NO C2C******

Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.
Dice Id: 90803333
Position Id: 232bb1d3f336f6ae607bb2272c8f8bf6
Posted 11 days ago

Create job alert

Never miss an opportunity! Create an alert based on the job you applied for.

Similar Jobs

SVARA SOFTWARE SOLUTIONS INC
Full Time :: Need Senior AI Data Engineer at NYC NY
New York, New York
•
Yesterday
Role- Senior AI Engineer Location- Alpharetta, GA Experience: 8+ Years Key Responsibilities Design, build, and deploy Python-based AI/LLM applications using FastAPI/Flask.Develop REST APIs and scalable enterprise solutions.Build AI-powered data analytics and self-service platforms.Optimize data pipelines for performance and scalability.Collaborate with Risk stakeholders to deliver AI-driven solutions.Provide technical leadership, architecture guidance, and mentor engineering teams.Required Skill
Easy Apply
Full-time
120,000 - 130,000

Perplexity AI
Member of Technical Staff (Software Engineer, API Platform)
New York, New York
•
Today
Perplexity API Platform Perplexity innovates at the frontier of AI infrastructure, search, and orchestration to serve the world's most discerning users. The Perplexity API Platform brings our technology to the world's most discerning developers. From exabyte-scale knowledge indexes to codegen-first agent runtimes, the building blocks behind Perplexity's applications are some of the most battle-tested AI systems in the industry. We believe these same building blocks can and should power the asp
Full-time
USD 220,000.00 - 405,000.00 per year

NxT Level
Senior Software Engineer, Backend
New York, New York
•
Today
Senior Backend Engineer (GenAI / LLM Workflows) Compensation: $170,000-$230,000 base + equity Location: San Francisco, CA or (open to remote to start) About Our Client Our client has built a leading AI-powered clinical documentation platform for home health, used daily by thousands of clinicians and reducing documentation time by 80%+. By giving clinicians their time back, the platform helps home health organizations deliver better care to older adults who want to remain at home. The company
Full-time
USD 170,000.00 - 230,000.00 per year

Michael J. Hennessy Associates
Senior Product Engineer, AI & Platforms
Cranbury, New Jersey
•
Today
At MJH Life Sciences our success is measured by your success! If you set your standards high and want to contribute to a winning team, we'll provide you with every opportunity to help grow our company and your career. Our associates come from all backgrounds, sharing one key quality: determination to succeed. We value being Service Focused, having a Passion for Winning, Innovation, Respect, Integrity, and Teamwork. Nothing means more to us than hiring people with these attributes. If you believe
Full-time
USD 100,000.00 - 125,000.00 per year

Search all similar jobs

More jobs at WOONGJIN, INC in Englewood Cliffs, NJ