Role overview
- Lead the design and development of an enterprise-grade Large Language Model (LLM) Gateway serving as the central API layer for all application teams
- Architect mediation pipelines for prompt cleansing, PII detection/masking, guardrails, linting, throttling, and quota enforcement
- Build centralized controls to enforce organizational compliance, governance, and security policies for all LLM usage
- Develop retrieval-augmented generation (RAG) and shared vector database integration capabilities to support enterprise-wide use cases
- Apply distributed systems principles and formal methods to ensure system scalability, correctness, and resilience
What we're looking for
- Strong background in Computer Science, Machine Learning, or related field (M.S. preferred)
- Deep expertise in LLM integration, prompt/response mediation, and GenAI system design
- Hands-on experience with vector databases (e.g., Pinecone, Weaviate, Milvus, FAISS, Vespa, pgvector) and embedding models
- Knowledge of information retrieval methods: dense retrieval, semantic search, filtering, hybrid approaches
- Experience building distributed, high-scale backend systems with strong governance/security requirements
- Competitive compensation package
- Full Benefits; Medical, Vision, Dental, and more!
- Opportunity to get in with an industry leading organization.
Why IDR?
- 25+ Years of Proven Industry Experience in 4 major markets
- Employee Stock Ownership Program
- Dedicated Engagement Manager who is committed to you and your success.
- Medical, Dental, Vision, and Life Insurance
- ClearlyRated's Best of Staffing® Client and Talent Award winner 12 years in a row.
Tags & focus areas
Used for matching and alerts on DevFound Fulltime Ai Ai Engineer Machine Learning Generative Ai