AI/ML Engineer

Job Description

Level: Senior | Experience: 6–10 Years | Location: Lahore, Pakistan (On-Site)

About the Project

The platform must handle 10–15 AI queries per user per day across a workforce of 6,000+, with response latency under 3 seconds, from infrastructure hosted entirely within Saudi Arabia under PDPL compliance.

The Role

You will also design the evaluation methodology that tells us, on an ongoing basis, whether the system is actually working.

• Architect and implement the full RAG (Retrieval-Augmented Generation) pipeline using Azure AI Search, with native multilingual vector indexing for both Arabic and English document corpuses — no intermediate translation layer
• Integrate Azure OpenAI Service as the primary LLM, and design the supplementary model integration layer for Sovereign AI models (Jais, Falcon) to ensure Arabic-first quality across all user-facing outputs
• Build the AI orchestration layer that routes user intents to the correct backend actions — leave submissions, ticket creation, room bookings, document generation — via a service account architecture with strict authorization guardrails
• Design and implement the prompt engineering framework, including version control, prompt registry, and audit trail for all prompts in production
• Implement confidence scoring and source attribution on all AI responses, so users and administrators can see which knowledge base article or SAP record an answer draws from
• Build the AI monitoring stack — latency dashboards, hallucination rate tracking, accuracy metrics, and bias detection — integrated with Azure Monitor and Application Insights
• Design and implement content filtering, prompt injection prevention, and adversarial input handling
• Implement conversation memory within sessions and opt-in persistent memory across sessions for the Enterprise AI Assistant
• Define and execute the model evaluation methodology, including test datasets for Arabic and English, benchmark metrics, and continuous improvement cycles
• Configure and manage token budget controls, rate limits, and graceful fallback logic when AI models are unavailable
• Collaborate with the backend team on AI service integration points, and with the frontend team on AI interaction patterns and response formatting

• 6 to 10 years of engineering experience, with at least 3 years working directly on LLM systems, RAG pipelines, or production AI applications
• Hands-on experience with Azure OpenAI Service — model deployment, API integration, prompt management, and cost optimization
• Deep understanding of RAG architecture — chunking strategies, vector embedding models, retrieval evaluation, and re-ranking
• Experience with Azure AI Search or equivalent vector database solutions (Qdrant, Weaviate, Pinecone)
• Demonstrated work with multilingual NLP — Arabic language experience or experience with other RTL/morphologically complex languages is a strong advantage
• Proficiency in Python for AI/ML engineering — building pipelines, evaluation scripts, and production AI services
• Experience building guardrail systems — content filters, authorization enforcement in AI outputs, and adversarial input handling
• Strong grasp of AI evaluation methodology — how to measure what matters, not just what is easy to measure
• Familiarity with MLOps practices — version control for models and prompts, deployment pipelines, and monitoring in production
• Ability to communicate complex AI decisions clearly to non-technical stakeholders

• Direct experience with Arabic LLMs — Jais, Falcon, or fine-tuned Arabic variants of open-source models
• Experience with AI agent frameworks — LangChain, Semantic Kernel, or custom orchestration systems
• Background building AI systems for enterprise HR, telecom, or regulated industry environments
• Understanding of PDPL or similar data protection frameworks and their implications for AI data handling