Software Development Engineer in Test – GenAI / LLM

Job Description

Software Development Engineer in Test – GenAI / LLM

Role Description

Work Arrangement: 1 day onsite per week

Experience Required: 6–8 years

Key Roles & Responsibilities

• Validate chatbot responses for accuracy, relevance, completeness, and consistency.
  
• Create and execute test scenarios, test cases, and regression test suites.
  
• Identify hallucinations, incorrect responses, formatting issues, and knowledge gaps.
  
• Perform chatbot conversation analysis and recommend quality improvements.
  
• Develop and maintain test automation for chatbot validation where possible.
  
• Work closely with developers, AI engineers, and content teams to resolve defects.
  
• Track, document, and report defects, test results, and quality metrics.
  
• Support UAT, release validation, and production quality monitoring.
  

Key Skillsets

• 5 years of strong QA SDET test automation experience.
  
• Experience testing AI, LLM, GenAI, chatbots, virtual assistants, or conversational platforms.
  
• Knowledge of prompt engineering concepts.
  
• Experience with Jira, Confluence, GitHub, and Agile delivery.
  
• Experience with API testing and automation frameworks.
  
• Strong analytical and troubleshooting skills.
  
• Experience with Python, Selenium, Playwright, PyTest, or similar tools.
  
• Strong communication and documentation skills.
  

Essential Skills

• QA SDET and test automation.
  
• AI, LLM, GenAI, chatbots, and conversational platforms.
  
• Prompt engineering concepts.
  
• API testing and automation frameworks.
  
• Python, Selenium, Playwright, or PyTest.
  
• Jira, Confluence, GitHub, and Agile delivery.
  
• Analytical and troubleshooting skills.
  
• Communication and documentation skills.
  

Desirable Skills

• Digital: Python
  
• Digital: Chatbots / Conversational Agents
  
• Generative AI – Quality Engineering
  

Requirements
Java