Research Engineer, Post-Training (Hybrid)

Job Description

Job Description

Research Engineer, Post-Training COOP / Internship Hybrid - Brampton Weʼre looking for a co-op student whoʼs curious about how LLM agents behave in real-world applications—especially when it comes to hallucination, grounding, and evaluation. Youʼll be working on the post-training lifecycle of Themis Agents (e.g., knowledge base assistants, alarm summarizers, and SCADA-aware chatbots), ensuring that models are not only powerful but also accurate and trustworthy. Youʼll contribute to designing and running evaluation frameworks, comparing retrieval strategies, testing prompt chains, and analyzing model behavior across a range of tasks. This is a hands-on, research-oriented role ideal for someone with AI-related coursework and a passion for building better, safer AI systems.   What you’ll be doing? Evaluation & Grounding

• Evaluate Themis Agents for accuracy, factual consistency, hallucination, and tool correctness.
• Analyze grounding failures—when models “go off-scriptˮ from retrieved knowledge or internal documents.
• Score and compare outputs across tasks like Q&A, summarization, and event reasoning Prompt & Retrieval Testing
• Experiment with prompt templates, few-shot examples, and retrieval settings.
• Compare vector store search performance using embedding models, chunking strategies, and context window variations.
• Run A / B tests across model versions and prompt chains. Tooling & Automation
• Build or extend evaluation pipelines in Python and frameworks like
• LangChain, OpenAI API, or Transformers.
• Visualize and organize test results using tools like Streamlit, pandas, or Dash.
• Help define “hallucination typesˮ and build reproducible test suites for failure tracking. You Might Be a Good Fit If You...
• Are pursuing a Bachelorʼs degree in Computer Science, Engineering, Math, Physics, or a related field.
• Have completed coursework in machine learning, natural language processing, or AI systems.
• Enjoy debugging model outputs and understanding why a chatbot provided an unexpected answer.
• Have worked on side projects involving chatbots, retrieval-based systems, or LLMs.
• Can write clean Python code and think critically about accuracy, context, and grounding BonusPointsFor
• Experience with LangChain, LlamaIndex, or RAG architectures.
• Familiarity with evaluation frameworks like lm-eval-harness, RAGAS, or custom harnesses.
• Knowledge of vector databases (e.g., Qdrant) and prompt engineering techniques
• Interest in the safety, reliability, or interpretability of AI agents. If you are interested in joining a growing organization in this exciting role, please submit your cover letter and resume. About Us : Themis Intelligence (a Survalent Company) is pioneering a new era of grid modernization through Human-Guided Intelligence (HGI) – blending AI with the real-world insight of utility operators. Our mission is to help utilities operate with greater foresight, resilience, and confidence by turning vast streams of operational data into reliable, actionable intelligence. Built for the physics of the grid and the people who manage it, our platform supports critical workflows : from forecasting and alarm analysis to outage response and SCADA-centric analytics – with AI agents designed specifically for utility environments. By combining deep technical rigor with an understanding of physical infrastructure, we enable smarter, faster decisions that drive safer and more efficient power delivery. Backed by Survalent’s long-standing leadership in ADMS, Themis Intelligence offers a modern foundation for data-driven decision-making in one of the world’s most essential sectors. We’re building tools that matter – systems that learn, adapt, and scale in lockstep with the evolving grid. For those passionate about solving high-impact problems at the intersection of AI, data, and energy, this is where your work can power the future. Survalent values a diverse workplace and strongly encourages women, people of all races, color, creed, ancestry, ethnic origin, sexual orientation, gender identity or expression, age, religion, national origin, citizenship status, disability, marital status, family status, and those with disabilities to apply.  Survalent is an equal opportunity employer. We are committed to providing accommodation for people with disabilities. If you require accommodation, we will work with you to meet your needs.  While we appreciate the interest of all applicants, only those selected for an interview will be contacted