Talent.com

Program evaluation Jobs in Dartmouth, NS

Create a job alert for this search

Program evaluation • dartmouth ns

Last updated: 11 hours ago

GenAI Test Engineer

J M Group IncHalifax, NS, Canada
Full-time
Quick Apply

GenAI Testing:</b> LLM, RAG, Agentic AI, API and Test Automation<br /> <br /> What are the top 3 skills required for this role?<br /> 1.GenAI, LLM, RAG and agentic AI testi... Show more

 • New!

Sr. Staff Technical Program Manager

Rayleigh Solar TechDartmouth, NS, CA
Permanent

Are you passionate about accelerating deep-tech innovation through world-class execution? Do you love driving clarity, alignment, and momentum across complex, fast-paced engineering programs? Join ... Show more

Psychologist (Masters) - Forensic Sexual Behaviour Program - Mental Health & Addictions

Nova Scotia Health and IWK HealthDartmouth, Nova, CA
Part-time +1

Click here to apply as an internal applicant.Location: Central Zone, Nova Scotia Hospital  .Department: MHA CZ ECFH Sex Offender Pgm.Type of Employment: Permanent Hourly PT (40%) x 1 .Union: NSGEU ... Show more

Psychologist

Homewood HealthHalifax, NS BMG, CA
Full-time

Make a difference in a growing, innovative mental health organization and centre of excellence in people-centered care.Work with supportive colleagues who are passionate about improving lives and c... Show more

Program Manager

P4GBedford, Nova Scotia, CA
CA$87,150.00–CA$119,832.00 yearly
Full-time +1
Quick Apply

Search conducted by Placemaking 4G at.Physician Assessment Centre of Excellence.The Physician Assessment Centre of Excellence (PACE) knows that building a diverse and inclusive workforce is a conti... Show more

Management Development Program - New Graduates - Atlantic Plumbing

EMCO CorporationHalifax, Nova Scotia, Canada
Full-time

Base Salary - $52,500 + Profit Sharing.We are currently seeking new or recent graduates to join our .Management Development Program.This is a full-time, salaried position with a comprehen... Show more

CCA RPL Program Assistant - 12 Month Contract

Health Association Nova ScotiaBedford, NS, Canada
Temporary

Health Association Nova Scotia (HANS) is a not-for-profit, non-government, membership-based association with 60 years’ experience in providing exceptional human resource shared services, inno... Show more

Remote Graphic design Expert - Data Labelling

WorkadaCole Harbour, Nova Scotia, ca
Remote
Full-time

At Workada, we're not just labeling data, we're helping improve the technology people use every day.Our work creates the examples and feedback that make advanced systems more accurate, consistent, ... Show more

 • Promoted

2526-279 - Adult Learning Program (ALP) Counsellor - IT Campus

Nova Scotia Community CollegeHalifax, NS, CA
CA$91,498.20 yearly
Full-time +2

Adult Learning Program (ALP) Counsellor (PS5)EducationIT CampusNova ScotiaSchool of Access, Education and LanguageContinuing full-time Position: Commencing approximately December 8, 2025Application... Show more

MHTS AMTU/PTT Technician and Customer Service Point of Issue (CSPI)

Bluedrop Training & SimulationHalifax, NS, CA
Permanent

The MHTS AMTU/PTT Technicians and CSPI operate support the Aircraft Maintenance Trainer - Upper, Part Task Trainers and the Training Tool Room.The AMTU/PTT Technicians and CSPI Operator are also re... Show more

Graduate Program Administrator – Sobey School of Business

Saint Mary’s UniversityHalifax, NS, CA
CA$54,107.00 yearly
Full-time +1

Date Posted: November 4, 2025Posting Deadline: November 18, 2025Department: Sobey School of BusinessType of Employment: Permanent, Full TimeGroup Affiliation: APC Group, Group 4 ($54,107 - $69,... Show more

Psychologist

Homewood HealthHalifax, NS, CA
Full-time

Make a difference in a growing, innovative mental health organization and centre of excellence in people-centered care.Work with supportive colleagues who are passionate about improving lives and c... Show more

Corporate Counsel Expert - Remote

YO AI LabsHalifax, NS, Canada
Remote
Full-time +1
Quick Apply

You will apply your expertise in.No prior AI experience is required.Perform contract drafting, redlining, and simulated negotiations.Review MSAs, NDAs, DPAs, and other technology agreements.Evaluat... Show more

Cybersecurity Program Manager - Remote

NTT DATA ServicesHalifax, NS, CA
Remote
Full-time

NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us.If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.... Show more

Psychologist (Masters) - Forensic Sexual Behaviour Program - Mental Health & Addictions

Nova Scotia Health AuthorityDartmouth, Nova, CA
Part-time +1

Click here to apply as an internal applicant.Location: Central Zone, Nova Scotia Hospital  .Department: MHA CZ ECFH Sex Offender Pgm.Type of Employment: Permanent Hourly PT (40%) x 1 .Union: NSGEU ... Show more

Data Labeling Specialist: Remote Contract

WorkadaCole Harbour, Nova Scotia, ca
Remote
Full-time

At Workada, we're not just labeling data, we're helping improve the technology people use every day.Our work creates the examples and feedback that make advanced systems more accurate, consistent, ... Show more

 • Promoted

Program Manager (Pharmacy Solutions)

The Government of Nova ScotiaHALIFAX, NS, CA
Permanent

Sep-26  (Applications are accepted until 11:59 PM Atlantic Time).The Department of Health and Wellness offers opportunities to grow and advance in a dynamic sector that touches the lives of all Nov... Show more

Career and Experiential Learning (CEL) Program Manager

NSCAD UniversityHalifax, NS, CA
Permanent

Reporting to the Dean, Student Affairs, the CEL Program Manager is responsible for maintaining, enhancing, and growing the operational and logistical excellence associated with programs, events, an... Show more

Senior Manager, Risk Management & Compliance

Royal Bank of Canada>HALIFAX, Canada
Full-time

The Senior Manager, Risk Management & Compliance is responsible for developing and implementing global risk management reports, systems, and processes to minimize investments risks.Applies extensiv... Show more

Forex & Crypto Trader – Remote - Entry Level

Maverick FXDartmouth, Nova Scotia
Remote
Full-time

Maverick Currencies is a proprietary trading firm that has been funding and developing traders since 1997.We provide capital, structured training, and ongoing mentorship — so you trade our mo... Show more

People also ask
GenAI Test Engineer

GenAI Test Engineer

J M Group IncHalifax, NS, Canada
11 hours ago
Job type
  • Full-time
  • Quick Apply
Job description

GenAI Testing: LLM, RAG, Agentic AI, API and Test Automation

What are the top 3 skills required for this role?
1. GenAI, LLM, RAG and agentic AI testing
2. Python-based API and test automation, including PyTest or Robot Framework
3. LLM evaluation, prompt-injection and hallucination testing, CI/CD and observability

Additional Information:
Team size, direct reports, key deliverables, unique selling points, additional qualifications, team culture etc. Key Responsibilities
Develop and execute comprehensive test strategies for GenAI applications, including LLM-powered solutions, RAG pipelines, chatbots, copilots and agentic AI workflows.
Validate functional and non-functional requirements across prompts, model responses, APIs, user interfaces, integrations and end-to-end business workflows.
Create reusable evaluation datasets, test cases and scoring rubrics for relevance, groundedness, factual accuracy, completeness, coherence, instruction adherence and response consistency.
Test RAG solutions by evaluating retrieval quality, context precision and recall, source attribution, answer faithfulness and resistance to unsupported or hallucinated responses.
Validate agentic AI behaviour, including tool selection, function calling, reasoning flows, multi-step task completion, memory handling, fallback paths and failure recovery.
Perform adversarial, security and safety testing for prompt injection, jailbreaks, sensitive-data exposure, toxic or prohibited content, authorization bypass and insecure tool use.
Conduct performance, scalability and reliability testing for response latency, throughput, token usage, cost, concurrency, rate limits and degradation under load.
Build automated regression suites in Python and integrate GenAI evaluations, API tests and quality gates into CI/CD pipelines.
Monitor production quality through traces, evaluation metrics, user feedback and drift indicators; identify defects, analyse root causes and support remediation.
Collaborate with product, development, data science, security and business teams to define acceptance criteria, test coverage and release-readiness standards.

Required Qualifications and Skills:
Strong experience in software quality engineering, test strategy, test design, defect management and automation for enterprise applications.
Hands-on knowledge of GenAI concepts, LLM behaviour, prompt engineering, embeddings, vector databases, RAG architectures and agentic AI systems.
Proficiency in Python and experience with automation frameworks such as PyTest or Robot Framework, together with API testing tools and libraries.
Experience evaluating LLM outputs using deterministic checks, semantic similarity, model-based evaluation and human-review workflows.
Practical experience testing hallucinations, bias, robustness, non-deterministic behaviour, prompt injection, guardrails and content-safety controls.
Working knowledge of GenAI evaluation and observability tools such as Ragas, DeepEval, Promptfoo, LangSmith, Arize Phoenix or equivalent platforms.
Strong SQL skills and experience validating source data, document ingestion, chunking, metadata, embeddings and retrieval results.
Experience with GitHub, GitHub Actions or comparable CI/CD tools; familiarity with Docker, Kubernetes, AWS or Azure is desirable.
Ability to define measurable acceptance thresholds, analyse evaluation results and communicate quality risks clearly to technical and business stakeholders.
Strong analytical, investigative and problem-solving skills, with attention to responsible AI, privacy, security and regulatory expectations.

Preferred Qualifications:
Bachelor's or Master's degree in Computer Science, Software Engineering, Data Science or a related field, with at least five years of experience in quality engineering, test automation, data testing, AI/ML testing or a related discipline.
Experience testing applications in banking or another regulated environment, including model governance, audit evidence and responsible-AI controls, is an asset.