Hiring Now: AI Evaluation Engineer Near You – Apply Today!

Your Next AI Evaluation Engineer Job Starts Here – Apply Now!

AI Evaluation Engineer

Job Posted: 4 hours ago

Salary: £ 700 - 750 / Day
Job Type: Contract
Location: London

Expire in: a month

Job Description

AI Evaluation Engineer

Rate / Duration / Location

£700-£750/day Inside IR35 | Duration: 4 months | Clearance: BPSS + SC eligible | Location: London, Bristol or Manchester | Hybrid: 2 days onsite per week.

Overview

We're looking for an experienced AI Evaluation Engineer to join a specialist Public Sector Technology team. This hands-on role focuses on building evaluation frameworks, tooling and harnesses for AI systems, particularly LLM and agentic AI solutions. The role requires ownership, experimentation and rapid delivery.

Key Responsibilities

Design evaluation frameworks and tooling
Develop evaluation harnesses
Evaluate model and agentic AI systems
Define metrics and methodologies
Build repeatable evaluation processes
Investigate AI failures
Work directly with government teams
Prototype and test solutions
Present findings and recommendations
Contribute to strategic direction.

What We Are Looking For

Strong software engineering experience
Experience building and evaluating AI systems
Understanding of LLMs and agentic AI
Experience with AI evaluation frameworks
Ability to code independently
Strong problem-solving and communication skills
Comfortable working autonomously.

Technical Experience

AI/LLM Evaluation, RAG Evaluation, Ragas, Agentic AI Evaluation, Evaluation Harnesses, LLM Testing and Benchmarking, Prompt Engineering, Python Development, Model and Agent Integration.

Ideal Background

AI Evaluation Engineer, Applied AI Engineer, AI Engineer, LLM Engineer, Harness Engineer, Prompt Engineer or Software Engineer specialising in AI.

Team

Small Public Sector Technology team of approximately 2 to 6 people within a wider organisation.

What Will Make Someone Stand Out

Experience building evaluation tooling, evaluating LLMs, using Ragas, identifying AI failure modes, combining engineering with strategic thinking, and thriving in fast-moving environments.

Summary

Hands-on AI Evaluation Engineer role supporting government AI initiatives through evaluation, testing, benchmarking and framework development

Before You Apply

Do not include the following in your job application, CV, or cover letter:

Bank details.
National Insurance number.
Date of birth.

You should not be asked for payment or irrelevant information. If you have concerns about a job advert or employer, seek guidance on how to proceed.

Looking for your next career move? Join a top company hiring AI Evaluation Engineer job near me in London! This is your chance to work on exciting projects, grow professionally, and enjoy a rewarding career with competitive pay and excellent benefits. Whether you're an experienced professional or looking to take the next step, this role offers the perfect opportunity to enhance your skills and make an impact. Don’t miss out—apply today via Vita CV and take your career to the next level!