AI EVALUATION · APPLICATION CHECKS · REGRESSION
Artificial Intelligence Testing Services
Evaluate the behaviour, not just the happy path.
An AI feature needs more than a successful demo. Akkenna helps define evaluation cases, review model responses and test the application around them, so your team can understand observed quality, failures and the limits of the evidence.
Based in Peelamedu, Coimbatore · Working with your product and development team.
01 / THE SCOPE
Test the model behaviour and the product around it.
The methods depend on the system: a prediction model, document-search assistant and tool-using workflow need different data and checks. We agree the use case and evaluation criteria before execution.
Task and quality evaluation
Define representative inputs and review outputs against an agreed rubric. Accuracy measures, acceptable answers and error costs depend on the task.
Use case · Evaluation set · Review criteria
Grounding and retrieval checks
For systems using documents or search, examine whether the response uses the intended sources and whether citations support the relevant statements.
Source selection · Citation support · Missing evidence
Robustness and edge cases
Review incomplete, ambiguous, conflicting or unusual inputs. Document where the system should ask a question, decline a task or hand control to a person.
Ambiguity · Boundaries · Recovery
Application and integration testing
Check the surrounding experience: authentication, input handling, response display, timeouts, tool permissions and recovery when a dependency fails.
Permissions · Errors · User feedback
Consistency and regression
Re-run a versioned evaluation set after changes to prompts, models, retrieval or configuration. Compare meaningful differences without treating one run as universal behaviour.
Versions · Repeat runs · Change review
Operational evaluation
Scope latency, usage or cost observations alongside the task. Define monitoring signals and escalation responsibilities appropriate to the deployed system.
Observed timings · Usage · Monitoring plan
02 / A PRACTICAL EXAMPLE
Turn an open-ended answer into reviewable criteria.
This sample evaluation plan is for an internal document assistant. It describes checks a team could agree; it is not an AI benchmark or a report of client results.
Illustrative evaluation matrix — no model score or client result is asserted.
| Scenario | Input condition | What the reviewer examines |
|---|---|---|
| Supported question | The answer exists in an approved source document. | Does the response answer the question, preserve the relevant detail and cite evidence that supports it? |
| Missing information | The approved source set does not contain the answer. | Does the response explain the gap instead of presenting an unsupported answer as fact? |
| Ambiguous request | The request has more than one plausible meaning. | Does the system ask a useful clarification question or make its assumption explicit? |
| Restricted source | The signed-in user is not authorised to view a document. | Do retrieval and output respect the user’s access rules, including summaries and citations? |
| Dependency failure | Retrieval or another connected service is unavailable. | Does the product report the problem and follow the agreed fallback or escalation route? |
The evaluation plan must define data provenance, reviewer responsibilities and how disagreements are resolved. Automated scores can support review but do not replace subject expertise where judgement is required.
03 / HOW THE ENGAGEMENT WORKS
Build an evaluation that can be repeated.
Frame the decision
Identify users, intended tasks, unacceptable outcomes and who is accountable for the feature.Output: use-case boundaries and review priorities.
Prepare the cases
Select representative data, reference answers where appropriate, edge cases and permission to use the material.Output: a versioned evaluation set and data notes.
Define the rubric
Agree what reviewers should assess and how task-specific measures will be interpreted.Output: scoring or review instructions and acceptance criteria.
Run and investigate
Record configuration and outputs, review failures and distinguish model behaviour from application or data problems.Output: an evaluation record and a classified issue list.
Compare and hand over
Review changes, unresolved weaknesses and the checks that should be repeated after future updates.Output: findings, evaluation assets and a regression plan.
04 / WHAT YOU RECEIVE
Keep the evidence connected to the configuration.
The written proposal defines the deliverables, execution effort, review rounds and handover included.
- 01
Evaluation plan
The intended use, test dimensions, assumptions, exclusions and responsibilities.
- 02
Dataset and rubric inventory
The agreed cases, reference material, labels or review criteria, with data provenance recorded.
- 03
Run records
Model or service version, prompt and retrieval configuration, inputs, outputs and relevant execution context.
- 04
Failure analysis
Examples grouped by task quality, grounding, permissions, integration or other agreed categories.
- 05
Review and monitoring recommendations
Known limitations, suggested follow-up checks and the responsibilities for changes after release.
05 / CLEAR RESPONSIBILITIES
Make the limits of the evaluation visible.
Representative data
A test set covers the cases it contains. Discuss under-represented inputs, language variation and changes that could make the set less useful over time.
Human and specialist review
Subject experts should review domain-specific claims and consequential outputs. Specialist security, legal or regulated-domain assessments need an explicit scope.
Authorised environments
Agree which systems and tools may be exercised. Use controlled test accounts and data when an AI workflow can create records, send messages or change external systems.
No blanket assurance
An evaluation does not establish that a model is unbiased, secure, compliant or correct in every situation. Report observed results, methods and remaining uncertainty.
06 / PRIORITISE THE WORK
Match the evaluation to the use case.
A single score rarely describes everything the team needs to know. Start with the task and choose the checks that make a release or improvement decision more informed.
- PredictionDefine target labels, data splits and the cost of different errors with the team responsible for the model.
- GenerationReview usefulness, factual support, completeness and the required response boundaries against a task-specific rubric.
- ActionsCheck permissions, proposed actions, confirmation points, failure handling and recovery in an authorised environment.
07 / RELATED DEVELOPMENT WORK
Applications with real business rules.
Access rules and operational workflows matter before an AI feature is introduced. This is a related custom-application project from Akkenna’s portfolio; it is not presented as an AI-testing engagement or a model-evaluation result.
Win Energie
A custom CRM project covering installer matching, quotation workflows, different user roles and shared documentation.
Explore the development project08 / QUESTIONS BEFORE WE BEGIN
Make the scope clear before testing starts.
Can you evaluate an LLM or document-search assistant?
We can discuss an evaluation built around its intended tasks, approved source material and application behaviour. The scope identifies datasets, rubrics, configuration, reviewers and repeatable checks.
Is AI testing the same as ordinary software testing?
It includes conventional application checks, but model outputs may also need task-specific evaluation and human review. The product, data, model and connected services can each create different failure modes.
Can you eliminate hallucinations or bias?
No universal elimination claim is appropriate. We can scope checks for unsupported outputs or differences in performance on selected cases, document the method and report findings and limitations.
Do you need our production data?
Start with representative sample or redacted data. Access, permission, storage and retention should be agreed before any sensitive material is shared.
Can you compare model or prompt versions?
Yes, when the configurations and evaluation cases can be recorded and repeated. The comparison should identify changed conditions and avoid drawing broad conclusions from a small sample.
Does this include a security or compliance certification?
No certification is implied. Security testing and any sector-specific assessment must be separately defined with the appropriate specialists and requirements.
09 / START WITH THE REQUIREMENT
Tell us what the AI feature needs to get right.
Share the task, intended users, application context and the decision you need the evaluation to support. We will help define the cases, reviewers and deliverables.
Akkenna · Peelamedu, Coimbatore · +91 99523 25513