A recording that performs well in a speech lab can fail on a cocoa farm because the field changes the sound, the language and the stakes. A voice service needs evaluation with real farmer speech, farm noise, code-switching and safe escalation before it can be trusted with advice.
At 4:40 in the afternoon, Kofi stands at the edge of his cocoa plot near Kumasi with his phone close to his mouth. A motorbike passes on the road behind him. Leaves scrape together above the pods he has been checking since lunch. He sends a voice note in Asante Twi, then switches into English for the name on a pesticide packet because that is how he knows it.
The recording reaches the system with wind, motorbike noise and a phrase that changes language halfway through. In a lab, the same question might be spoken slowly into a quiet microphone, with a clean transcript waiting to score against. Here, one mistaken word could turn a question about cocoa into something else entirely. If the system confidently selects the wrong advice, Kofi could spray the wrong thing or enter a treated plot too soon.
That possibility has to remain open long enough for the system to do the responsible thing: recognise uncertainty and send the question to the named extension officer instead of guessing. A helpful answer matters. A correct escalation matters too.
A clean recording measures only part of the job
Lab recordings are useful. They let a team compare speech recognition models, find obvious failures and repeat a test under the same conditions. Neuralis has used held-out read speech to compare its Twi speech stack, including a result where Meta Omnilingual ASR outperformed its fine-tuned Whisper model on that material.
Read speech is still a narrow slice of how a farmer asks for help. A person reading a prepared sentence knows where it starts and ends. They are less likely to pause to inspect a leaf, call to someone nearby, restart a sentence or use the English product name that comes naturally in the moment.
The gap can be small in audio and large in meaning. A recogniser may catch most words yet miss the one word that identifies the crop, the product or the question being asked. In cocoa advice, that is enough to make a polished response unsafe.
This is why AgriVoice’s pilot entry gates include at least 20 real farmer questions recorded and transcribed for field ASR measurement. The purpose is not to collect a flattering score. It is to find out where the system stops hearing the question people actually ask.
Noise is only one source of uncertainty
Farm noise is easy to picture, but it is not the whole problem. Everyday speech carries tonal variation, local pronunciation, false starts and borrowed English words. A farmer may say a Twi sentence around “fungicide,” “COCOBOD,” a brand name or a number. The system has to treat that mixed language as part of the question, not as an error to erase.
Numbers deserve special care. Advice involving dosage, timing, re-entry or pre-harvest intervals cannot survive a casual transcription mistake. Three AgriVoice chemical blocks remain withheld because a Ghanaian agronomist has not yet verified the required dosage and interval information. Refusing to invent an answer is the current safe behaviour.
Translation introduces another risk. In early work, `kokoo`, the Twi word for cocoa, was translated as “chicken.” The sentence sounded fluent while the domain noun had changed. That is why the team is evaluating direct Twi block selection against a Twi-to-English route, including a second pass on real ASR transcripts. The comparison must track unsafe extra selections and escalation accuracy alongside ordinary accuracy.
For a related example of why a single swapped word can change the whole task, read What Happens When Cocoa Translates as Chicken?.
Field evaluation should reward safe uncertainty
A system should not be judged only by how often it answers. It should also be judged by how often it correctly says that it cannot answer safely.
AgriVoice is designed so the language model selects from reviewed advice blocks rather than writing agronomy guidance from scratch. When no reviewed block fits, when the audio is too uncertain or when a chemical question lacks verified content, the correct path is human escalation. That design creates a testable standard: did the service give a reviewed answer, or did it route the farmer to someone accountable?
Back at the plot, Kofi’s voice note does not need a dramatic AI response. It needs a reliable outcome. If the audio is clear and the question matches reviewed content, he should hear advice he can understand. If the product name or spray timing is unclear, the question should remain open for the extension officer.
The next morning, Kofi can return to the same pods with a response that has a clear source, or with a person responsible for resolving the uncertainty. Silence and confident guessing both leave him alone with the risk.
Build the test set where the decision happens
The practical work starts with consented recordings from the people the service is meant to support. Collect questions across common cocoa concerns, ambiguous requests, pesticide topics, out-of-domain questions, code-switched speech, damaged audio and questions that need more than one reviewed block.
Then transcribe them carefully, have a Twi-speaking agriculture reviewer assign the correct answer blocks or escalation outcome, and record disagreements instead of hiding them. Forty questions can provide a directional read. It cannot prove that two approaches are equivalent when they differ by one or two cases.
Kofi’s message should be tested in the conditions that shaped it: wind in the trees, an interrupted sentence, a product name in English and an urgent decision waiting at the other end. That is where a voice service earns the right to answer.
Comments
No comments yet.