Farmers often switch between Twi, English, and technical terms when asking for advice on their cocoa farms. AgriVoice will test a direct Twi selection process against a method that uses machine translation to ensure the most accurate and safe answers for these complex queries.
Imagine Akua, a cocoa farmer in a small village outside Kumasi. It’s early morning, the air still cool, and she's walking through her farm, mentally preparing for the upcoming purchasing season. She spots a suspicious discoloration on some cocoa pods. Her neighbor mentioned "black pod disease," but Akua also remembers a COCOBOD extension officer talking about "fungicide application" last season. She pulls out her phone and records a voice note for AgriVoice: "My cocoa pods, they have some dark spots. Ɛyɛ black pod anaa? (Is it black pod?) What kind of aduro (medicine/chemical) should I use, and when can I fa yɛn nnɔbae to gua (sell our harvest) after spraying?" She switches between Twi for the disease and general terms, and English for the technical chemical concept, all while mentally calculating the impact on her income for the coming purchasing season. She needs a clear, reliable answer right now, before the disease spreads or she inadvertently contaminates her harvest. The stakes are her family's income.
The Challenge of Mixed-Language Queries
Akua's question highlights a common reality in Ghana: conversations often blend local languages with English, especially when discussing technical topics like farming or health. For an AI voice assistant like AgriVoice, this presents a significant challenge. Does the system first translate Akua's entire voice note into English, then process that English query to find an answer, and finally translate the answer back into Twi? Or can it understand and respond directly in Twi, even with the English and technical terms mixed in?
The traditional approach involves a machine translation (MT) step. The user speaks, their speech is converted to text, that text is translated into a common language (like English) for processing by a large language model (LLM), and then the LLM's response is translated back into the user's language for speech synthesis. While this seems logical, it introduces potential points of failure. As we've seen, machine translation can silently swap critical domain-specific nouns. "Cocoa" could become "chicken," leading to dangerously inaccurate advice for a farmer like Akua. When lives and livelihoods depend on precise information, such errors are unacceptable.
Our Approach: Direct Twi Selection
To address this, AgriVoice is rigorously testing two paths for handling farmer questions. The first path incorporates the machine translation step. The second path, however, aims to bypass machine translation entirely. In this direct Twi selection approach, Akua's voice note is transcribed into Twi text. An advanced LLM, specifically trained to understand nuanced Twi, then directly interprets her question, including the English code-switches and purchasing-season context. This LLM's role is not to generate new agronomic advice but to select from a set of pre-approved, expert-reviewed knowledge blocks. These blocks, already translated and vetted by farming-aware Twi speakers and Ghanaian agronomists, ensure that the information provided is both accurate and safe.
If Akua asks about black pod disease, the LLM identifies the relevant block on cocoa diseases, retrieves the safe, Twi-spoken advice on identification and treatment, and that advice is then converted into her language using our custom Twi voice. If her question involves chemical application, the system is designed to either provide verified information or, if details like dosage or re-entry intervals are missing, to escalate the query to a human extension officer. This ensures Akua never receives unsafe or unverified chemical advice, especially crucial as the purchasing season approaches. What Happens When a Farmer Hears the Reply but Still Cannot Act Safely? discusses this in more detail.
Testing for Safety and Accuracy
Our pilot with 20-50 cocoa farmers is designed to compare these two approaches head-to-head. We've collected about 40 real Twi questions from farmers, covering common issues, ambiguous phrasing, pesticide inquiries, out-of-domain topics, code-switched language, and even questions with ASR errors. Each question has a "gold standard" answer assigned by a Twi-speaking agriculture reviewer.
We'll measure several factors:
- Exact-set accuracy: Does the system select precisely the right information blocks?
- Any-correct-block recall: Does it at least find one correct block, even if it misses others?
- Unsafe-extra-block rate: Crucially, does it ever provide additional, potentially unsafe information?
- Escalation accuracy: Does it correctly identify and escalate questions that require human intervention?
- Latency and cost: How fast is each method, and what are its operational costs?
This rigorous comparison will help us determine if direct Twi selection performs as well as, or better than, the MT-based approach, especially on critical safety metrics. Akua, back on her farm after getting clear advice from AgriVoice, feels a weight lift. She knows exactly which chemical to use, how to apply it safely, and when her harvest will be ready for market, securing her family's income for the purchasing season. The clear, direct Twi response, with specific guidance, means she doesn't have to second-guess what an English translation might have missed.
Comments
No comments yet.