Why generic LLMs miss agriculture

What does a language model trained on the open internet actually know about a soybean field in Latur? A great deal, it turns out, and still not enough to be trusted with one.

KissanGPT has been in users’ hands for five months now. The product wraps GPT-3.5 and Whisper around a voice interface and a thin agricultural prompt layer, and it works well enough for a substantial fraction of farmer queries: well enough that the system now handles tens of thousands of queries a month.

Inside that “well enough” is a more interesting story. In about a third of the queries we see, the generic model under the hood fails in specific, reproducible ways that no amount of prompt engineering fully fixes. These failures are why we are now building a domain-specific language model of our own. They are also why the conversation about “ChatGPT for agriculture” needs to land somewhere closer to the logs than to the marketing.

This is a record of what we observed in 2023, using GPT-3.5 and the general-purpose systems available at the time. Later models will improve, but the deployment lesson remains: an answer can be fluent and still miss the local agronomy, vocabulary, product availability, or timing that makes it useful. The patterns and examples below come from real farmer conversations. Some examples combine details from more than one conversation to protect privacy without changing the underlying failure mode.

Failure mode 1: confident wrong answers on regional specificity

A farmer in Latur asks (in Marathi): “should I sow my soybean early this year given the monsoon delay?”

GPT-3.5’s answer cites the standard recommended sowing window for soybean in India (with the arrival of the monsoon, late June into July). It does not condition on the fact that Marathwada has its own sub-window because of the dryland conditions, that this year’s IMD forecast is for a delayed onset, that the farmer’s specific district has had below-normal pre-monsoon showers, or that the agricultural extension service in Maharashtra publishes a district-level advisory for exactly this decision every season.

The model’s answer is correct in a textbook sense and wrong in a deployable sense. The textbook answer might cost the farmer a fortnight of unnecessary anxiety and possibly a too-early sowing on dry soil.

Why this happens: GPT-3.5’s training data has Indian agricultural content but at the country-aggregate level, not at the agro-climatic-zone level. The level of specificity the answer needs is below the level of specificity the model has.

Failure mode 2: wrong language register

A farmer in Bihar wants to know what is eating his mango tree. He has no name for the pest, so he describes it, in the Hindi of his district: green, about a finger-joint long, curls up when touched, eating the young leaves from the edge inward.

The model cannot work from the description. It guesses at a species, responds in formal textbook Hindi with the standard agronomy taxonomy and a Latin binomial (Helicoverpa armigera, in the transcript I am looking at), and appends a control schedule for the pest it guessed. The response is linguistically correct and doubly unusable. The identification is a guess dressed up as an answer, and the register is one the farmer cannot extract information from: his vocabulary does not include Latin nomenclature, and the formal sentence structure is not how his neighbour or the local agronomist would talk to him.

The gap here is a training-data gap. GPT-3.5 learned its Hindi from news and Wikipedia. The dialectal Hindi farmers speak, with regional pest vocabulary in place of Latin names and spoken sentence structure in place of textbook prose, is undertrained, and a model that has never seen a pest described the way a farmer describes one cannot map the description to the organism.

Failure mode 3: generic product answers when the farmer needs a local one

The most dangerous failure is the helpful one. Asked what to use for bollworm on a cotton crop, the model gives a generic chemistry answer or names a brand without knowing whether that brand is available and appropriate in the farmer’s state. The farmer is usually not asking for an abstract active ingredient. They are trying to work out which locally available product to ask for, and whether it fits the crop and situation in front of them.

That gap cannot be solved by sounding more confident. Brand availability, product labels, and regional restrictions change. A model trained on a static corpus has no reliable view of the shelf at a local retailer or the current product information for a particular state.

The safer product behavior is to retrieve current local product information before becoming specific, and to ask for the crop, location, stage, and observed problem first. A general-purpose model without that live context should not turn a generic answer into a brand recommendation.

Failure mode 4: stage-conditional advice given as generic advice

“Should I apply nitrogen to my rice now?” Eight words, and the correct response is a question back, because the answer depends on the growth stage of the rice (vegetative vs reproductive vs ripening), the variety being grown, the soil’s current nutrient status, the season’s rainfall to date, and the basal application that was already applied.

GPT-3.5 has none of this context and asks for none of it. It answers with a general fertiliser schedule for rice, broadly correct as a textbook answer, and textbook answers in agriculture, applied without conditional reasoning, drift toward over-application of inputs. Over-application is both expensive for the farmer and environmentally bad. The aggregate effect of a generic model giving agronomic advice without conditional checks is a steady push toward over-input.

A domain model trained with explicit reasoning over conditional context would ask the follow-up questions before answering. A generic model trained on general instruction-following defaults to answering.

Failure mode 5: the long tail of crops

The model is competent on rice, wheat, cotton, sugarcane, maize, and soybean. It is noticeably worse on millets (ragi, jowar, bajra), pulses (pigeon pea, chickpea, lentil), horticulture (mango, pomegranate, banana, vegetables), and minor commodity crops.

The asymmetry is structural. The training corpus contains more text about major commodity crops than about the long tail. A farmer in southern Karnataka growing ragi gets noticeably worse answers than the same farmer would have gotten growing rice. The same farmer growing pomegranate gets answers that frequently confuse pomegranate management with citrus or grape management, which are botanically distinct enough that confusing them is not a minor error.

Training distribution mirrors source distribution, and that is the whole explanation. Agronomy publications in English skew toward commodity crops because that is where research funding has concentrated globally, while the Indian regional sources that cover the long-tail crops well are largely not in the English-language corpus.

The resulting build list

None of these failures is fatal. KissanGPT works around them through prompt engineering, conservative defaults, fall-through to “please consult an agronomist” responses, and a degree of user education about which questions to expect good answers on. The product is shipping value in spite of all five.

Cataloguing these failures gives us a build list. The generic model supplies the language. We still have to add the agronomy, regulation, dialect vocabulary, and conditional reasoning deliberately. A larger pretraining run will not do that work for us.

Five months of KissanGPT turned that list from a theory into a specification for the domain model we are building. We will report the results when we have them.