Dhenu 1.0: training a domain LLM for Indian agriculture

This morning at the GPAI Summit in Delhi we announced Dhenu 1.0, a 7B-parameter language model built for Indian agriculture. The weights and model card go up on Hugging Face next week under KissanAI/Dhenu-1.0-full-sft, along with the tokenizer, the training recipe, and the evaluation set. This post is for engineers who want to know what is inside the model, what is not inside it, and how we built it.

The problem in the KissanGPT logs

Nine months of KissanGPT logs gave us the reason to train Dhenu. A general-purpose model, even GPT-4, could know the agricultural fact and still mishandle the question because it did not understand the grammar of how a farmer asked it.

A farmer in Marathwada will ask, “kapsala paani aaj de ka udyatla deu” (should I water the cotton today or tomorrow). The model has to know that this is irrigation timing for Gossypium hirsutum in a Maharashtra district that sits in a particular monsoon pattern, that the question is conditional on yesterday’s rain and tomorrow’s forecast, that the answer depends on which BT variety and what growth stage, and that the answer must come back in the same Marathi the question came in.

The general model handles the language. Dhenu is our attempt to handle the agricultural context wrapped inside it.

Training data: where we spent the most time

Most of this project was data work. The training run was the smaller job.

We assembled 300,000 instruction-response pairs in English and Hindi, with another 80,000 supplementary pairs across seven Indic languages we will release in the next checkpoint. Of those 300k pairs:

• 145,000 are agronomy advisories, drawn from publicly available extension publications (ICAR, KVK bulletins, state agriculture department PDFs), normalized and converted into question-answer form
• 78,000 are crop-protection conversations, including pest and disease identification, IPM steps, and pesticide application guidance keyed to crop and growth stage
• 41,000 are mandi-price and market commentary pairs, taken from Agmarknet historical data joined with trade-paper interpretations
• 22,000 are scheme-and-eligibility pairs covering PM-KISAN, KCC, crop insurance, and 17 state-level schemes
• 14,000 are weather-advisory pairs grounded in IMD bulletins

Reviewing three hundred thousand pairs by hand would have taken months, so the pipeline leans on synthetic generation from our curated knowledge base and the KissanGPT conversation logs, with agronomists reviewing samples from every batch. A batch that failed review got regenerated, and roughly one draft pair in eight was thrown out because the source was outdated, regionally wrong for the question, or written for a different crop’s life cycle than the question implied.

The training run was easy to schedule. This curation work was not, and it is where the model’s competence came from.

Architecture and training

Dhenu 1.0 is a 7B-parameter decoder-only transformer built on OpenHathi, Sarvam AI’s Hindi-English base model, which is itself a Llama-2 7B adaptation. We chose OpenHathi over a raw Llama-2 base because its Devanagari tokenization and Hindi pretraining meant our compute went into agriculture rather than into teaching the model the script. On that base we ran continued pre-training on a 4.2B-token agricultural corpus we built (extension publications, agronomy textbooks, ICAR reports, trade press, multilingual mandi commentary), then instruction tuning on the 300k pairs above using a standard SFT pipeline. NimbleBox.ai worked with us on the data curation pipeline that fed both stages. We did not do RLHF on this release. The next checkpoint will include preference data from a panel of agronomists.

Compute is the least interesting line item. Instruction-tuning a 7B model is a few A100s for a working day, not a data-centre story; the heavy lifting sits in the base model’s pre-training, which is exactly what building on OpenHathi bought us. Nothing in the training math is unusual.

Where we departed from a standard recipe:

1. Code-switched training samples. Many of our SFT pairs are deliberately in Hinglish or Marathi-English code-switched form, because that is how farmers ask questions in voice transcripts. A model trained only on clean Hindi misses 40% of real input.
2. Negative examples for hallucination. We included 11,000 “I do not know” responses paired with questions that exceed the model’s training domain (specific pesticide brands not in our corpus, regional schemes we did not include). This is the cheapest mitigation we know of against confident wrong answers in a domain where wrong answers cost a season’s income.

Evaluation: what we measured and what we did not

Evaluation is the weakest part of the story right now, and I would rather say that plainly than dress it up. We reviewed a few thousand held-out questions by hand against GPT-4 Turbo and Llama-2-7B-chat, looking at factuality, regional appropriateness, actionable advice, linguistic naturalness, and safety. Dhenu reads clearly better on Hindi and Hinglish agronomy questions; the general models still win on fluent English exposition. There is no public benchmark for this domain yet, and until one exists, every number in this space, ours included, deserves suspicion.

What we did not measure well: long-horizon dialogue (multi-turn conversations where the farmer reveals context across messages), and reasoning chains for crop scheduling. Both are next-checkpoint problems.

What is open and what is next

The model weights, the tokenizer, the eval set, and the training config go up at huggingface.co/KissanAI next week, once the model card and the eval writeup are finished. The dataset itself will not be fully open at first, because the SFT pairs include attributions to advisories that require permission to re-release in bulk. We are negotiating those releases.

Next on the bench: the seven additional Indic languages in instruction-tuned form (the 80,000 supplementary pairs above are the start of that set), writing up the eval methodology so others can check our claims, and a distilled checkpoint around 1.3B parameters that can run on a phone for offline regions. The deployment math for India does not work without that last one.