DakiKobo — AI Agricultural Advisor for Burkina Faso 🌾

DakiKobo is a French-language AI assistant for smallholder farmers in Burkina Faso. It uses a Retrieval-Augmented Generation (RAG) pipeline restricted to reviewed local sources. Currently only the CILSS and MAERAH/OAPH orientation summaries are eligible; technical guides remain candidates pending review. Unsupported field questions must receive an honest fallback, not an invented recommendation.

The focus crops are mil (millet), sorgho (sorghum), maïs (maize), niébé (cowpea) and arachide (groundnut). All output — answers, UI labels and voice — is in French, and the interface is mobile-first for use on phones.

Live Space: https://kimcomehome-dakikobo.hf.space/

Deployment identity can be checked with python3 scripts/verify_deployment.py after fetching both remotes. It compares the accepted GitHub and HF trees and checks the live version and health. See the release checklist for the offline dry run, LFS normalization, and required release records. Field release remains REPORTÉE pending the recorded human and hosting gates.

1. Contexte parcelle (optionnel) — culture, stade, lieu ; activez Français simple pour des phrases plus claires.
2. Question texte — ex. « Quand semer le mil ? » → carte Conseil agricole + sources.
3. Engrais — ex. « Dose d'engrais pour le sorgho » → conseil déterministe, doses chiffrées suspendues jusqu’à validation agronomique + confirmation par un agent.
4. Photo de feuille — dépistage prudent (pas un diagnostic).
5. Sources & limites — preuves, météo/sol indicatifs, confirmation terrain obligatoire.

Script détaillé : DEMO_SCRIPT.md.

Collecte de données (plus tard, hors code) : Data/reviews/DATA_COLLECTION_TASKS.md.
Continuité agent : SESSION.md.

Ce que DakiKobo n'est pas : un oracle de rendement, un diagnostic officiel, ou un remplacement de l'agent agricole.

• Grounded French answers — RAG over a local document corpus; off-topic questions fall back to an honest "je ne sais pas" instead of hallucinating.
• Source citations and evidence ledger — one scored retrieval grounds both the generated answer and its ranked source cards; every kept or dropped chunk is recorded by salted question hash with its score and citation decision for private offline evaluation.
• Fast inference — Groq-hosted openai/gpt-oss-120b, with reasoning tokens hidden so answers stay short and farmer-readable.
• Multilingual retrieval — paraphrase-multilingual-MiniLM-L12-v2 embeddings for good French matching, stored in a persistent ChromaDB (built once, fast on restart).
• Hosted warm-up — the Docker Space can prepare RAG in the background after startup so the first public question is less likely to pay the full indexing cost.
• Shared answer and context cache — SQLite WAL serves repeat grounded answers immediately and shares weather, soil, and privacy-safe ops state across two threaded workers; Chroma startup is serialized across workers to prevent concurrent schema migrations.
• Voice output (TTS) — answers can auto-play in French via gTTS and be replayed from their answer bubble.
• Voice input (STT) — records a short browser audio clip and transcribes it with Groq Whisper, with native browser speech recognition as a fallback.
• Quota-safe public examples — one-tap demo answers for text, fertilizer guidance and a sample image case, without spending live API calls.
• Focused mobile UI — examples stay visible, while weather and soil tools sit behind an Outils drawer so the conversation remains the main workspace.
• Trust panel — a compact Sources & limites dialog explains evidence, approximate signals, and required field confirmation.
• Safety-gated fertilizer guidance — fertilizer intent and crop handling remain deterministic, but exact numeric doses are withheld online and offline until an agronomist verifies each value against an exact source, formulation, zone and timing.
• Offline-first PWA — after one connected load, the app shell, registry, demo examples and previously saved answers remain available; fertilizer questions receive the same safety-gated response offline, while uncached questions receive an honest offline notice.
• Leaf disease screening (optional) — upload a leaf photo for a hedged French screening via Gemini Vision, with a "ceci n'est pas un diagnostic" disclaimer (requires a Gemini API key).
• Weather-aware field signals — Open-Meteo rainfall, ET0, soil moisture and short-term forecast cards for selected Burkina Faso locations.
• Soil-aware fertilizer context — SoilGrids texture, organic carbon, pH and retention-risk classes combined with deterministic fertilizer guidance.
• Private field journal — explicit consent, anonymous browser ownership, 90-day retention, deletion, optional re-encoded follow-up photos, and a seven-day in-app follow-up date. Legacy unowned records are never exposed through public journal routes. A saved case also retains and replays its source cards, including each source's declared scope shown under the French label « Portée et limites »; cases saved before this feature simply replay without sources (their sources are never reconstructed).
• Français simple — plain-language mode is enabled by default, with a field glossary (NPK, microdose, OAPH, etc.).
• Reviewed-source ingestion — Markdown and PDF candidates fail closed unless their metadata records an approved owner or agronomist review; startup no longer ingests network content directly.
• Shared resource budgets — per-client and global SQLite limits work across server workers without retaining raw client addresses.
• Field weather enrichment — known locations can attach rainfall / water-stress signals to answers.
• Mobile-first responsive UI — fills the screen on phones, input pinned to the bottom.

• Python 3.10 or 3.11
• A Groq API key (from the Groq console)
• uv recommended for fast environments (plain venv + pip also works)

Using uv (recommended):

uv venv .venv --python 3.11
source .venv/bin/activate
uv pip install -r requirements.txt

Or with standard tools:

python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

Export secrets into the local process environment. The Flask web process does not load secret files:

export GROQ_API_KEY='your_real_key_here'

Optional — leaf disease screening needs a Google Gemini key (from Google AI Studio):

export GEMINI_API_KEY='your_gemini_api_key_here'

For hosted deployments, set the same names in the provider's Secrets or Environment panel. Do not upload a .env or place one below public_html.

Verify it works anytime with:

python scripts/test_gemini.py

Optional overrides (defaults in config.py are fine for development):

# LLM_MODEL=openai/gpt-oss-120b
# LLM_REASONING_FORMAT=hidden
# LLM_REASONING_EFFORT=low
# APP_VERSION=0.1.0
# LOG_LEVEL=INFO
# LLM_MAX_TOKENS=1024
# LLM_TEMPERATURE=0.1
# LLM_TIMEOUT_SECONDS=30.0
# LLM_MAX_RETRIES=1
# STT_MODEL=whisper-large-v3-turbo
# STT_LANGUAGE=fr
# STT_TIMEOUT_SECONDS=30.0
# STT_MAX_RETRIES=1
# MAX_AUDIO_UPLOAD_MB=5.0
# GEMINI_MODEL=gemini-2.5-flash
# GEMINI_TIMEOUT_SECONDS=45.0
# WEATHER_TIMEOUT_SECONDS=10.0
# SOIL_TIMEOUT_SECONDS=12.0
# WEB_FETCH_TIMEOUT_SECONDS=15.0
# FIRECRAWL_API_KEY=your_firecrawl_api_key_here
# FIRECRAWL_HTTP_TIMEOUT_SECONDS=45.0
# FIRECRAWL_SCRAPE_TIMEOUT_MS=60000
# FIRECRAWL_MAX_RETRIES=2
# FLASK_DEBUG=true
# PREFER_MARKDOWN_KB=true # use Data/markdown before PDF fallback
# REBUILD_VECTORSTORE=true   # force a fresh index rebuild

Place reviewed Markdown documents under Data/markdown/. DakiKobo uses this Markdown corpus first because it is smaller and cleaner than extracting PDFs at startup. Keep original Burkina Faso agriculture PDFs anywhere under Data/ as source files and as a fallback; subfolders are discovered recursively.

python app.py

Open http://127.0.0.1:5000 in your browser.

First run: DakiKobo builds the vector index from the reviewed Markdown corpus and saves it to chroma_db/. On CPU this one-time build can take several minutes. Subsequent starts load the saved index and are fast. To rebuild later (e.g. after adding documents or changing the embedding model), start with REBUILD_VECTORSTORE=true to replace the saved index.

• Ask a question about Burkina Faso agriculture in French.
• Quick chips above the input send common questions in one tap.
• Voice output: tick "Activer la lecture vocale" to hear answers read aloud.
• Voice input: tap the microphone, speak, then tap again to stop or wait for auto-stop.
• Save a useful answer: accept the 90-day private-journal notice, then use 👍 / 👎. DakiKobo links the saved answer and later field outcome to the exact retrieval decisions.
• Return later: Mes conseils lists only entries owned by the current anonymous browser session. GET /journal/due returns that same owner's due digest without question or answer text.
• Delete: remove one or every server-side journal entry separately from cached answers and preferences stored on the device. Clearing browser cookies loses access to the anonymous journal.

Run a repeatable public demo check against the Hugging Face Space:

python scripts/evaluate_rag.py

The script waits for /healthz, exercises RAG, fertilizer, weather, soil, and off-topic safety cases, then writes reports/rag_eval_results.md.

The same evaluator is wired into GitHub Actions as .github/workflows/hf-smoke.yml. It can run manually from the Actions tab and also runs daily on schedule, uploading the Markdown report as an artifact. The Docker image is checked by .github/workflows/docker-build.yml, which builds the Space container and smoke-tests /healthz without pushing an image.

Two further workflows exercise acceptance rehearsals in CI:

• .github/workflows/browser-rehearsal.yml runs a headless-browser (Chromium) rehearsal of the UI at 320 px and 1280 px against a synthetic fixture that calls no model or provider, and uploads screenshots and results as artifacts. Playwright is a test-only dependency (requirements-browser.txt) and is kept out of the production requirements. This is headless-browser evidence only, not physical-phone or participant testing.
• .github/workflows/docker-journal-rehearsal.yml builds the production image and runs a bounded synthetic rehearsal proving a private journal saved into a local bind mount survives a container replacement. This is local bind-mount persistence evidence only, not hosting-provider disk durability or a real browser.

Useful options:

python scripts/evaluate_rag.py --base-url http://127.0.0.1:8005
python scripts/evaluate_rag.py --strict

Firecrawl is available for collecting candidate web pages, but scraped output is not ingested directly. It is written to Data/scraped/pending/ with source metadata and a review checklist. URLs are checked against Data/scraped/source_allowlist.csv by default.

Use the first curated FAO Burkina Faso seed batch:

python scripts/firecrawl_ingest.py --list-seeds
python scripts/firecrawl_ingest.py --seed-batch

Scrape a single allowlisted URL:

python scripts/firecrawl_ingest.py \
  --url https://www.fao.org/in-action/mafap/where-we-work/burkina-faso/en \
  --publisher "Publisher name" \
  --topics "semis, fertilite" \
  --crops "mil, sorgho"

For manual experiments outside the allowlist, pass --allow-unlisted. Do not promote those files without adding a reviewed allowlist row first.

After human review and cleanup, promote the pending file into the active Markdown corpus:

python scripts/firecrawl_ingest.py \
  --promote Data/scraped/pending/20260702_example_abc12345.md \
  --reviewer your_name

Then rebuild the vector store with REBUILD_VECTORSTORE=true.

All tunables live in config.py (overridable via server environment variables where shown). The web process never loads .env; export variables locally or configure them in the hosting provider's Secrets/Variables panel. Never store a secret file inside public_html or the container build context.

Notes on the audio and upload limits:

• TTS storage is bounded. Generated MP3s are named after the answer they speak, so identical answers reuse one file instead of writing a new one every time. TTS_CACHE_TTL_SECONDS drops files by age first; if static/audio/ is still above TTS_CACHE_MAX_BYTES, the least-recently-used files are removed until it fits. The file being served is never evicted.
• The upload limit and the transport ceiling are separate. MAX_IMAGE_UPLOAD_MB / MAX_AUDIO_UPLOAD_MB are the advertised per-file limits checked inside each route. Flask's MAX_CONTENT_LENGTH caps the whole multipart body and is set to the larger per-file limit plus MULTIPART_OVERHEAD_BYTES, so a file exactly at the documented size is not rejected once boundary markers, part headers, and field-context values are counted.

dakikobo/
├── app.py               # Flask entry point + routes (/ask, /feedback, /journal/due)
├── config.py            # Central configuration
├── core/
│   ├── crops.py         # Canonical crop ids, French labels and capabilities
│   ├── places.py        # Canonical place ids, labels and weather coverage
│   ├── retrieval.py     # Citation ranking, confidence and stable chunk ids
│   ├── case_contract.py # Shared field-case data contract
│   ├── case_log.py      # SQLite field journal and evidence ledger
│   ├── llm_chain.py     # LLM + RetrievalQA setup and French prompt
│   └── rag_pipeline.py  # Markdown/PDF ingestion, embeddings, Chroma, TTS
├── templates/index.html # Chat UI
├── static/              # CSS, JS, images, generated audio
├── scripts/             # Utility scripts, including public RAG evaluation
├── reports/             # Generated evaluation reports
├── Data/                # Source PDFs + reviewed Markdown knowledge corpus
└── requirements.txt

• .env, chroma_db/, generated audio, data/feedback.csv and data/case_log.sqlite3* are git-ignored.
• /healthz reports readiness; /version reports app version, commit if exposed by the host, key runtime config flags, and the field-journal schema version.
• Runtime logs are JSON lines with route, status, latency, model/feature, failure type and confidence where available. Raw questions, answers, images and audio are not logged.
• Leaf photos and voice recordings are used only for the requested analysis or transcription. The app warns users not to send faces, names, numbers or personal documents.
• This tool gives general guidance; users should confirm specifics (e.g. fertilizer doses) with a local agricultural extension agent.

Offline answer replay requires a nonempty X-DakiKobo-Safety revision matching the saved policy marker. Missing, blank, or mismatched revisions are refused; reconnect to obtain a current sourced answer.

Audio cleanup also removes abandoned partial writes on the next cleanup pass. It skips symlinks and active writes using POSIX advisory locks (Linux/macOS). The MP3 size cap excludes active/recent partials and preserves the current response file; it is not a hard quota for all concurrent temporary writes. Source-review preparation: five-crop coverage matrix.

If a saved audio file expires or playback fails, the replay control displays a French status message and keeps the text advice visible. Users can check their connection and retry. The browser does not regenerate the deleted recording.