Course Materials RAG System - Test Cosmin

A Retrieval-Augmented Generation (RAG) system designed to answer questions about course materials using semantic search and AI-powered responses.

This application is a full-stack web application that enables users to query course materials and receive intelligent, context-aware responses. It uses ChromaDB for vector storage, Anthropic's Claude for AI generation, and provides a web interface for interaction.

• Python 3.13 or higher
• uv (Python package manager)
• An Anthropic API key (for Claude AI)
• For Windows: Use Git Bash to run the application commands - Download Git for Windows

1. Install uv (if not already installed)

   curl -LsSf https://astral.sh/uv/install.sh | sh

2. Install Python dependencies

   uv sync

3. Set up environment variables

   Create a .env file in the root directory:

   ANTHROPIC_API_KEY=your_anthropic_api_key_here

Use the provided shell script:

chmod +x run.sh
./run.sh

cd backend
uv run uvicorn app:app --reload --port 8000

The application will be available at:

• Web Interface: http://localhost:8000
• API Documentation: http://localhost:8000/docs

Requires Docker with Compose. You still need a .env file at the project root containing ANTHROPIC_API_KEY.

docker compose up --build

The app is served at http://localhost:8000. Notes:

• On first startup the container downloads the all-MiniLM-L6-v2 embedding model from HuggingFace (~90 MB). It is cached in the hf_cache volume, so subsequent starts are offline and fast.
• The ChromaDB vector store is persisted in the chroma_data volume.
• ./docs is mounted into the container; any .txt/.pdf/.docx course files placed there are ingested on the next startup.

To run the image directly without Compose:

docker build -t course-rag .
docker run --rm -p 8000:8000 --env-file .env \
  -v course-rag-chroma:/app/backend/chroma_db \
  -v course-rag-hf:/app/.cache/huggingface \
  course-rag

The project uses modern Python development tools to maintain code quality and consistency.

• Black — Automatic code formatting for consistent style (line length: 100 characters)
• Ruff — Fast Python linter for code quality checks (unused imports, style issues, etc.)
• Pytest — Testing framework for comprehensive test suite

Convenient shell scripts are provided in the scripts/ directory:

bash scripts/format.sh

Automatically formats all Python code in backend/ and the project root.

bash scripts/check-format.sh

Verifies that code matches Black's formatting style. Useful in CI/CD pipelines.

bash scripts/lint.sh

Runs Ruff to check for style issues, unused imports, and potential bugs.

bash scripts/test.sh

Executes the full pytest test suite.

bash scripts/quality.sh

Runs formatting check, linting, and tests in sequence. Provides a comprehensive quality report.

If you prefer to run tools directly:

# Format with Black
uv run black backend main.py

# Format with Black (with custom line length)
uv run black backend main.py --line-length 100

# Check formatting without changes
uv run black backend main.py --check

# Run Ruff linter
uv run ruff check backend main.py

# Fix common linting issues automatically
uv run ruff check backend main.py --fix

The project includes a comprehensive test suite to verify RAG system functionality and prevent regressions.

Run a fast diagnostic that confirms the system configuration and identifies issues:

python3 backend/tests/test_bug_diagnosis.py

This diagnostic:

• Verifies Config.MAX_RESULTS is set to a reasonable value (not 0)
• Traces the bug flow end-to-end
• Runs without requiring ChromaDB or heavy ML dependencies

Install test dependencies:

uv add --dev pytest

Run all tests:

cd backend
uv run pytest tests/ -v

Run specific test file:

cd backend
uv run pytest tests/test_search_tools.py -v

Run specific test:

cd backend
uv run pytest tests/test_rag_system.py::TestRAGSystemConfigRegression::test_config_max_results_is_positive -v

• ✓ Content queries return empty results when MAX_RESULTS=0
• ✓ Content queries return real search results when configured correctly
• ✓ Tool-calling doesn't loop beyond one round (one-round-trip constraint)
• ✓ CourseSearchTool populates sources correctly
• ✓ Filtering by course name and lesson number works
• ✓ Session history tracking functions properly

The system was previously broken when Config.MAX_RESULTS was set to 0. This caused ChromaDB to be queried with n_results=0, always returning empty results. The config has been fixed to MAX_RESULTS: int = 5 (matching VectorStore's default). The test suite includes a regression test to prevent this bug from being re-introduced.

For details on the investigation, see TEST_REPORT.md and TESTING_SUMMARY.md.