querybot

A text-to-SQL system I built from scratch to learn how language models actually work under the hood — not just calling an API, but fine-tuning a real model on my own machine.

The idea is simple: you give it a database schema and ask a question in plain English, and it writes the SQL and runs it for you. The whole thing runs locally on a 6GB GPU with no API keys, no cloud, no subscription.

---

What this actually is

I wanted to understand the full pipeline — data prep, model fine-tuning, inference, evaluation — so I built each piece myself instead of using a pre-made solution. It uses:

• Phi-2 (2.7B parameter model by Microsoft) as the base
• QLoRA to fine-tune it without needing 80GB of VRAM
• 70,000+ SQL examples from the sql-create-context dataset (built from Spider + WikiSQL)
• SQLite for actually running the generated queries

The fine-tuned model runs on a laptop GPU (tested on RTX 4050 6GB).

---

Project layout

querybot/
├── config.yaml              ← hyperparameters, paths, LoRA settings
├── run.py                   ← single entry point for all stages
├── requirements.txt
│
├── src/
│   ├── csv_to_sqlite.py     ← convert any CSV/JSON to SQLite with type inference
│   ├── prepare_spider.py    ← download and format training data
│   ├── baseline.py          ← test base model before fine-tuning
│   ├── train.py             ← QLoRA fine-tuning with SFTTrainer
│   ├── inference.py         ← run the fine-tuned model
│   ├── inference_checkpoint.py  ← test mid-training checkpoints
│   ├── evaluate.py          ← exact match + execution accuracy
│   └── utils.py             ← shared helpers
│
├── scripts/
│   ├── check_env.py         ← verify GPU, packages, CUDA before training
│   ├── make_sample_data.py  ← generate a sample school dataset for testing
│   └── merge_model.py       ← merge LoRA adapters into base for deployment
│
└── data/
    ├── raw/                 ← your own CSV files go here
    ├── processed/           ← train.jsonl and val.jsonl (generated)
    └── spider/              ← cached dataset files

---

Setup

Requires Python 3.10+ and an NVIDIA GPU. Tested on RTX 4050 Laptop 6GB.

git clone https://github.com/YOUR_USERNAME/querybot
cd querybot

python -m venv venv
source venv/bin/activate

# install PyTorch with CUDA first
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121

# then everything else
pip install -r requirements.txt

Verify your setup:

python scripts/check_env.py

All checks should pass before continuing.

---

Running it

Step 1 — Download and prepare training data

python run.py --stage data

Downloads ~22MB of SQL examples from HuggingFace and builds train.jsonl (70k rows) and val.jsonl (7.8k rows).

Step 2 — Test the base model before training

python run.py --stage baseline

Downloads Phi-2 (~5GB, cached after first run). Lets you see what the untuned model can do so you have a baseline to compare against.

Step 3 — Fine-tune

python run.py --stage train

Runs QLoRA fine-tuning. On a 6GB laptop GPU this takes roughly 6-8 hours for one epoch. Checkpoints save every 200 steps so you can stop and resume:

# resume after interruption
python run.py --stage train --resume

# quick test run (500 examples, ~20 min)
python run.py --stage train --quick

Step 4 — Run inference

python run.py --stage infer

Loads your fine-tuned model and gives you an interactive prompt. Use :db to point it at a SQLite file and it will generate SQL and run it:

:schema CREATE TABLE students (id INTEGER, name TEXT, fees_paid INTEGER, grade TEXT);
:db data/raw/school.db
Show all students who have not paid fees

Output:

SELECT * FROM students WHERE fees_paid = 0;

┌────┬──────────┬───────────┬───────┐
│ id │ name     │ fees_paid │ grade │
├────┼──────────┼───────────┼───────┤
│  3 │ Rohan A  │ 0         │ B     │
│  7 │ Priya C  │ 0         │ C     │
└────┴──────────┴───────────┴───────┘
2 rows returned

Step 5 — Evaluate

python run.py --stage eval

Runs against the validation set and prints exact match accuracy.

---

Using your own data

Drop any CSV into data/raw/ and convert it:

python src/csv_to_sqlite.py --file data/raw/yourfile.csv

It infers column types automatically and creates a .db file. Then point the inference script at it.

---

Testing mid-training

If training is running and you want to test a checkpoint without stopping:

# in a second terminal
python src/inference_checkpoint.py --interactive

This loads the tokenizer from the base model and the latest checkpoint adapters. Works at any point during training.

---

Config

Everything is in config.yaml. Key settings:

model:
  name: "microsoft/phi-2"    # swap to a different model here
  load_in_4bit: true

lora:
  r: 16                      # increase to 32 for more capacity
  lora_alpha: 32

training:
  num_train_epochs: 3
  per_device_train_batch_size: 2
  max_seq_length: 512        # reduce to 384 if you hit OOM

---

Hardware notes

Built and tested on:

• ASUS ProArt PX13
• NVIDIA RTX 4050 Laptop GPU (6.1GB VRAM)
• 16GB RAM, Python 3.14

If you get CUDA out of memory, open config.yaml and set per_device_train_batch_size: 1 and max_seq_length: 384.

---

What it can do after full training

Works well on single-table queries, filtering, aggregations, basic JOINs. Struggles with deeply nested subqueries and window functions — that's a model size limitation, not a training issue.

Rough accuracy on validation set after 3 epochs: ~60-65% exact match.

---

What I learned building this

• How QLoRA actually works (4-bit base weights + small trainable adapters)
• Why prompt format matters as much as training data
• How to debug CUDA memory issues on a laptop GPU
• That the Spider dataset has been restructured like 4 times and nothing points to the right files anymore