valiant

Compile-time checked SQL for Haskell.

Inspired by Rust's sqlx & resolute, built from scratch for Haskell. No Template Haskell. No libpq. No C dependencies. Raw .sql files validated against a live Postgres database at prepare time, with a GHC source plugin that enforces type safety at compile time. The fastest Haskell PostgreSQL library: 5x faster than hasql on multi-row reads, 10x faster on pipelined batch writes, competitive with asyncpg (Python) on throughput.

  .sql files ──> valiant prepare ──> .valiant/ cache ──> GHC plugin ──> type-safe Haskell
                    (live DB)        (committed)      (compile time)

1. Write SQL in standalone .sql files with full editor support
2. Run valiant prepare to validate queries against your database and cache type metadata
3. The GHC source plugin reads the cache at compile time and verifies your Haskell types match
4. At runtime, a custom Postgres wire protocol driver executes queries with binary format encoding

-- sql/users/find_by_id.sql:
--   SELECT id, name, email FROM users WHERE id = $1

{-# OPTIONS_GHC -fplugin=Valiant.Plugin
                -fplugin-opt=Valiant.Plugin:sql-dir=sql #-}

module MyApp.Queries.Users where

import Valiant

findById :: Statement Int32 (Maybe (Int32, Text, Maybe Text))
findById = queryFile "users/find_by_id.sql"

The plugin verifies at compile time that:

• The .sql file exists
• Int32 matches the $1 parameter (Postgres int4)
• (Int32, Text, Maybe Text) matches the result columns
• email is correctly wrapped in Maybe (it's nullable)

If anything is wrong, you get a clear compile error:

src/MyApp/Queries/Users.hs:12:1: error: [VALIANT-003]

    -- Result type mismatch
    |
    |  Comparing column by column:
    |
    |    Column    Postgres type    Your type       Expected
    |    id        int4             Int32           Int32       ok
    |    name      text             Text            Text        ok
    |    email     text (nullable)  Text            Maybe Text  MISMATCH
    |
    -- Fix: wrap the field in Maybe:  Maybe Text

import Valiant
import MyApp.Queries.Users qualified as Q

main :: IO ()
main = do
  pool <- newPool defaultPoolConfig
    { poolConnString = "postgres://user:pass@localhost:5432/mydb"
    , poolSize = 10
    }

  -- Fetch one row
  mUser <- withResource pool $ \conn ->
    fetchOne conn Q.findById 42

  -- Fetch all rows
  users <- withResource pool $ \conn ->
    fetchAll conn Q.listAll ()

  -- Execute a command
  n <- withResource pool $ \conn ->
    execute conn Q.insert ("Alice", Just "alice@example.com")

  -- Batch insert (pipelined, eliminates per-row round-trips)
  withResource pool $ \conn ->
    executeBatch conn Q.insert
      [ ("Alice", Just "alice@example.com")
      , ("Bob", Just "bob@example.com")
      , ("Carol", Nothing)
      ]

  -- Transactions
  withTransaction pool $ \tx -> do
    execute (txConn tx) Q.insert ("Dave", Just "dave@example.com")
    execute (txConn tx) Q.insert ("Eve", Nothing)

  -- Constant-memory streaming (no cursor/transaction needed)
  total <- withResource pool $ \conn ->
    executeWithFold conn Q.listAll () (RowFold 0 (\n _ -> n + 1))

# Validate all .sql files against your database
$ valiant prepare
  [1/10] sql/users/find_by_id.sql ............ ok
  [2/10] sql/users/find_by_email.sql ......... ok
  ...
  Wrote 10 cache files to .valiant/

# Check cache freshness (for CI, no database needed)
$ valiant check

# Print inferred Haskell types
$ valiant types
  sql/users/find_by_id.sql
    Params: Int32
    Result: (Int32, Text, Maybe Text)

# Auto-generate Haskell binding modules
$ valiant generate --module-prefix MyApp.Queries --output-dir src/MyApp/Queries/
  Generated src/MyApp/Queries/Users.hs (7 queries)
  Generated src/MyApp/Queries/Posts.hs (3 queries)

# Watch for changes and re-prepare
$ valiant watch

valiant is the fastest Haskell PostgreSQL library. It implements its own wire protocol in pure Haskell with binary format encoding, direct byte writes, pipelined execution, async sender/receiver split, and zero unnecessary copies.

Benchmarks against hasql (libpq FFI, binary) and postgresql-simple (libpq FFI, text). Single connection, native Postgres 16.

Reads (Linux benchmark host, Postgres 16, Unix socket):

valiant is 1.6x slower on single-row lookups (the async sender/receiver split has per-query coordination overhead that dominates when there's nothing to pipeline) but 5x faster once row decoding dominates.

Writes (pipelined):

Pipelined batch inserts are 9.9x faster than hasql and 16.6x faster than postgresql-simple.

vs asyncpg (Python) (CI-verified on the same GitHub Actions runner, Postgres 16, Unix socket):

See docs/benchmark-results/asyncpg-comparison.md for the full head-to-head comparison.

Why it's fast:

• Binary format decoding -- direct from network buffer, no FFI boundary
• Direct byte writes -- unsafeCreate + pokeByteOff, 5-6x faster than Builder for fixed-size types (30ns per Int32)
• Pipelined execution -- executeBatch sends N Bind+Execute pairs with 1 Sync, 40-100x faster than sequential
• Sender/receiver split -- dedicated writer+reader threads per connection, automatic pipelining for concurrent workloads
• Fused Builder encoding -- all protocol messages in a batch fused into one Builder, one allocation, one send()
• Direct-to-Vector parsing -- DataRow parsed directly into mutable Vector, no intermediate list
• Pre-computed message sizes -- single-pass protocol encoding, no double-copy
• Message coalescing + TCP_NODELAY -- single syscall per message sequence
• Fused row decoding -- decode as rows arrive, no intermediate list
• Pure Haskell -- no libpq, no C toolchain, no system dependencies

See docs/PERFORMANCE.md for the full deep-dive: codec benchmarks, pool benchmarks, architecture comparison, optimization techniques, and the complete optimization journey. Archived numbers with CSV data (Apple Silicon / macOS / Docker Postgres) are in docs/benchmark-results/; the bench-compare suite reproduces the read, insert, and asyncpg comparisons locally or via the Benchmarks workflow.

valiant is a multi-package Cabal project:

valiant/
├── wire/                      # pg-wire: wire protocol, connection, pool, auth, TLS
│   ├── src/PgWire/            # Protocol messages, builders, parsers, async I/O
│   └── test/                  # Wire protocol unit tests
├── runtime/                   # valiant: runtime library
│   ├── src/Valiant/           # Binary codecs, execute, batch, pipeline, fold, copy, streaming
│   ├── bench/                 # Codec + concurrent benchmarks (criterion)
│   ├── integration/           # Integration tests (require Postgres)
│   └── test/                  # Codec unit tests
├── src/                       # valiant-cli source
│   └── Valiant/CLI/           # Commands, cache, type map, discovery, nullability
├── plugin/                    # GHC source plugin
│   └── src/Valiant/Plugin/    # AST traversal, verification, error messages
├── adapters/                  # Streaming and effect system adapters (8 packages)
│   ├── valiant-conduit/       # Conduit adapter
│   ├── valiant-pipes/         # Pipes adapter
│   ├── valiant-streaming/     # streaming library adapter
│   ├── valiant-streamly/      # Streamly adapter
│   ├── valiant-bluefin/       # Bluefin effect system adapter
│   ├── valiant-effectful/     # Effectful effect system adapter
│   ├── valiant-fused-effects/ # Fused-effects adapter
│   └── valiant-mtl/           # MTL monad transformer adapter
├── example/                   # Example REST API (scotty)
├── bench-compare/             # Comparative benchmarks vs hasql, pg-simple
├── scripts/                   # pg-setup.sh, pg-teardown.sh
├── docs/                      # TUTORIAL.md, PERFORMANCE.md, ASYNC_ARCHITECTURE.md
└── .valiant/                  # Cached query metadata (committed to VCS)

• One SQL statement per .sql file with full editor support
• Optional metadata comments: -- valiant:name, -- valiant:result, -- valiant:single
• Directory structure maps to Haskell module structure

• 6 structured error codes (VALIANT-001..006) with column-by-column diagnostics
• Nullability inference from pg_attribute
• Did-you-mean suggestions for mistyped file paths (Levenshtein distance)
• Type inference (no signature required) or typed hole discovery
• addDependentFile tracking: GHC recompiles when .sql files change

Nullable columns are wrapped in Maybe. Custom types are auto-discovered from pg_type at prepare time: enums map to Text, domains unwrap to their base type, ranges map to PgRange BaseType. Manual overrides via valiant-types.json.

• Custom PostgreSQL v3 wire protocol implementation (no FFI, no libpq)
• Async sender/receiver split: dedicated writer+reader threads per connection with automatic pipelining for concurrent workloads (7.2x scaling at 32 threads)
• Binary format encoding/decoding for all supported types
• Extended query protocol (Parse/Bind/Execute/Sync) with prepared statement caching
• Cross-connection shared statement cache at the pool level
• Pipelined batch execution (executeBatch) for high-throughput writes
• Pipeline Applicative for combining independent queries into one round-trip
• Fused Builder encoding and direct-to-Vector DataRow parsing
• Constant-memory streaming via RowFold (no cursor/transaction needed)
• Server-side cursors for large result sets within transactions
• Connection pooling with idle reaping, max lifetime, health checking, and pool-level type cache
• SCRAM-SHA-256 (with channel binding), MD5, and cleartext authentication
• TLS 1.2/1.3 via the tls library, client certificates, CA validation
• Multi-host failover with target_session_attrs and load_balance_hosts
• Transactions with configurable isolation levels and savepoints
• LISTEN/NOTIFY for async notifications (callback-based, no polling)
• COPY IN/OUT for bulk data transfer (text, CSV, and binary formats)
• Query cancellation with cancelQuery and withQueryTimeout
• Composite, range, array, interval, and Scientific binary codecs
• PgEnum type class for Haskell sum types mapping to PG enums
• Protocol tracing via setTraceHandler callback
• Logging hooks for query timing and connection events

# 1. Write a query
echo "SELECT id, name FROM users WHERE active = true" > sql/users/list_active.sql

# 2. Validate against your dev database
export DATABASE_URL="postgres://localhost:5432/mydb"
valiant prepare

# 3. Write (or generate) the Haskell binding
# 4. Build (the plugin checks everything at compile time)
cabal build

steps:
  - name: Verify query cache
    run: valiant check          # no database needed

  - name: Build
    run: cabal build
    env:
      VALIANT_OFFLINE: "true"   # plugin reads from .valiant/ only

# Start Postgres via docker-compose (tuned for benchmarks)
docker compose up -d --wait
export DATABASE_URL="postgres://valiant_test:valiant_test@localhost:5433/valiant_test"

# Codec benchmarks (pure, no database needed)
cabal bench valiant-bench --benchmark-options='--match prefix codec'

# Query benchmarks
cabal bench valiant-bench --benchmark-options='--match prefix query'

# Concurrent benchmarks (the async split showcase)
cabal bench valiant-bench --benchmark-options='+RTS -N -RTS --match prefix concurrent'

# Comparative benchmarks vs hasql and postgresql-simple
# (requires system libpq: `brew install libpq` on macOS, then add its bin to PATH)
cabal run --project-file=cabal.project.bench bench-compare

# Teardown
docker compose down

Requires GHC 9.10.3 and Cabal 3.0+. v0.1 ships against a single GHC version; multi-GHC support (9.6, 9.8, and newer) is planned for 0.1.x once the plugin's GHC AST shims are in place.

git clone https://github.com/joshburgess/valiant.git
cd valiant
cabal build all
cabal test pg-wire-test valiant-test valiant-cli-test valiant-plugin-test

All packages compile with -Werror.

Why not Template Haskell? TH has stage restrictions, cross-compilation issues, and makes it hard to produce good error messages. A GHC source plugin runs after typechecking, has access to the full AST, and can emit rich diagnostics with source locations and custom formatting.

Why a separate prepare step? Connecting to Postgres from inside the compiler (as Rust's sqlx does) causes well-known compilation speed issues and complicates CI. A separate CLI step + JSON cache keeps compilation fast and enables fully offline builds.

Why a custom wire protocol driver? Full control over binary format encoding, connection management, and protocol features (pipelining, async I/O, COPY, LISTEN/NOTIFY, cursors) without depending on libpq or any existing Haskell database library. No system C dependencies means simpler builds and cross-compilation. And as the benchmarks show, pure Haskell binary decoding is faster than FFI-based alternatives on multi-row reads.

Why sender/receiver split? The same architecture behind asyncpg's 3x advantage over psycopg2. Dedicated writer+reader threads per connection allow multiple green threads to pipeline queries automatically on a single connection, achieving 7.2x throughput scaling at 32 threads.

BSD-3-Clause