SoldierDB GitHub
Field manual FM-DB · Edition 1

Holds the line
on your data.

A relational database built from scratch in Go. A crash-safe page store, a copy-on-write B+tree, transactions, SQL and vector search, served to Rust and C++ through a stable C ABI.

Read your orders Enlist on GitHub
4,096bytes per page, nothing more, nothing less
2alternating, checksummed meta pages
0overwrites of a page reachable from a committed root
3languages served: Go, Rust and C++
Section 1

The field manual

Every layer is written from the page up, with tests against an in-memory oracle. No borrowed storage engine, no magic.

FM 1-01

Crash-safe page store

A file of 4 KiB pages. Commits write the inactive meta page last and fsync it, so a crash always leaves one valid root to recover from.

FM 1-02

Copy-on-write B+tree

Lookups, ordered range scans, splits and merges. A mutation never touches a committed page; freed pages return through a persistent free list.

FM 1-03

Atomic transactions

BEGIN, COMMIT, ROLLBACK. Writes are buffered and published as one batch behind a single meta-page flip.

FM 1-04

A real SQL front end

Hand-written lexer and recursive-descent parser. Filters, ORDER BY, LIMIT, aggregates, GROUP BY / HAVING, primary keys, UNIQUE, NOT NULL and defaults.

FM 1-05

Vector search

Fixed-dimension VECTOR(n) columns and distance functions, so you can rank rows by similarity right in SQL.

FM 1-06

One engine, three languages

A handle-based C ABI with clear memory ownership, wrapped by a safe Rust crate and an RAII C++ client. Each has its own CLI.

Section 2

Chain of command

Orders travel down, results report up. Each layer owns one job and trusts the one beneath it.

Shell & clients cmd/ · capi/ · clients/SQL REPL, C ABI, Rust and C++ wrappers
SQL engine/sqlLexer, parser, catalog, executor
Transactions engine/dbDurable KV facade, BEGIN / COMMIT / ROLLBACK
B+tree engine/btreeCopy-on-write nodes, search, insert, delete
Pager engine/pagerFixed-size pages, meta pages, fsync
Section 3

Your orders

Requires Go 1.24+. Rust and a C++20 toolchain are only needed for the clients.

Order 1 · Report for duty
# clone, test and start the SQL shell
git clone https://github.com/Youssef-joe/SoldierDB
cd SoldierDB
make test
make run
Order 2 · Hold the line
CREATE TABLE users (
  id INTEGER PRIMARY KEY,
  name TEXT NOT NULL, age INTEGER);
INSERT INTO users VALUES (1, 'Ada', 36), (2, 'Grace', 45);
SELECT name, age FROM users WHERE age > 40;
Order 3 · Find the nearest
CREATE TABLE docs (id INTEGER PRIMARY KEY, embedding VECTOR(3));
SELECT id FROM docs
ORDER BY cosine_distance(embedding, VECTOR(1, 0, 0))
LIMIT 10;
Order 4 · Full inspection
# fmt check, Go tests, then the C, Rust and C++ clients
make check

# or build just the shared library + header
make shared
Section 4

The campaign

Ground taken and the objectives ahead. The full plan lives in ROADMAP.md.

SecuredMilestones M0–M9

Engine, SQL and clients

  • Pager, B+tree, free list
  • Transactions, SQL, shell
  • C ABI, Rust, C++, vectors
Phase 2

Durability & storage

  • Per-page checksums
  • Overflow pages, VACUUM
  • Optional WAL
Phase 3

Indexes & planner

  • CREATE INDEX
  • Pushdown, EXPLAIN
  • Joins and subqueries
Phase 4

Concurrency & MVCC

  • Snapshot readers
  • Multi-process locking
  • Concurrent writers
Phase 5

Vector search

  • HNSW / IVF index
  • Filtered search
  • Recall benchmarks
Phase 6

Networking

  • soldierdb serve
  • Wire protocol, TLS
  • Prepared statements
Phase 7

Tooling & release

  • CI on Linux and macOS
  • Benchmarks
  • Tagged binary releases

Field intelligence: open issuesAll issues →

  1. #1Lost writes: two handles on the same file silently overwrite each other's commits
  2. #2B+tree split overflows a page with mixed-size entries
  3. #3Failed statement inside BEGIN…COMMIT keeps its partial writes
  4. #4INTEGER overflow wraps silently (arithmetic, SUM, ABS, auto-rowid)
  5. #5Torn trailing page after a crash makes the database unopenable
  6. #6SELECT silently truncates results when a row fails to decode
  7. #7sql.Engine is not goroutine-safe
  8. #8Finish the YousDB → SoldierDB rename