Skip to content
BranchesTags

Repository files navigation

WitDatabase

A high-performance embedded key-value database for .NET with support for multiple storage engines, ACID transactions, and encryption.

.NET License

Features

  • Two Storage Engines

    • B-Tree - Optimized for read-heavy workloads with excellent random access
    • LSM-Tree - For sustained high-volume ingest into tables with few or no secondary indexes; see Docs/WitSQL.md § 14.9 for where it wins and where it does not

  • Encryption

    • AES-256-GCM with hardware acceleration
    • ChaCha20-Poly1305 via BouncyCastle (Blazor WASM compatible)
    • Password-based key derivation (PBKDF2)

  • ACID Transactions

    • Atomicity, Consistency, Isolation, Durability
    • Write-Ahead Logging (WAL)
    • Crash recovery

  • Concurrency

    • Reader-writer locking
    • File locking for multi-process safety
    • Async/await support

  • SQL Support

    • Full SQL parser (WitSQL dialect)
    • ADO.NET provider
    • Entity Framework Core provider
    • Window functions, CTEs, subqueries
    • LATERAL / CROSS APPLY / OUTER APPLY, VALUES as a table source, TOP n
    • 90+ built-in functions, plus window functions - Docs/WitSQL.md § 5 and § 7

  • User-Defined Functions and Stored Procedures

    • CREATE FUNCTION — a scalar expression over its parameters, callable anywhere an expression may appear: a SELECT list, a WHERE, a CHECK, a computed column, an index key
    • CREATE PROCEDURE / CALL — a body of statements, invoked as one unit of work, and reachable from ADO.NET through CommandType.StoredProcedure
    • SQL bodies only: no external code and no assembly loading
    • Reported by INFORMATION_SCHEMA.ROUTINES and .PARAMETERS

  • Fluent API

    • Easy configuration with builder pattern
    • Extensible via extension methods
    • Simple static factory methods

  • Provider System

    • Pluggable storage, encryption, cache, and journal providers
    • Auto-detection of settings when reopening databases
    • Easy registration of custom providers

Packages

Package Description
OutWit.Database.Core Core storage engine (B+Tree, LSM-Tree, MVCC)
OutWit.Database.Core.BouncyCastle ChaCha20-Poly1305 encryption provider
OutWit.Database.Core.IndexedDb IndexedDB storage for Blazor WebAssembly
OutWit.Database.Parser SQL parser (ANTLR4-based)
OutWit.Database SQL execution engine
OutWit.Database.AdoNet ADO.NET provider
OutWit.Database.EntityFramework Entity Framework Core provider

Installation

# Core storage engine
dotnet add package OutWit.Database.Core

# SQL engine with ADO.NET
dotnet add package OutWit.Database.AdoNet

# Entity Framework Core
dotnet add package OutWit.Database.EntityFramework

# Optional: BouncyCastle encryption (for Blazor WASM)
dotnet add package OutWit.Database.Core.BouncyCastle

# Optional: IndexedDB storage (for Blazor WASM)
dotnet add package OutWit.Database.Core.IndexedDb

Quick Start

Key-Value Storage (Core API)

using OutWit.Database.Core.Builder;

// Create a new database
using var db = WitDatabase.Create("mydata.db");

// Or with encryption
using var db = WitDatabase.Create("secure.db", "my-password");

// Store and retrieve data
db.Put("user:1"u8, """{"name": "John", "age": 30}"""u8);
var value = db.Get("user:1"u8);
db.Delete("user:1"u8);

SQL with ADO.NET

using OutWit.Database.AdoNet;

using var connection = new WitDbConnection("Data Source=mydb.witdb");
connection.Open();

using var cmd = connection.CreateCommand();
cmd.CommandText = "CREATE TABLE Users (Id INT PRIMARY KEY, Name VARCHAR(100))";
cmd.ExecuteNonQuery();

cmd.CommandText = "INSERT INTO Users (Id, Name) VALUES (@id, @name)";
cmd.Parameters.AddWithValue("@id", 1);
cmd.Parameters.AddWithValue("@name", "John Doe");
cmd.ExecuteNonQuery();

Functions and Stored Procedures

using var cmd = connection.CreateCommand();

// A function is a scalar expression over its parameters, callable anywhere an
// expression may appear - a SELECT list, a WHERE, a CHECK, an index key.
cmd.CommandText = "CREATE FUNCTION Doubled(N INT) RETURNS INT AS BEGIN RETURN N * 2; END";
cmd.ExecuteNonQuery();

cmd.CommandText = "SELECT Doubled(21)";
var answer = cmd.ExecuteScalar();          // 42

// A procedure is a body of statements. The last statement's result is the call's.
cmd.CommandText = @"
    CREATE PROCEDURE RecentUsers AS BEGIN
        SELECT * FROM Users ORDER BY Id DESC;
    END";
cmd.ExecuteNonQuery();

// Invoked the ordinary ADO.NET way.
using var call = connection.CreateCommand();
call.CommandType = CommandType.StoredProcedure;
call.CommandText = "RecentUsers";

using var reader = call.ExecuteReader();

Bodies are SQL — no external code and no assembly loading. See Docs/WitSQL.md § 2.10–2.11 for the rules: what a body may contain, how determinism decides whether a function may key an index, and why a trigger body may not CALL.

Entity Framework Core

using Microsoft.EntityFrameworkCore;
using OutWit.Database.EntityFramework.Extensions;

public class AppDbContext : DbContext
{
    public DbSet<User> Users => Set<User>();
    
    protected override void OnConfiguring(DbContextOptionsBuilder options)
        => options.UseWitDb("Data Source=myapp.witdb");
}

// Usage
using var context = new AppDbContext();
context.Database.EnsureCreated();
context.Users.Add(new User { Name = "John" });
context.SaveChanges();

Blazor WebAssembly

using OutWit.Database.Core.Builder;
using OutWit.Database.Core.IndexedDb;
using OutWit.Database.Core.BouncyCastle;

// In Blazor component
var db = new WitDatabaseBuilder()
    .WithIndexedDbStorage("MyDatabase", JSRuntime)
    .WithBouncyCastleEncryption("password")  // Works in browser
    .WithBTree()
    .Build();

await ((StorageIndexedDb)db.Store).InitializeAsync();

Configuration

Storage Engines

Method Description
WithFilePath(path) Use file-based storage
WithMemoryStorage() Use in-memory storage
WithBTree() Use B-Tree engine (default)
WithLsmTree() Use LSM-Tree engine

Encryption

Method Description
WithEncryption(password) AES-GCM with password
WithBouncyCastleEncryption(password) ChaCha20-Poly1305

Transactions

Method Description
WithTransactions() Enable ACID transactions
WithMvcc() Enable MVCC
WithFileLocking() Enable file locking

Architecture

+---------------------------------------------------------------+
|                      WitDatabaseBuilder                       |
|            (Fluent API for database configuration)            |
+---------------------------------------------------------------+
|                    TransactionalStore                         |
|                 (ACID transactions, locking)                  |
+---------------------------------------------------------------+
|      +----------------+    +----------------+                 |
|      |   StoreBTree   |    |    StoreLsm    |                 |
|      |  (B+Tree engine)|    | (LSM-Tree engine)|              |
|      +----------------+    +----------------+                 |
+---------------------------------------------------------------+
|                     ProviderRegistry                          |
|          (Pluggable providers for all components)             |
+---------------------------------------------------------------+

Performance

Measured with Benchmarks/OutWit.Database.Benchmarks (BenchmarkDotNet, ShortRun, in-process) on a Ryzen 9 5950X under .NET 10, against SQLite (Microsoft.Data.Sqlite) and LiteDB. Re-measured 2026-08-11. Every figure is the mean of two identical passes taken on an otherwise idle machine, and anything whose two passes disagreed by more than 10% is left out rather than quoted - three write rows fell out that way. Default configuration means a bare Data Source=... connection string - MVCC on, durable commit, B+Tree - which is what an ADO.NET or EF Core consumer gets.

Reads and lookups, default configuration

Operation WitDatabase LiteDB SQLite
Point query by primary key, x100 0.26 ms 1.27 ms 4.86 ms
Seek on a UNIQUE index, x100 (5,000 rows) 0.52 ms 1.95 ms 5.09 ms
Sequential reads, x100 0.35 ms 7.59 ms 5.02 ms
Index range scan (BETWEEN), 5,000 rows 0.93 ms 4.58 ms 0.20 ms
SELECT ... LIMIT 100 0.12 ms 0.15 ms 0.07 ms
Full scan, 1,000 rows 0.80 ms 1.40 ms 0.18 ms
GROUP BY, 1,000 rows 1.01 ms 1.50 ms 0.22 ms
ORDER BY, 1,000 rows 1.66 ms 1.78 ms 0.23 ms
INNER JOIN over 4 tables 0.48 ms 0.18 ms 0.09 ms

Writes, default configuration

Operation WitDatabase LiteDB SQLite
Mixed transaction (insert/update/select) 4.85 ms 7.33 ms 6.96 ms
100 inserts in one transaction 5.07 ms 1.00 ms 6.82 ms
UPDATE by key, in a transaction 5.16 ms 1.74 ms 6.88 ms
100 inserts without a transaction 301 ms 10.7 ms 925-1233 ms

Sustained ingest, 200,000 rows in batches of 1,000

microseconds per row B+Tree LSM
MVCC on - what a plain connection string gives 36.8 772
MVCC=false, no secondary indexes 13.6-17.2 11.0-18.3
MVCC=false, three secondary indexes 22.9-23.2 33.6-38.6

Read all of it honestly:

  • Lookups and indexed access are the strong side - 3-20x faster than LiteDB and, on small operations, faster than SQLite too. Part of the SQLite margin is the P/Invoke crossing its managed wrapper pays per call; that is not an engine result, but it is what a .NET consumer experiences, because there is no other way to reach SQLite from managed code.
  • Scans, sorts and joins are the weak side - SQLite is 4-13x faster there, and the gap is allocation: a scan costs roughly 2.4 KB per row returned. LiteDB, also managed, pays about the same, so this does not move the comparison against it.
  • Autocommit is expensive by choice. Every statement outside a transaction is durable, which is what makes 301 ms against LiteDB's 10.7 - and 3-4x faster than SQLite doing the same thing. Batch writes in a transaction when throughput matters.
  • Store=lsm is not simply "write-optimised", and with the default settings it is much worse. With MVCC=false it matches the B+Tree on indexless ingest and is ~1.6x behind with three secondary indexes. With MVCC on - the default - it costs 20x the B+Tree per row, and no batch size amortises it. See Docs/WitSQL.md § 14.9 before choosing it.
  • COUNT(*) is deliberately not in the tables above. WitDatabase answers it from a cached per-table counter in ~1 microsecond while the other two count rows; that is a real property worth knowing and it is not a speed comparison.

Reproduce with:

dotnet run -c Release --project Benchmarks/OutWit.Database.Benchmarks -- verify
dotnet run -c Release --project Benchmarks/OutWit.Database.Benchmarks -- --filter "*"

The first command is the equivalence check: it runs every benchmark body once and compares what WitDatabase, SQLite and LiteDB actually return. A timing comparison between engines that do not compute the same thing is not a measurement, so that check is meant to be green before any figure above is believed.

Earlier figures withdrawn twice, and both times are worth recording. Releases before 11.1.0 advertised transaction ratios measured on the least discriminating workload in the suite, and before that "4-20x faster" numbers from a benchmark project no longer in the repository. What changed in between: the benchmark suite had no assertion of any kind in 113 methods, so nothing had ever checked that the three engines compute the same answer, and several published claims turned out to describe an engine that no longer existed.

The 2026-08-11 re-measurement found the write paths 1.5-1.7x slower than the 2026-08-02 baseline on the same benchmark and the same machine, while SQLite's and LiteDB's numbers on those same rows moved by less than 13%. The heavy read paths did not move - full scan, ORDER BY, point query and unique-index seek are all within 6% of the baseline - but three lighter ones did: LIMIT 100 0.08 -> 0.12 ms, sequential reads 0.31 -> 0.35, index range scan 0.83 -> 0.93. All of that is recorded rather than explained: it is a real difference between 11.1.0 and 12.8.0 and it has not yet been diagnosed.

Requirements

  • .NET 10.0
  • Windows, Linux, or macOS

Project Structure

WitDatabase/
+-- Sources/
|   +-- Core/
|   |   +-- OutWit.Database.Core/           # Storage engine
|   |   +-- OutWit.Database.Core.BouncyCastle/  # ChaCha20 encryption
|   |   +-- OutWit.Database.Core.IndexedDb/     # Blazor WASM storage
|   +-- Engine/
|   |   +-- OutWit.Database.Parser/         # SQL parser
|   |   +-- OutWit.Database/                # SQL engine
|   +-- Providers/
|       +-- OutWit.Database.AdoNet/         # ADO.NET provider
|       +-- OutWit.Database.EntityFramework/ # EF Core provider
+-- Tools/
|   +-- OutWit.Database.Studio/             # Database management tool
+-- Samples/
+-- Benchmarks/

Documentation

The audit and the phase plans are working papers rather than documentation and are no longer shipped in Docs/. Source comments that cite one by name - AUDIT-2026-07.md, NEXT-SESSION-PLAN.md, PHASE*-*.md - refer to files that remain in the git history.

Running Tests

# All tests
dotnet test

# Specific project
dotnet test Sources/Core/OutWit.Database.Core.Tests

License

Licensed under the Apache License, Version 2.0. See LICENSE.

Attribution (optional)

If you use WitDatabase in a product, a mention is appreciated (but not required), for example: "Powered by WitDatabase https://witdatabase.io/".

Trademark / Project name

"WitDatabase" and the WitDatabase logo are used to identify the official project by Dmitry Ratner.

You may:

  • refer to the project name in a factual way (e.g., "built with WitDatabase");
  • use the name to indicate compatibility (e.g., "WitDatabase-compatible").

You may not:

  • use "WitDatabase" as the name of a fork or a derived product in a way that implies it is the official project;
  • use the WitDatabase logo to promote forks or derived products without permission.

Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

  1. Fork the repository
  2. Create your feature branch (git checkout -b feature/amazing-feature)
  3. Commit your changes (git commit -m 'Add amazing feature')
  4. Push to the branch (git push origin feature/amazing-feature)
  5. Open a Pull Request

Changelog

See CHANGELOG.md for version history. Each package carries its own ROADMAP.md for what is planned in that layer.

About

Pure .NET relational file database

Resources

Readme
Apache-2.0 license
Activity

Stars

1 star

Watchers

1 watching

Forks

2 forks
Report repository

Releases

Packages

Contributors

Languages