SPEC.md

RowGuard Specification

Current as of 0.6.0

Shipped surface:

  • Synchronous Core API: select, execute, validate_rows, compile_plan, stream

  • Async Core API: aselect, aexecute, astream

  • StreamResult[T] / AsyncStreamResult[T] with context-managed cleanup

  • SQLAlchemy Core Table / Select with sync or async session/connection

  • SQLAlchemy ORM mapped classes (projected + single-entity)

  • SQLModel table sources (rowguard[sqlmodel])

  • orm_validation, unloaded_attributes, attribute_map, source_identity

  • SQLRules pushdown (use_sqlrules, optional compiled_rules)

  • Rejection policies: raise, collect, skip, log, callback, quarantine

  • Callback / quarantine async reject handlers on async APIs

  • Rejection thresholds (max_rejections, max_rejection_rate)

  • Staged immutable ExecutionPlan and planning diagnostics

  • Streaming options: yield_per, StreamObserver / BaseStreamObserver (observers remain sync callables)

Deferred (not available yet):

  • Nested relationship / graph validation — later

  • Plugin system / provider registries — 0.7.0

  • Reflection / raw text() — 0.8.0

Overview

RowGuard is a validation-first query engine built on top of SQLAlchemy, Pydantic, and SQLRules.

Its purpose is to guarantee that every row returned from a database query is classified as either:

  1. A valid Pydantic model.

  2. A rejected row with structured diagnostics.

Unlike an ORM, RowGuard does not own your schema. It works with existing SQLAlchemy Core tables, ORM mapped classes, and SQLModel table models, and is designed to extend to reflected schemas and raw SQL in later releases.

Scope

RowGuard is responsible for:

  • Building SQLAlchemy queries

  • Integrating SQLRules WHERE expressions

  • Executing queries

  • Adapting rows to dictionaries

  • Validating rows with Pydantic

  • Handling rejected rows

  • Returning typed results

  • Producing diagnostics and statistics

RowGuard is not responsible for:

  • Owning ORM persistence or relationship graphs (0.5 does validate ORM / SQLModel reads)

  • Schema migrations

  • SQL generation beyond applying SQLRules / where= expressions

  • Database drivers

  • Database connections

Core Pipeline

Pydantic Model
      │
      ▼
SQLRules
Compile SQL-safe constraints
      │
      ▼
SQLAlchemy Statement
      │
      ▼
Database
      │
      ▼
Row Adapter
      │
      ▼
Pydantic Validation
      │
      ├── Valid Models
      └── Rejected Rows

Primary API

result = rowguard.select(
    session=session,
    table=users,
    model=UserRead,
)

Additional APIs (0.2.0+):

rowguard.execute(...)
rowguard.validate_rows(...)
rowguard.compile_plan(...)

Streaming (0.3.0):

with rowguard.stream(...) as stream:
    for model in stream:
        ...

QueryResult

Every query returns a QueryResult.

QueryResult[T]

Properties:

  • models

  • rejected

  • statistics

  • statement

  • diagnostics

  • execution_time (derived from statistics.execution_time_ns)

RejectedRow

A rejected row contains:

  • row index

  • adapted mapping (when adaptation succeeded)

  • ValidationError and/or adaptation error

  • model type

  • optional raw row

Rejected rows are first-class objects.

Rejection Policies

Supported in 0.1.0+:

  • raise

  • collect

  • skip

Planned for later releases:

  • log

  • callback

  • quarantine

Policy selection is explicit.

SQLRules Integration

Before execution, RowGuard asks SQLRules to compile SQL-safe constraints.

Pydantic
   ↓
SQLRules
   ↓
WHERE expressions

These expressions are added to the SQLAlchemy statement before execution.

Validation still occurs after retrieval.

Validation

Validation uses:

Model.model_validate(mapping)

No custom validation engine is introduced.

Nested models, field validators, and model validators continue to work because Pydantic remains the source of truth.

Supported Inputs

Current (0.6.0):

  • SQLAlchemy Table

  • SQLAlchemy Select

  • SQLAlchemy ORM mapped classes

  • SQLModel table models (rowguard[sqlmodel])

  • SQLAlchemy Session / Connection

  • SQLAlchemy AsyncSession / AsyncConnection

  • Buffered (select / execute / aselect / aexecute) and streaming (stream / astream) result modes

Future:

  • reflected metadata

  • raw text() queries

Streaming

Shipped in 0.3.0. Large result sets are processed incrementally without retaining accepted models:

with rowguard.stream(
    session=session,
    table=users,
    model=UserRead,
    on_reject="skip",
) as stream:
    for model in stream:
        ...
    statistics = stream.statistics

StreamResult is separate from QueryResult. Async streaming shipped in 0.4.0 as astream / AsyncStreamResult with the same rejection and lifecycle semantics.

Async

Shipped in 0.4.0:

await rowguard.aselect(...)
await rowguard.aexecute(...)

async with rowguard.astream(...) as stream:
    async for model in stream:
        ...

Async behavior mirrors synchronous behavior for validation and rejection policies. Await only DB I/O; Pydantic validation remains synchronous on the event loop. See the docs site page Supported vs planned and docs/architecture/ASYNC.md.

Statistics

Every QueryResult exposes QueryStatistics with:

  • rows_read

  • rows_validated

  • rows_accepted

  • rows_rejected

  • adaptation_time_ns

  • validation_time_ns

  • rejection_time_ns

  • execution_time_ns

QueryResult.execution_time is a convenience property over execution_time_ns.

Diagnostics

Diagnostics may include:

  • planning codes (source resolved, pushdown applied/skipped, …)

  • rejected fields

  • validation summaries

  • SQL statement

  • rejection policy used

Error Model

Public exceptions derive from:

RowGuardError

Examples:

  • QueryExecutionError

  • RowValidationError

  • RowAdaptationError

  • PlanningError

  • ConfigurationError

  • RejectHandlerError

SQLModel Position

SQLModel and RowGuard solve different problems.

SQLModel models tables.

RowGuard validates query results.

The libraries should integrate naturally without overlapping responsibilities.

Performance Goals

  • Stream large datasets (0.3.0+)

  • Avoid duplicate validation

  • Reuse SQLRules compilation

  • Linear scaling with row count

Plugin Opportunities

Future extension points:

  • reject handlers

  • row adapters

  • result serializers

  • diagnostics

  • dialect helpers

Compatibility

Initial release:

  • Python 3.10+

  • Pydantic v2

  • SQLAlchemy 2.x

  • SQLRules 0.x

Design Principles

  • Validation-first

  • SQLAlchemy-native

  • Pydantic-native

  • Deterministic

  • Observable

  • Composable

  • Explicit rejection handling

Success Criteria

A successful RowGuard application can state:

  • Every returned model satisfies the Pydantic contract.

  • Every rejected row is accounted for.

  • SQL filtering and Pydantic validation work together.

  • Invalid data never silently reaches application code.