GrimleyTK: Declarative Data Architecture Toolkit

Database architecture is often scattered across migration files, handwritten SQL, framework conventions, and undocumented ownership rules.

As systems grow, it becomes difficult to answer basic architectural questions: Which domain owns a table? Which domains may read from it? Is a model writable or read-only? Does a cross-domain reference violate an ownership boundary? What SQL will be executed before it reaches the database?

Traditional migration workflows describe a sequence of changes, but they do not necessarily preserve a clear representation of the intended architecture.

GrimleyTK addresses this problem by treating the declared data architecture as the source of truth. Instead of beginning with SQL operations, developers describe domains, ownership, tables, columns, relationships, and read models in a YAML specification that can be validated before any statement is executed.

GrimleyTK was created to explore a more explicit approach to database architecture.

The project is based on a simple idea: structural decisions should be visible before they become database operations. Domain ownership, read access, table definitions, and cross-domain dependencies should not exist only as implicit knowledge distributed across application code and migration history.

By separating configuration, validation, planning, and execution, GrimleyTK creates a reviewable path from architectural intent to PostgreSQL objects.

The goal is not to replace every migration framework. It is to provide a focused toolkit for systems where explicit ownership, predictable SQL generation, architectural validation, and inspectable execution plans are more important than hiding the database behind abstractions.

GrimleyTK follows a staged architecture that separates declaration from database execution.

1. Declarative configuration

The project architecture is defined in a grimley.yaml file. The specification describes domains, ownership, read access, tables, columns, primary keys, references, and read models.

2. Configuration loader

The configuration package reads the YAML document and transforms it into the internal data model used by the remaining stages.

3. Validation layer

Before SQL is generated, validators inspect structural integrity, domain ownership, cross-domain references, read-only boundaries, sensitive column names, PostgreSQL identifiers, and accepted SQL types.

Invalid configurations are rejected before reaching the executor.

4. Planning layer

The planner converts the validated architecture into deterministic PostgreSQL statements. The current implementation generates schemas, tables, columns, primary keys, and views without producing destructive operations.

5. Execution layer

The executor applies the generated plan inside a database transaction. If any statement fails, the transaction is rolled back to prevent a partially applied architecture.

6. CLI layer

The Cobra-based CLI exposes commands for initializing a specification, creating architectural elements, validating the configuration, inspecting the model, generating a plan, and applying it to PostgreSQL.

This separation keeps architectural intent, policy validation, SQL generation, and database mutation independently inspectable.

The current MVP was validated through automated tests, a complete CLI smoke test, and an integration test against a temporary PostgreSQL 16 instance.

The test architecture contained multiple domains, owned tables, columns, a primary key, cross-domain reading rules, and a read model implemented as a PostgreSQL view.

GrimleyTK successfully:

- loaded the declarative YAML specification;

- validated structure, ownership, references, identifiers, and SQL types;

- generated a deterministic execution plan;

- created schemas, tables, columns, a primary key, and a view;

- applied the same plan again without corrupting the existing structure;

- retrieved data through the generated view;

- rolled back the complete transaction after an intentionally introduced SQL failure;

- preserved the data that existed before the failed execution.

The PostgreSQL environment used only temporary test credentials. Its containers and volumes were removed after verification.

This test demonstrates that the core workflow works against a real PostgreSQL instance while also defining the current boundary of the project: GrimleyTK is a usable MVP, not yet a complete schema reconciliation or migration platform.