Schema Migrations and Patches

TIP

All of the schema migration functionality is surfaced through Marten's command line support and that is the Marten team's recommended approach for using the schema migration functionality described in this page.

While it's going to be far less mechanical work than persisting an application via relational tables, Marten still needs to create matching schema objects in your Postgresql database and you'll need some mechanism for keeping your database schema up to date with the Marten StoreOptions configuration in your system.

Development Time with "Auto Create" Mode

WARNING

Heads up, all the API methods for invoking schema checks or patches or migrations are now asynchronous as of Marten V4.

As long as you have rights to alter your Postgresql database, you can happily set up Marten in one of the permissive "AutoCreate" modes and not worry about schema changes at all as you happily code new features and change existing document types:

var store = DocumentStore.For(opts =>
{
    // Marten will create any new objects that are missing,
    // attempt to update tables if it can, but drop and replace
    // tables that it cannot patch.
    opts.AutoCreateSchemaObjects = AutoCreate.All;

    // Marten will create any new objects that are missing or
    // attempt to update tables if it can. Will *never* drop
    // any existing objects, so no data loss
    opts.AutoCreateSchemaObjects = AutoCreate.CreateOrUpdate;

    // Marten will create missing objects on demand, but
    // will not change any existing schema objects
    opts.AutoCreateSchemaObjects = AutoCreate.CreateOnly;

    // Marten will not create or update any schema objects
    // and throws an exception in the case of a schema object
    // not reflecting the Marten configuration
    opts.AutoCreateSchemaObjects = AutoCreate.None;
});

^{snippet source | anchor}

As long as you're using a permissive auto creation mode (i.e., not None), you should be able to code in your application model and let Marten change your development database as needed behind the scenes to match the active configuration.

TIP

In all of the usages shown below, the database migration functionality is able to function across the databases in a multi-tenancy by separate databases strategy.

Exporting Database Migrations

It's somewhat unlikely that any self-respecting DBA is going to allow your application to have rights to execute schema changes programmatically, so we're stuck needing some kind of migration strategy as we add document types, Javascript transformations, and retrofit indexes. Fortunately, we've got a strong facility to detect and generate database migration scripts.

In usage, you would first need to tell Marten about every possible document type, any event store usage, and any javascript transforms so that Marten "knows" how to make the full comparison:

using var store = DocumentStore.For(_ =>
{
    // This is enough to tell Marten that the User
    // document is persisted and needs schema objects
    _.Schema.For<User>();

    // Lets Marten know that the event store is active
    _.Events.AddEventType(typeof(MembersJoined));
});

^{snippet source | anchor}

The easiest possible way to export SQL migration files is to use Marten's command line tooling with the single command:

dotnet run -- db-patch [filename]

or for backward compatibility with Marten <V5.0

dotnet run -- marten-patch [filename]

If you'd rather write a database SQL migration file with your own code, bootstrap your IDocumentStore pointing to the database connection you want to update, and use:

// All migration code is async now!
await store.Storage.Database.WriteMigrationFileAsync("1.initial.sql");

^{snippet source | anchor}

The command above will generate a file called "1.initial.sql" to update the schema, and a second file called "1.initial.drop.sql" that attempts to rollback all of the changes from "1.initial.sql." Today, the migration mechanism covers:

1. Creates any missing database schemas
2. Document storage tables, "upsert" functions, and any configured indexes -- including missing columns or column type changes
3. Javascript transforms
4. The Hilo support table
5. The Event Store schema objects

Include in your ci/cd pipeline

While there are many options to include these exported scripts in your ci/cd pipeline, we have an example using grate on the DevOps page.

Apply All Outstanding Changes Upfront

To programmatically apply all detectable schema changes upfront , you can use this mechanism:

await store.Storage.ApplyAllConfiguredChangesToDatabaseAsync();

^{snippet source | anchor}

With the command line tooling, it's:

dotnet run -- db-apply

or in Marten <V5.0:

dotnet run -- marten-apply

Lastly, Marten V5 adds a new option to have the latest database changes detected and applied on application startup with

// The normal Marten configuration
services.AddMarten(opts =>
    {
        // This helps isolate a test, not something you need to do
        // in normal usage
        opts.ApplyChangesLockId += 18;

        opts.Connection(ConnectionSource.ConnectionString);
        opts.DatabaseSchemaName = "apply_changes";
        opts.RegisterDocumentType<User>();
    })

    // Direct the application to apply all outstanding
    // database changes on application startup
    .ApplyAllDatabaseChangesOnStartup();

^{snippet source | anchor}

In the option above, Marten is calling the same functionality within an IHostedService background task.

Assert that a Schema Matches the Configuration

As a possible environment test, Marten can do a complete check of its known configuration versus the active Postgresql database and assert any differences by throwing an exception:

await store.Storage.Database.AssertDatabaseMatchesConfigurationAsync();

^{snippet source | anchor}

The exception will list out all the DDL changes that are missing.

With the command line tooling, it's:

dotnet run -- db-assert

or

dotnet run -- marten-assert