Event Store Multi-Tenancy

The event store feature in Marten supports an opt-in multi-tenancy model that captures events by the current tenant. Use this syntax to specify that:

var store = DocumentStore.For(opts =>
{
    opts.Connection("some connection string");

    // And that's all it takes, the events are now multi-tenanted
    opts.Events.TenancyStyle = TenancyStyle.Conjoined;
});

^{snippet source | anchor}

Global Streams & Projections Within Multi-Tenancy 8.5

Document storage allows you to mix conjoined- and single-tenanted documents in one database. You can now do the same thing with event storage and projected aggregate documents from SingleStreamProjection<TDoc, TId> projections.

Let's say that you have a document (cut us some slack, this came from testing) called SpecialCounter that is aggregated from events in your system that otherwise has a conjoined tenancy model for the event store, but SpecialCounter should be global within your system.

Let's start with a possible implementation of a single stream projection:

public partial class SpecialCounterProjection: SingleStreamProjection<SpecialCounter, Guid>
{
    public void Apply(SpecialCounter c, SpecialA _) => c.ACount++;
    public void Apply(SpecialCounter c, SpecialB _) => c.BCount++;
    public void Apply(SpecialCounter c, SpecialC _) => c.CCount++;
    public void Apply(SpecialCounter c, SpecialD _) => c.DCount++;

}

^{snippet source | anchor}

Or this equivalent, but see how I'm explicitly registering event types, because that's going to be important:

public partial class SpecialCounterProjection2: SingleStreamProjection<SpecialCounter, Guid>
{
    public SpecialCounterProjection2()
    {
        // This is normally just an optimization for the async daemon,
        // but as a "global" projection, this also helps Marten
        // "know" that all events of these types should always be captured
        // to the default tenant id
        IncludeType<SpecialA>();
        IncludeType<SpecialB>();
        IncludeType<SpecialC>();
        IncludeType<SpecialD>();
    }

    public void Apply(SpecialCounter c, SpecialA _) => c.ACount++;
    public void Apply(SpecialCounter c, SpecialB _) => c.BCount++;
    public void Apply(SpecialCounter c, SpecialC _) => c.CCount++;
    public void Apply(SpecialCounter c, SpecialD _) => c.DCount++;

    public override SpecialCounter Evolve(SpecialCounter snapshot, Guid id, IEvent e)
    {
        snapshot ??= new SpecialCounter { Id = id };
        switch (e.Data)
        {
            case SpecialA _:
                snapshot.ACount++;
                break;
            case SpecialB _:
                snapshot.BCount++;
                break;
            case SpecialC _:
                snapshot.CCount++;
                break;
            case SpecialD _:
                snapshot.DCount++;
                break;
        }

        return snapshot;
    }
}

^{snippet source | anchor}

And finally, let's register our projection within our application's bootstrapping:

var builder = Host.CreateApplicationBuilder();
builder.Services.AddMarten(opts =>
{
    opts.Connection(builder.Configuration.GetConnectionString("marten"));

    // The event store has conjoined tenancy...
    opts.Events.TenancyStyle = TenancyStyle.Conjoined;

    // But we want any events appended to a stream that is related
    // to a SpecialCounter to be single or global tenanted
    // And this works with any ProjectionLifecycle
    opts.Projections.AddGlobalProjection(new SpecialCounterProjection(), ProjectionLifecycle.Inline);
});

^{snippet source | anchor}

The impact of this global registration is that any events appended to a stream with an aggregate type of SpecialCounter or really any events at all of the types known to be included in the globally registered single stream projection will be appended as the default tenant id no matter what the session's tenant id is. There's a couple implications here:

1. The event types of a globally applied projection should not be used against other types of streams
2. Marten "corrects" the tenant id applied to events from globally projected aggregates regardless of how the events are appended or how the session was created
3. Marten automatically marks the storage for the aggregate type as single tenanted
4. Live, Async, or Inline projections have all been tested with this functionality
5. AppendOptimistic() and AppendPessimistic() do not work (yet) with this setting, but you should probably be using FetchForWriting() instead anyway.

Per-Tenant Event Partitioning 9.4

TIP

This is an advanced, opt-in option aimed at large multi-tenanted event stores where a single, shared event store becomes a scalability bottleneck. It builds on the conjoined event tenancy described above by physically isolating each tenant's events and giving the async daemon a per-tenant view of progress. See JasperFx/marten#4596 and CritterStack #209 for the full design.

For systems with many tenants and very high event volumes, you can opt into per-tenant event partitioning. This layers native PostgreSQL LIST partitioning by tenant_id on top of the conjoined event tenancy model so that each tenant's events and streams live in their own physical partitions, get their own event sequence, and are tracked independently by the asynchronous projection daemon:

var store = DocumentStore.For(opts =>
{
    opts.Connection("some connection string");

    // Per-tenant partitioning requires conjoined event tenancy
    opts.Events.TenancyStyle = TenancyStyle.Conjoined;

    // Per-tenant partitioning only supports the "quick" append modes
    opts.Events.AppendMode = EventAppendMode.Quick;

    // Opt into per-tenant event partitioning
    opts.Events.UseTenantPartitionedEvents = true;
});

When UseTenantPartitionedEvents is enabled, Marten:

• Partitions mt_events and mt_streams by tenant_id using native PostgreSQL LIST partitioning. This reuses the same managed-partition machinery (the mt_tenant_partitions lookup table) as document partitioning, but opting into per-tenant events does not implicitly partition your multi-tenanted document tables.
• Gives each tenant its own event sequence (mt_events_sequence_{tenant_suffix}) instead of a single global sequence, so high-volume tenants no longer contend on one shared sequence.
• Keys mt_event_progression by (name, tenant_id), so projection progress is tracked per tenant rather than for the store as a whole.
• Runs the async daemon with a vectorized per-tenant high-water mark — one query per database reports the high-water position for every active tenant in a single round trip — plus per-tenant rebuild isolation, so a projection can be rebuilt for a single tenant without tearing down or replaying every other tenant's progress.

Constraints

Per-tenant partitioning is validated at DocumentStore construction. The following combinations throw immediately rather than failing opaquely later:

• Requires TenancyStyle.Conjoined. There is nothing to partition by when every event lives in the default tenant.
• Requires a "quick" append mode (EventAppendMode.Quick or EventAppendMode.QuickWithServerTimestamps). The per-tenant sequence pick is wired into the quick-append code path only; EventAppendMode.Rich assigns sequences ahead of time from a shared reader and is explicitly out of scope. See "Rich" vs "Quick" Appends.
• Cannot currently be combined with UseArchivedStreamPartitioning. Sub-partitioning the event tables by both tenant_id and is_archived is a planned follow-up; pick one for now.

Registering Tenants

As with document-level managed partitioning, a tenant's partitions must exist before its events can be appended. Register tenants through the admin API:

await store.Advanced.AddMartenManagedTenantsAsync(
    cancellationToken,
    "tenant-a", "tenant-b", "tenant-c");

This creates the LIST partitions (and per-tenant sequence) for each tenant across the partitioned event tables. When rebuilding a projection across every tenant, the daemon discovers the full set of registered tenants from the mt_tenant_partitions table and fans out into independent per-tenant rebuilds.

TIP

Per-tenant event partitioning composes with the Sharded Multi-Tenancy with Database Pooling model: sharding distributes tenants across a pool of databases, and per-tenant partitioning physically isolates each tenant's events within whichever database hosts that tenant.

Dropping Tenants

Both routes that remove a tenant under UseTenantPartitionedEvents clean up the full per-tenant footprint -- partition tables, the freestanding mt_events_sequence_{tenantId} sequence, and the per-tenant mt_event_progression rows (one per projection's per-tenant catch-up plus the HighWaterMark:{tenantId} row):

// Wipe all data for a tenant, keep the partition registered (re-seeding works after this)
await store.Advanced.DeleteAllTenantDataAsync("tenant-a", cancellationToken);

// Remove the tenants entirely (drops their partitions + cleanup)
await store.Advanced.RemoveMartenManagedTenantsAsync(
    new[] { "tenant-b", "tenant-c" }, cancellationToken);

Store-global progression rows (the HighWaterMark constant, MyProjection:All without a tenant suffix) are intentionally preserved by the per-tenant cleanup -- they belong to the store as a whole. Other tenants' partitions, sequences, and progression rows are untouched.

The cleanup identifies per-tenant mt_event_progression rows by parsing the ShardName grammar rather than pattern-matching the name, so a projection whose name happens to end with a tenant id is never mistakenly deleted (#4683).