Full Text Searching

Full Text Indexes in Marten are built based on GIN or GiST indexes utilizing Postgres built in Text Search functions. This enables the possibility to do more sophisticated searching through text fields.

WARNING

To use this feature, you will need to use PostgreSQL version 10.0 or above, as this is the first version that support text search function on jsonb column - this is also the data type that Marten use to store it's data.

Defining Full Text Index through Store options

Full Text Indexes can be created using the fluent interface of StoreOptions like this:

• one index for whole document - all document properties values will be indexed

var store = DocumentStore.For(_ =>
{
    _.Connection(ConnectionSource.ConnectionString);

    // This creates
    _.Schema.For<User>().FullTextIndex();
});

^{snippet source | anchor}

INFO

If you don't specify language (regConfig) - by default it will be created with 'english' value.

• single property - there is possibility to specify specific property to be indexed

var store = DocumentStore.For(_ =>
{
    _.Connection(ConnectionSource.ConnectionString);

    // This creates
    _.Schema.For<User>().FullTextIndex(d => d.FirstName);
});

^{snippet source | anchor}

• single property with custom settings

var store = DocumentStore.For(_ =>
{
    _.Connection(ConnectionSource.ConnectionString);

    // This creates
    _.Schema.For<User>().FullTextIndex(
        index =>
        {
            index.Name = "mt_custom_italian_user_fts_idx";
            index.RegConfig = "italian";
        },
        d => d.FirstName);
});

^{snippet source | anchor}

• multiple properties

var store = DocumentStore.For(_ =>
{
    _.Connection(ConnectionSource.ConnectionString);

    // This creates
    _.Schema.For<User>().FullTextIndex(d => d.FirstName, d => d.LastName);
});

^{snippet source | anchor}

• multiple properties with custom settings

var store = DocumentStore.For(_ =>
{
    _.Connection(ConnectionSource.ConnectionString);

    // This creates
    _.Schema.For<User>().FullTextIndex(
        index =>
        {
            index.Name = "mt_custom_italian_user_fts_idx";
            index.RegConfig = "italian";
        },
        d => d.FirstName, d => d.LastName);
});

^{snippet source | anchor}

• more than one index for document with different languages (regConfig)

var store = DocumentStore.For(_ =>
{
    _.Connection(ConnectionSource.ConnectionString);

    // This creates
    _.Schema.For<User>()
        .FullTextIndex(d => d.FirstName) //by default it will use "english"
        .FullTextIndex("italian", d => d.LastName);
});

^{snippet source | anchor}

Defining Full Text Index through Attribute

Full Text Indexes can be created using the [FullTextIndex] attribute like this:

• one index for whole document - by setting attribute on the class all document properties values will be indexed

[FullTextIndex]
public class Book
{
    public Guid Id { get; set; }

    public string Title { get; set; }

    public string Author { get; set; }

    public string Information { get; set; }
}

^{snippet source | anchor}

• single property

public class UserProfile
{
    public Guid Id { get; set; }

    [FullTextIndex] public string Information { get; set; }
}

^{snippet source | anchor}

INFO

If you don't specify regConfig - by default it will be created with 'english' value.

• single property with custom settings

public class UserDetails
{
    private const string FullTextIndexName = "mt_custom_user_details_fts_idx";

    public Guid Id { get; set; }

    [FullTextIndex(IndexName = FullTextIndexName, RegConfig = "italian")]
    public string Details { get; set; }
}

^{snippet source | anchor}

• multiple properties

public class Article
{
    public Guid Id { get; set; }

    [FullTextIndex] public string Heading { get; set; }

    [FullTextIndex] public string Text { get; set; }
}

^{snippet source | anchor}

INFO

To group multiple properties into single index you need to specify the same values in IndexName parameters.

• multiple indexes for multiple properties with custom settings

public class BlogPost
{
    public Guid Id { get; set; }

    public string Category { get; set; }

    [FullTextIndex] public string EnglishText { get; set; }

    [FullTextIndex(RegConfig = "italian")] public string ItalianText { get; set; }

    [FullTextIndex(RegConfig = "french")] public string FrenchText { get; set; }
}

^{snippet source | anchor}

Text Search

Postgres contains built in Text Search functions. They enable the possibility to do more sophisticated searching through text fields. Marten gives possibility to define (full text indexes)(/documents/configuration/full_text) and perform queries on them. Currently four types of full Text Search functions are supported:

• regular Search (to_tsquery)

var posts = session.Query<BlogPost>()
    .Where(x => x.Search("somefilter"))
    .ToList();

^{snippet source | anchor}

• plain text Search (plainto_tsquery)

var posts = session.Query<BlogPost>()
    .Where(x => x.PlainTextSearch("somefilter"))
    .ToList();

^{snippet source | anchor}

• phrase Search (phraseto_tsquery)

var posts = session.Query<BlogPost>()
    .Where(x => x.PhraseSearch("somefilter"))
    .ToList();

^{snippet source | anchor}

• web-style Search (websearch_to_tsquery, supported from Postgres 11+

var posts = session.Query<BlogPost>()
    .Where(x => x.WebStyleSearch("somefilter"))
    .ToList();

^{snippet source | anchor}

All types of Text Searches can be combined with other Linq queries

var posts = session.Query<BlogPost>()
    .Where(x => x.Category == "LifeStyle")
    .Where(x => x.PhraseSearch("somefilter"))
    .ToList();

^{snippet source | anchor}

They allow also to specify language (regConfig) of the text search query (by default english is being used)

var posts = session.Query<BlogPost>()
    .Where(x => x.PhraseSearch("somefilter", "italian"))
    .ToList();

^{snippet source | anchor}

Partial text search in a multi-word text (NGram search)

Marten provides the ability to search partial text or words in a string containing multiple words using NGram search. This is quite similar in functionality to NGrams in Elastic Search. As an example, we can now accurately match rich com text within Communicating Across Contexts (Enriched). NGram search uses English by default. NGram search also encompasses and handles unigrams, bigrams and trigrams. This functionality is added in v5.

var result = await session
    .Query<User>()
    .Where(x => x.UserName.NgramSearch(term))
    .ToListAsync();

^{snippet source | anchor}

var store = DocumentStore.For(_ =>
{
    _.Connection(Marten.Testing.Harness.ConnectionSource.ConnectionString);

    _.DatabaseSchemaName = "ngram_test";

    // This creates an ngram index for efficient sub string based matching
    _.Schema.For<User>().NgramIndex(x => x.UserName);
});

await using var session = store.LightweightSession();

string term = null;
for (var i = 1; i < 4; i++)
{
    var guid = $"{Guid.NewGuid():N}";
    term ??= guid.Substring(5);

    var newUser = new User(i, $"Test user {guid}");

    session.Store(newUser);
}

await session.SaveChangesAsync();

var result = await session
    .Query<User>()
    .Where(x => x.UserName.NgramSearch(term))
    .ToListAsync();

^{snippet source | anchor}

var result = await session
    .Query<User>()
    .Where(x => x.Address.Line1.NgramSearch(term))
    .ToListAsync();

^{snippet source | anchor}