Most RAG examples make ingestion look like a preprocessing step. Thats ok for a demo because the documents usually never change. Production data behaves differently, a failed deployment stops an indexing worker halfway through a document, an embedding model changes while millions of vectors still belong to the previous model. The vector index now contains another representation of application data, produced asynchronously through several fallible transformations. It has its own records, identifiers, schema, lifecycle, failure modes and operational state. Whatever product name appears on the invoice, the application has acquired another database. Calling it an index doesn't remove the distributed systems problem. It just makes that problem easier to miss.
Your demo of RAG hides the truth
A basic RAG flow is easy to describe. The application reads source data, extracts text, creates chunks, generates an embedding for each chunk and stores the result. A later request embeds the user's query, searches for similar chunks and passes the best matches to a language model. Microsoft's current .NET RAG documentation describes the same basic path, process each source, chunk it, convert those chunks into a searchable form, store them and retain metadata that links the searchable representation back to its source. Microsoft.Extensions.DataIngestion now provides readers, processors, chunkers and vector-store writers for this pipeline, while Microsoft.Extensions.VectorData provides common CRUD and search abstractions across vector stores. Those libraries remove a useful amount of plumbing. They cannot decide what should happen when the source update succeeds and vectorisation fails, or when an old indexing attempt completes after a newer one. Those decisions belong to the application.
The simplest implementation often puts the two writes in the request path:
app.MapPut("/documents/{documentId:guid}", async (
Guid documentId,
UpdateDocumentRequest request,
DocumentsDbContext dbContext,
DocumentProjector projector,
CancellationToken stopToken) =>
{
var document = await dbContext.Documents
.SingleAsync(x => x.Id == documentId, stopToken);
document.ReplaceContent(request.Content);
await dbContext.SaveChangesAsync(stopToken);
await projector.ProjectAsync(document.Id, stopToken);
return Results.NoContent();
});
The code looks neat because its happy path reads in the same order as the requirement. It also creates a consistency gap immediately. The relational save and the vector-store update are independent remote operations. No ordinary database transaction covers both. If the first write succeeds and the second fails, the API reports failure even though the document changed. A client retry updates the source again and may create another version. If vectorisation succeeds but the HTTP connection disappears before the response reaches the client, the caller cannot tell whether anything happened. If several chunks are written before the vector store rejects the next one, search can observe a partially updated document. Moving ProjectAsync into a background task shortens the request, but it doesn't close any of those gaps. It changes where they occur.
A vector index is a materialised search projection
A better architectural classification is to treat the vector index as a materialised view of authoritative data. The source system owns the current document, its lifecycle and its access rules. The vector store holds a representation shaped specifically for semantic retrieval. The Azure Architecture Center's Materialized View pattern makes two points that fit RAG particularly well. A view can be stored separately from its source and optimised for a narrow set of queries. It should also be disposable and rebuildable from the source rather than updated as an independent authority. An embedding is plainly derived data. So is the extracted text from a PDF, the chunk boundary, an AI-generated summary and every piece of metadata copied into the search record. Each value depends on a particular document version and on a particular version of the projection pipeline.
That relationship gives the system a clear ownership rule. Business operations update the authoritative store. Projection workers read committed source state and create searchable representations. Retrieval treats vector matches as candidates and verifies them against current application state before those matches become model context.
The verification step is important. Approximate nearest neighbour search answers which stored vectors are close to the query vector. It doesn't prove that the underlying documents still exist, that they remain current or that this user may read them.
Every projection needs a source version
Suppose a document is edited three times while a slow indexing worker is processing its first version. Without explicit versioning, whichever worker finishes last can overwrite the index. Completion order becomes the accidental consistency policy. Give every authoritative document a monotonically increasing version. Carry that version into every chunk produced from it. A chunk then identifies both the document and the exact source state from which it was derived.
Here is a deliberately small EF Core entity:
public sealed class Document
{
private Document()
{
}
public Guid Id { get; private set; }
public Guid TenantId { get; private set; }
public long Version { get; private set; }
public string Content { get; private set; } = string.Empty;
public bool IsDeleted { get; private set; }
public DateTimeOffset UpdatedAt { get; private set; }
public void ReplaceContent(string content, TimeProvider clock)
{
Content = content;
Version++;
UpdatedAt = clock.GetUtcNow();
}
public void Delete(TimeProvider clock)
{
IsDeleted = true;
Version++;
UpdatedAt = clock.GetUtcNow();
}
}
The version belongs to the source record rather than the vector store. It advances for every change capable of altering retrieval, including content changes, deletion and any security metadata copied into the projection.
The vector record carries more than an embedding:
using Microsoft.Extensions.VectorData;
public sealed class DocumentChunk
{
[VectorStoreKey]
public string Key { get; init; } = string.Empty;
[VectorStoreData]
public Guid TenantId { get; init; }
[VectorStoreData]
public Guid DocumentId { get; init; }
[VectorStoreData]
public long DocumentVersion { get; init; }
[VectorStoreData]
public int ChunkNumber { get; init; }
[VectorStoreData]
public string Text { get; init; } = string.Empty;
[VectorStoreData]
public string ContentHash { get; init; } = string.Empty;
[VectorStoreData]
public string EmbeddingModel { get; init; } = string.Empty;
[VectorStoreData]
public string ProjectionSchema { get; init; } = string.Empty;
[VectorStoreVector(
dimensions: 1536,
DistanceFunction = DistanceFunction.CosineSimilarity)]
public ReadOnlyMemory<float> Vector { get; init; }
}
The exact attributes and supported filters depend on the selected Microsoft.Extensions.VectorData connector. The current .NET vector search guidance uses the same key, data and vector distinction, with VectorStoreCollection<TKey, TRecord> providing upsert and search operations. The chunk key should be deterministic. A value such as tenantId/documentId/documentVersion/chunkNumber/projectionSchema makes a repeated attempt overwrite the same logical record. A retry after an uncertain response doesn't create a second copy, and two document versions cannot silently overwrite one another.
private static string CreateChunkKey(
Guid tenantId,
Guid documentId,
long documentVersion,
int chunkNumber,
string projectionSchema) =>
$"{tenantId:N}/{documentId:N}/{documentVersion}/{chunkNumber}/{projectionSchema}";
A random GUID generated during every attempt throws away that property. It makes duplicate detection, repair and deletion harder for no gain.
Record the need to index in the source transaction
Saving a document and publishing a message afterwards creates another two-write problem. The process can stop after the database commit but before message publication. The document is now current, yet no worker knows that its projection is missing. Record projection work in the same database transaction as the source change. This can be a compact work ledger owned by the document feature. It needs the document identity, the committed version, a status, attempt information and enough timing data for retries and operational queries.
public sealed class SearchProjectionWork
{
private SearchProjectionWork()
{
}
public Guid Id { get; private set; }
public Guid DocumentId { get; private set; }
public long DocumentVersion { get; private set; }
public ProjectionWorkStatus Status { get; private set; }
public int AttemptCount { get; private set; }
public DateTimeOffset CreatedAt { get; private set; }
public DateTimeOffset? NextAttemptAt { get; private set; }
public static SearchProjectionWork Create(
Guid documentId,
long documentVersion,
DateTimeOffset createdAt) =>
new()
{
Id = Guid.NewGuid(),
DocumentId = documentId,
DocumentVersion = documentVersion,
Status = ProjectionWorkStatus.Pending,
CreatedAt = createdAt
};
}
public enum ProjectionWorkStatus
{
Pending,
Processing,
Completed,
Superseded,
Failed
}
The command handler changes the document and adds the work record before calling SaveChangesAsync once:
public sealed record UpdateDocumentCommand(Guid DocumentId, string Content);
public sealed class UpdateDocumentHandler(
DocumentsDbContext dbContext,
TimeProvider clock)
{
public async Task Handle(
UpdateDocumentCommand command,
CancellationToken stopToken)
{
var document = await dbContext.Documents
.SingleAsync(x => x.Id == command.DocumentId, stopToken);
document.ReplaceContent(command.Content, clock);
dbContext.SearchProjectionWork.Add(
SearchProjectionWork.Create(
document.Id,
document.Version,
clock.GetUtcNow()));
await dbContext.SaveChangesAsync(stopToken);
}
}
After that transaction commits, the application has either both the new document version and its projection request, or neither. A worker can poll the ledger, or a relay can publish the work identifier to a broker for lower latency. The database row remains the recoverable record if broker delivery fails. This doesn't make the relational database and vector store participate in one transaction. It removes the dangerous interval in which committed source state can become permanently invisible to the indexing process.
Build immutable versions before publishing them
Writing chunks directly over the currently searchable records allows a half-finished attempt to leak into retrieval. The safer approach writes a new, immutable document version alongside the old one. The worker loads the requested source version, creates all chunks, generates all embeddings and upserts records whose keys include that version. Only after every required chunk has been stored does it mark the version as published in the authoritative database.
The publication state can remain very small:
public sealed class SearchProjectionState
{
private SearchProjectionState()
{
}
public Guid DocumentId { get; private set; }
public long? PublishedDocumentVersion { get; private set; }
public string? ProjectionSchema { get; private set; }
public DateTimeOffset? PublishedAt { get; private set; }
}
The distinction between stored and published is deliberate. A process can stop after writing two of ten chunks. Those two records exist, but retrieval will reject them because the corresponding version was never published. If the process stops after writing all ten chunks but before updating SearchProjectionState, the retry writes the same ten deterministic keys and then publishes. If publication succeeds but cleanup of the old version fails, both versions remain in the vector store, but only the current published version is eligible to reach the prompt. The projector can depend on narrow application interfaces while the infrastructure layer adapts a concrete VectorStoreCollection<string, DocumentChunk>:
public interface IVectorChunkStore
{
Task UpsertAsync(
IReadOnlyCollection<DocumentChunk> chunks,
CancellationToken stopToken);
Task DeleteDocumentVersionAsync(
Guid tenantId,
Guid documentId,
long documentVersion,
CancellationToken stopToken);
}
public interface IDocumentChunker
{
IReadOnlyList<string> Split(string content);
}
The implementation below focuses on the consistency boundary rather than connector specific batching:
using System.Security.Cryptography;
using System.Text;
using Microsoft.Extensions.AI;
public sealed class DocumentProjector(
DocumentsDbContext dbContext,
IDocumentChunker chunker,
IEmbeddingGenerator<string, Embedding<float>> embeddingGenerator,
IVectorChunkStore vectorStore,
ProjectionPublisher publisher)
{
private const string EmbeddingModel = "text-embedding-3-small";
private const string ProjectionSchema = "document-rag-v3";
public async Task ProjectAsync(
Guid documentId,
long requestedVersion,
CancellationToken stopToken)
{
var document = await dbContext.Documents
.AsNoTracking()
.SingleAsync(x => x.Id == documentId, stopToken);
if (document.Version != requestedVersion)
{
await publisher.MarkSupersededAsync(
documentId,
requestedVersion,
stopToken);
return;
}
if (document.IsDeleted)
{
await publisher.PublishDeletionAsync(
documentId,
requestedVersion,
stopToken);
return;
}
var texts = chunker.Split(document.Content);
var chunks = new List<DocumentChunk>(texts.Count);
for (var chunkNumber = 0; chunkNumber < texts.Count; chunkNumber++)
{
var text = texts[chunkNumber];
var vector = await embeddingGenerator.GenerateVectorAsync(
text,
cancellationToken: stopToken);
chunks.Add(new DocumentChunk
{
Key = CreateChunkKey(
document.TenantId,
document.Id,
document.Version,
chunkNumber,
ProjectionSchema),
TenantId = document.TenantId,
DocumentId = document.Id,
DocumentVersion = document.Version,
ChunkNumber = chunkNumber,
Text = text,
ContentHash = Convert.ToHexString(
SHA256.HashData(Encoding.UTF8.GetBytes(text))),
EmbeddingModel = EmbeddingModel,
ProjectionSchema = ProjectionSchema,
Vector = vector
});
}
await vectorStore.UpsertAsync(chunks, stopToken);
await publisher.TryPublishAsync(
document.Id,
document.Version,
ProjectionSchema,
stopToken);
}
private static string CreateChunkKey(
Guid tenantId,
Guid documentId,
long documentVersion,
int chunkNumber,
string projectionSchema) =>
$"{tenantId:N}/{documentId:N}/{documentVersion}/{chunkNumber}/{projectionSchema}";
}
TryPublishAsync must be conditional. It should publish version 12 only if the authoritative document is still at version 12 and the existing published version has not advanced beyond it. A transaction or conditional SQL update can enforce that rule. Reading the state, checking it in C# and saving without concurrency protection would allow an older worker to replace a newer publication. That final condition handles an easy to miss race. Version 12 can begin first, version 13 can finish first, and version 12 can then finish last. The vector records may arrive in either order. Publication must remain monotonic.
Retrieval should distrust its own search results
Versioned records prevent destructive overwrites, but old vectors can still be returned by similarity search. Physical cleanup is normally asynchronous, so correctness cannot depend on it completing immediately. Search should therefore have two stages. The vector store produces candidates. The application then loads current document and projection state for the returned document IDs, applies current authorisation, rejects stale or unpublished versions and only then constructs model context.
public sealed record VectorCandidate(
Guid TenantId,
Guid DocumentId,
long DocumentVersion,
string ProjectionSchema,
string Text,
double Score);
public sealed record GroundingChunk(
Guid DocumentId,
string Text,
double Score);
public interface ICurrentUserAccess
{
Task<HashSet<Guid>> GetReadableDocumentIdsAsync(
IReadOnlyCollection<Guid> documentIds,
CancellationToken stopToken);
}
public sealed class GroundingRetriever(
IVectorSearch vectorSearch,
DocumentsDbContext dbContext,
ICurrentUserAccess currentUserAccess)
{
public async Task<IReadOnlyList<GroundingChunk>> RetrieveAsync(
Guid tenantId,
string query,
int requiredCount,
CancellationToken stopToken)
{
var candidates = await vectorSearch.SearchAsync(
tenantId,
query,
top: requiredCount * 5,
stopToken);
var documentIds = candidates
.Select(x => x.DocumentId)
.Distinct()
.ToArray();
var states = await dbContext.Documents
.Where(x => documentIds.Contains(x.Id))
.Join(
dbContext.SearchProjectionStates,
document => document.Id,
projection => projection.DocumentId,
(document, projection) => new
{
document.Id,
document.TenantId,
document.Version,
document.IsDeleted,
projection.PublishedDocumentVersion,
projection.ProjectionSchema
})
.ToDictionaryAsync(x => x.Id, stopToken);
var readableDocumentIds = await currentUserAccess
.GetReadableDocumentIdsAsync(documentIds, stopToken);
var accepted = new List<GroundingChunk>(requiredCount);
foreach (var candidate in candidates.OrderByDescending(x => x.Score))
{
if (!states.TryGetValue(candidate.DocumentId, out var state))
{
continue;
}
if (state.TenantId != tenantId || state.IsDeleted)
{
continue;
}
if (state.Version != candidate.DocumentVersion ||
state.PublishedDocumentVersion != candidate.DocumentVersion ||
state.ProjectionSchema != candidate.ProjectionSchema)
{
continue;
}
if (!readableDocumentIds.Contains(candidate.DocumentId))
{
continue;
}
accepted.Add(new GroundingChunk(
candidate.DocumentId,
candidate.Text,
candidate.Score));
if (accepted.Count == requiredCount)
{
break;
}
}
return accepted;
}
}
The initial search applies the tenant filter inside the vector store. That reduces both leakage risk and wasted candidates, but it isn't the final authorisation decision. Fine grained access is checked using current application data before any text is placed in the prompt. Over fetching compensates for stale candidates that will be rejected. Five times the requested count is an example rather than a universal constant. A busy index with heavy churn may need iterative retrieval: request a page, validate it, then fetch more if too few current results survive.
This validation also changes how the system fails. A delayed index update can produce fewer results or a temporary "knowledge is still being prepared" response. It cannot silently feed a withdrawn document to the model simply because deletion cleanup is running late.
Deletion and access revocation are correctness paths
Teams often treat deletion from the vector store as storage maintenance. In a RAG system, deleted text can continue influencing generated answers. An access change can be even more urgent because the content still exists but the current caller is no longer entitled to see it. A source deletion should increment the document version and create projection work in the same commit. Current state validation will reject every earlier chunk as soon as that transaction completes, even while the physical vector deletion is pending. The cleanup worker can then remove all versions of the document and retain a tombstone or completion record for reconciliation.
Access revocation needs a similar immediate effect. If permissions are represented only as copied metadata in the vector index, the security boundary inherits indexing lag. Keeping current authorisation in the retrieval path closes that interval. Coarse, stable attributes such as tenant identity can still be indexed for efficient candidate filtering, while volatile user and role decisions remain authoritative elsewhere. There is a cost, retrieval now performs another read and possibly several authorisation checks. Batch those reads and cache only where invalidation semantics are acceptable. Saving a few milliseconds by trusting stale security metadata is a poor exchange when retrieved text is about to leave the deterministic part of the system and enter a prompt.
Partial success is normal ingestion behaviour
The .NET data-ingestion pipeline explicitly represents partial success. Its ProcessAsync API returns results per document so the caller can decide whether to retry failures or stop. Production indexing code needs the same assumption at every layer. Text extraction can succeed while enrichment fails. Nine chunks can embed successfully while the tenth is rate limited. Every vector upsert can succeed while publication times out. Cleanup of the superseded version can fail after the new one becomes current.
These aren't equivalent outcomes. Before publication, a failed attempt can be repeated using the same deterministic keys. After publication, cleanup can proceed independently because read time validation already protects correctness. Permanent failures should remain visible in the work ledger with their document version and projection schema, rather than disappearing into logs after a fixed number of retries. A dead letter state is useful only when something owns it. Operators need enough data to decide whether to retry, skip, correct the source document or roll back a projection release. "Embedding failed" without the model, source version, chunk number and provider response category rarely supports that decision.
Embedding changes are database migrations
An embedding model maps text into a particular vector space. Changing the model can change the vector dimensions and will change the meaning of the coordinates even when the dimensions happen to match. Query vectors from the new model should not be compared with document vectors from the old model. Chunking changes have similar consequences. Altering token limits, overlap, heading rules, OCR behaviour or enrichment prompts changes the searchable records. A source document at version 18 may therefore have several valid historical projections, each produced by a different pipeline.
Treat those changes as projection schema migrations. Give the entire pipeline a version such as document-rag-v3, record the embedding model separately and build the replacement into a new collection or index. Continue serving queries from the current index while the new one is populated and evaluated. Switch a configuration pointer or search alias only when the replacement is complete enough to serve production traffic. This blue green approach costs additional storage and embedding calls during migration. In return, it avoids a long period in which one search collection contains incompatible vectors or an unpredictable mixture of old and new chunking behaviour. Rollback also becomes possible. If retrieval quality falls after the switch, route queries back to the earlier projection while investigating. Re-embedding the entire corpus again shouldn't be the rollback plan.
Reconciliation is how the system discovers quiet failures
A durable work record covers known updates, but production systems also need a way to prove that the projection still corresponds to the source. Bugs, manual data fixes, expired dead letters and operational mistakes can all bypass the path engineers expected. A reconciler compares authoritative documents with SearchProjectionState. A live document whose current version isn't published needs new work. A deleted document with remaining vector records needs cleanup. A projection built with a retired schema belongs to a migration backlog. A work item stuck in Processing beyond its lease needs to be reclaimed.
This should be routine background work rather than a disaster recovery script written during an incident. Because the index is a derived projection, rebuilding one document, one tenant or the whole corpus should be an ordinary supported operation. The most useful operational measurement is projection lag: the time between the authoritative commit and publication of its searchable version. Track its median and tail percentiles, not only an average. Also expose the number of current documents awaiting projection, permanently failed versions, rejected stale search candidates and orphaned vector records awaiting deletion.
Those signals reveal different faults. Growing lag suggests insufficient worker capacity or provider throttling. A sudden rise in rejected candidates suggests cleanup failure or a stuck projection version. A stable queue with a rising age suggests poison documents repeatedly taking the same worker slots.
Decide what freshness the product promises
Not every RAG feature requires the same consistency behaviour. An internal assistant over slowly changing engineering guidance may tolerate a few minutes of indexing delay. A system answering questions about active insurance terms, revoked access or current prices may not. Make that promise explicit. A strict read path accepts chunks only when the published projection version equals the current source version. During indexing it returns fewer results or tells the caller that the document is still being prepared. A relaxed path may allow the last published version for a bounded period, provided the document still exists and the caller remains authorised. The important part is that the choice belongs to the product and domain. It should not emerge accidentally from how quickly a queue happens to drain. For highly sensitive changes, an update can mark the document unavailable to retrieval until its new version is published. That sacrifices temporary availability for currentness. Other domains may continue serving the previous version and display its effective timestamp. Both policies can be valid when they are chosen deliberately and observable in production.
Test the gaps between the successful lines
A test that inserts one document, indexes it and retrieves it proves the happy path already shown by most quickstarts. The valuable tests stop the workflow between its durable state transitions. Stop the projector after the source commit but before the first vector write and confirm that pending work survives. Stop it after several chunk writes and confirm that none are accepted because the version remains unpublished. Stop it after every upsert but before publication and confirm that retrying writes the same keys. Run version 12 and version 13 concurrently, complete them in reverse order and confirm that version 12 cannot replace version 13.
Delete a document while an older projection attempt is running. Revoke access while its chunks remain in the index. Make physical cleanup fail for several hours. Retrieval should reject the content in every case. Migration tests should populate old and new projection schemas together, generate query embeddings with the corresponding model and verify that traffic never crosses the two vector spaces. Reconciliation tests should remove a work row or projection record deliberately and prove that the missing state is discovered and repaired. These tests are more revealing than asserting a particular cosine score from a tiny in-memory store. They exercise ownership, ordering and recovery, which are the properties production failures will challenge.
The architecture behind reliable RAG
Microsoft.Extensions.DataIngestion can read, transform, chunk and enrich documents. Microsoft.Extensions.AI can generate embeddings. Microsoft.Extensions.VectorData can write records and search them through a common .NET abstraction. Together they provide a far better starting point than every application inventing that plumbing independently. Reliability still comes from the state surrounding those calls. The source record needs a version. Projection work needs to be committed with the source change. Chunk identities need to survive retries. Publication needs to be separate from storage and monotonic across concurrent attempts. Retrieval needs to verify freshness, deletion and authorisation. Reconciliation needs to prove that quiet failures haven't left the index behind.
Once those pieces exist, the vector store can fulfil its proper role, a fast, specialised and replaceable search projection. Without them, it becomes an unacknowledged second authority containing an unknown mixture of current, stale, partial and unauthorised data. The language model will answer from whichever chunks the application supplies. Keeping those chunks aligned with reality is a database consistency problem long before it becomes a prompt engineering problem.
