Getting started

RAGGuard tells you when your RAG index is wrong, before your users do. It compares the documents you meant to index, the record your pipeline keeps of what it indexed, and the vectors that are really searchable, and it shows you exactly what drifted, how serious it is and how to fix it safely. This chapter explains what problem RAGGuard solves, who it is for, what you need before you start, and how to get from a new account to your first audit result. It also tours the public landing page and explains how to read the rest of this guide.

On this page


What RAGGuard is

RAGGuard is an independent checker for a RAG system. RAG stands for retrieval-augmented generation: an application answers questions by first searching an index of your documents, then giving the best matches to a language model. The index is only useful if it is correct.

RAGGuard reads three places and compares them. It always reads. It never writes to them.

Word in RAGGuardSystemWhat it represents
SourceAmazon S3 (or an S3-compatible store)What exists right now: your documents
ManifestA table in PostgreSQLWhat your pipeline believes it did: one row per document
IndexQdrantWhat is actually searchable: your vectors

Comparing them is called an audit. An audit produces:

  • Issues: each disagreement it found, with evidence. There are 11 types.
  • A status for every document, and one health score for the whole index.
  • Repair plans: a list of what to fix, with estimated cost.

When you ask for a repair, RAGGuard does not change your systems. It sends a signed request to your own worker and then checks the real index again to see whether the fix worked. Removing old vectors needs your explicit approval.

Every term in bold is explained in Concepts you need.


The problem it solves: silent RAG failures

A RAG system can go wrong while every background job reports success. The application keeps running and nobody sees an error, but users get incomplete, outdated or wrong answers. The landing page lists six ways this happens:

What goes wrongWhat it looks like
Documents that were never embeddedThe file sits in storage, but nothing about it is searchable.
Updated documents with old vectorsThe source changed. The index keeps serving the previous version with full confidence.
Deleted documents that stay searchableWithdrawn or deleted content can still be returned to your users.
Half-indexed documentsOnly some chunks made it in, so answers are quietly incomplete.
Duplicates and empty vectorsRetries double-count chunks. Zero vectors can never be retrieved.
Drift nobody plannedThe embedding model, the chunking or the permissions changed. Old documents did not.

An example. Your ingestion job processes a new HR policy and records the status success. A crash halfway through means no vectors reached Qdrant. The job is green, the dashboard is green, and the assistant never mentions the policy. RAGGuard sees a document in S3, a manifest row that claims success, and zero vectors in the index. It reports Missing embeddings as Critical, because the pipeline said it worked.


What RAGGuard does and does not do

RAGGuard doesRAGGuard does not
Read your S3 bucket, manifest table and Qdrant index, read-onlyWrite to, change or delete anything in those systems
Find 11 kinds of disagreement, each with evidenceParse documents, create embeddings or host a vector database
Show how serious each issue is and whether bad content can really be retrievedRun a chatbot
Plan repairs and estimate their costCarry out repairs itself: your worker does that
Send signed repair requests to your worker and verify the result against the real indexRemove vectors without your approval
Notify you in the app and by email after every audit, alert you through a Slack-compatible webhook, and run audits from scripts through API keysConnect to Slack, Jira or PagerDuty directly (marked Not available yet in the interface)

Who it is for

RAGGuard is for teams who run a production RAG pipeline over Amazon S3, PostgreSQL and Qdrant and need to know three things: whether the index is trustworthy, which exact documents are wrong and why, and what it will cost to fix them.

If you are...You will mostly use...
An engineer connecting RAGGuardConnect your data sources, Settings, Integrations
An on-call engineer investigating an alertOverview, Integrity issues, Documents
An owner of the data pipelineRepair plans, Impact analysis, Audit history
A security or compliance owner of a multi-tenant productIsolation, Integrity issues
Someone automating checks in CIIntegrations (API keys), Alerts

RAGGuard works with S3-compatible stores such as MinIO, Cloudflare R2 and Supabase Storage, with PostgreSQL including Supabase, and with Qdrant self-hosted or on Qdrant Cloud.


Before you start

You need two things that already exist in your own environment, plus an account. A third, the manifest, is optional.

1. An S3 bucket with your documents

  • A bucket (or a folder inside one, set with Key prefix) that holds the documents your users should be able to search.
  • A read-only access key for it. On AWS, the key needs s3:ListBucket on the bucket and s3:GetObject on its objects.
  • Object keys that RAGGuard can split into a tenant and a document id. The default Key pattern expects tenant/path/to/document.pdf, for example customer-a/policies/leave-policy.md. For a bucket with no tenants, set the pattern to ^(?P<document_id>.+)$.
  • A way to know when a file changes. The simplest is a SHA-256 stored with each object under the metadata key sha256. Alternatives are Content hash compute (RAGGuard downloads and hashes) or etag (quick start, less reliable).

2. A PostgreSQL manifest table (optional)

Skip this if your pipeline records nothing. Audits work without a manifest: RAGGuard compares your files with the vector index directly (see The manifest is optional). This is the table your ingestion pipeline writes after it processes each document. By default RAGGuard looks for rag_document_manifest in the public schema, and you can change both. You can also map your own column names.

  • Minimum: a document_id column. It is the only mandatory column, and any other column your table lacks is simply skipped.
  • Recommended: tenant_id, source_uri, source_hash, source_version, processing_status, expected_chunk_count, indexed_chunk_count, indexed_version and updated_at. Add the pipeline columns (parser_version, chunking_version, chunk_size, chunk_overlap, embedding_model, embedding_model_version, embedding_dim, preprocessing) and acl_hash if you have them.
  • Optional: a second table with one row per chunk (tenant_id, document_id, chunk_id, chunk_index, chunk_hash), which lets RAGGuard name exactly which chunks are missing.
  • A database role that can only SELECT. RAGGuard also opens every transaction read-only.

3. A vector index: a Qdrant collection, or a pgvector table

Vectors in PostgreSQL with pgvector (Supabase, Neon, AWS RDS) work too: choose pgvector on Data Sources and give it the table.

  • A Qdrant URL that RAGGuard can reach. For Qdrant Cloud it looks like https://your-cluster.cloud.qdrant.io:6333. Add an API key if the cluster requires one.
  • Minimum payload on every vector: document_id and source_hash (the same value your pipeline stored in the manifest), plus tenant_id if you have more than one tenant.
  • Recommended: chunk_id and chunk_index (so partial indexes can be found), and the pipeline fields embedding_model, embedding_dim, chunking_version and the like (so drift can be found).
  • If your payload uses other names, a Payload field map translates them.

What the three must agree on

RAGGuard can only compare what lines up. For each document:

  • The tenant and the document id must be the same in S3, the manifest and the index.
  • The hash your pipeline records must be the same value RAGGuard reads from S3, in the manifest and in each vector's payload.

If they do not match, every document looks stale, missing or orphaned. The Test connection button on Data Sources checks this for you with a sample.

What each kind of check needs

To detect...RAGGuard needs...
Missing embeddingsThe S3 file and no vectors for it. The manifest only decides the severity: Critical if it claims success.
Stale versionsA hash or version on the vectors (or a hash in the manifest) to compare with S3.
Partial indexesA chunk table, or expected_chunk_count in the manifest, or chunk_index values on the vectors.
Orphan vectorsVectors whose document is gone from S3.
Wrong-tenant vectorsA tenant_id on the vectors and a tenant in the S3 key.
Pipeline driftA Target pipeline in Settings, and fingerprint fields on the vectors or in the manifest.
Empty embeddingsDetect empty embeddings switched on in the Qdrant settings.
Documents still being processedupdated_at and processing_status in the manifest, or a recent S3 modified time.

Network and account

  • In hosted mode RAGGuard only connects to public addresses. Hosts on a private network or on localhost are refused. Your S3 endpoint, Qdrant URL and database must be reachable from the internet.
  • You need an email address and a password of 10 to 128 characters to create an account. See Sign in and find your way around.
  • A current web browser.

Three ways RAGGuard can run

The person who runs the server chooses a mode. You can tell which one you are using from what you see first.

ModeWhat you see firstSign-inConnections
HostedThe landing page, then the sign-in and Create account screensEmail and password. Each account gets a private workspace.You enter them yourself in Data Sources
Live (single-user)The app straight away, or an API key required screen if the server is protectedNoneThe server's environment variables. Data Sources is read-only.
Demo (single-user)The app straight away, with an amber Demo data bannerNoneA built-in synthetic knowledge base

Everything in this guide applies to all three, except sign-in, the account menu's Sign out and the connection forms, which only exist in hosted mode.


What demo data means

Demo data is a synthetic knowledge base that RAGGuard can run on its own: invented documents in several tenants, with faults put in on purpose. It lets you explore every screen without connecting anything. On a demo server:

  • An amber banner on every page says Demo data. This server is running a synthetic knowledge base so you can explore the product. Nothing here comes from your own systems.
  • The environment card at the bottom of the sidebar shows DEMO, with an amber dot.
  • Settings > General has a Reset demo data button that regenerates the data and clears the results.

The screenshots and the video that go with this guide come from a hosted test workspace connected to test systems that hold synthetic documents with deliberately injected faults. They are examples of what the screens look like. The numbers you see on your own data will be different, and wherever this guide quotes a number it says "for example".


Ten-minute quick start

If your systems are ready, you can reach your first result in about ten minutes. The audit itself takes as long as your corpus needs. Each step names the chapter that explains it in detail.

  1. Open RAGGuard and create your account. Press Create your workspace on the landing page, type your email and a password of at least 10 characters, and press Create account. You land on the Overview, which asks you to connect your data. See Sign in and find your way around.
  2. Connect your source and your index. Open Data Sources. For Amazon S3 and for your vector index (Qdrant, or pgvector: choose it under Where are your vectors stored?) press Configure, fill in the form, press Test connection, read the result lines, then press Save. A manifest is optional. See Connect your data sources. The Run audit button stays disabled, with the tooltip "Connect your data sources first", until the source and the index are saved.
  3. Run your first audit. Press Run audit in the top bar, or Run first audit on the Overview. A notice appears when the audit starts and another when it finishes. See Sign in and find your way around and Reconciliation.
  4. Read the results. Start on the Overview: the health score, the number of open issues and the worst problems. Then open Integrity Issues to read each issue's evidence (Integrity issues), Isolation to check that no customer's documents are indexed under another customer (Isolation), and Documents to see every document's state (Documents).
  5. Fix what you found. Create a repair plan from the issues you want fixed, send it to your worker, verify it and approve cleanup (Repair plans). Your worker must be set up first (Integrations). To see what a pipeline change would affect before you make it, use Impact analysis.
  6. Keep watching. Set up Alerts, create an API key to run audits from CI or a scheduler (Integrations), and tune the grace period and target pipeline in Settings. Every audit is recorded in Audit history.

For one continuous worked example, read End-to-end journey. For common tasks, see Practical use cases.


The landing page

If you open a hosted RAGGuard server without being signed in, the page at the root address (/) is a public landing page. It says only things that are true of the product: there are no customer logos and no statistics. Single-user servers have no landing page; they open the app directly. A signed-in user who opens the root address goes to the Overview.

The RAGGuard landing page: the headline, the Create your workspace and Sign in buttons, the three-rail illustration and the Example finding card

The top of the landing page: headline, the two buttons, the three-rail illustration and the "Example finding" card.

The top bar and the hero

ElementWhat it saysWhere it goes
LogoThe RAGGuard markThe top of the landing page
Menu links The problem, How it works, What it finds, Security, FAQSection names. Hidden on narrow screens.Scroll to that section of the same page
Menu link GuidesHidden on narrow screens.The how-to guides (/guides): eight ways a RAG index drifts from its documents, each with a check you can run yourself, and how to use RAGGuard from an AI assistant. Public.
Menu link User guideHidden on narrow screens.The Help pages (/help): the video walkthrough and every chapter of this guide. You can read them without an account.
Sign inThe Sign in screen (/login)
Get startedThe Create account screen (/signup). Shown only when sign-ups are open.
Tag line"Independent verification for RAG", next to a green Free during beta label
Headline"Know when your RAG index is wrong, before your users do."
Sub-headingRAGGuard compares your source documents with your vector index, in Qdrant or pgvector, then shows exactly what drifted, how serious it is and how to fix it safely. No processing manifest needed; connect one for extra checks.
Create your workspaceThe Create account screen. Shown only when sign-ups are open.
Book a free auditOpens your email program with a message to kishore.knaresh07@gmail.com, already titled "Free RAGGuard audit", that asks where your documents and vectors are and roughly how many documents you have
Sign in (hero)Shown only when sign-ups are closed (the top bar always has one).The Sign in screen
Promise line"Read-only access to your systems. Credentials are encrypted and never shown again."
IllustrationThree rails labelled SOURCE (Amazon S3), MANIFEST (PostgreSQL (optional)) and INDEX (Qdrant or pgvector), with documents traced across them. One has "no vectors", one has "old version", one is "still searchable" after being "deleted". It is an illustration, not real data.
Example finding cardA Critical badge, "Document has no embeddings in the index", "The pipeline reported success, but nothing about this document is searchable.", Expected "All chunks indexed", Actual "0 vectors found". An example of what an issue looks like.

The sections

The landing page sections How it works and What it finds

The "How it works" and "What it finds" sections of the landing page.

SectionHeadingWhat it says
The problem"Your pipeline says success. Your users get the wrong answer."The six silent failures in the table earlier in this chapter.
How it works"Your documents and your index, one version of the truth"Three numbered steps: 1 Connect, read-only (point RAGGuard at S3 and your vector index in Qdrant or pgvector, plus the manifest table if you have one; it only lists, reads and queries), 2 Reconcile (every document is compared across the systems you connected, chunk by chunk, and retrieval probes check whether bad content can really be returned), 3 Repair and verify (plans go to your own worker as signed requests, then RAGGuard checks the real index again; old vectors are removed only after you approve). Below the steps, See it working plays the six-minute walkthrough from the video chapter; it loads only when you press play.
What it finds"11 kinds of silent failure, each with evidence"One card per issue type with a one-line meaning and the systems it involves: Missing embeddings, Partial index, Stale document version, Superseded vectors active, Orphan vectors, Duplicate chunks, Metadata mismatch, Pipeline drift, Empty embeddings, Tenant mismatch, Orphan manifest row.
What you get"Everything you need to act on it"Six benefits: a health score, evidence for every finding, cost estimates before you fix, blast radius, safe and verified repairs, and alerts and automation.
Security"Safe to point at production"Six promises: read-only by design, credentials stay secret, workspaces are isolated, private networks are blocked, repairs are signed and removals approved, and hardened sessions.
Works with"The tools you already run"Two connections are enough to start: Amazon S3 (and S3-compatible stores) and the vector index, Qdrant (self-hosted or Cloud) or pgvector (any PostgreSQL, including LangChain and LlamaIndex tables). The PostgreSQL manifest is marked optional.
FAQ"Questions, answered"Seven questions, each of which expands when you click it: does RAGGuard change my data, what do I need to connect, how does it know what my index should contain, how much does it cost (nothing during the beta), what is a free audit, can I run audits from CI or on a schedule, and how are my credentials protected.
Closing call to action"See what your index is really serving.""Free during beta", the two buttons again, and the line "Questions first? Read the FAQ or email kishore.knaresh07@gmail.com."
FooterThe RAGGuard mark and a one-line descriptionSign in, Create account (only when sign-ups are open), FAQ, Guides, User guide, Book a free audit, Contact, Privacy and Terms.
Button or linkDestination
Every Sign inThe Sign in screen, address /login
Get started, Create your workspace, Create accountThe Create account screen, address /signup
Read the FAQ, footer FAQ, menu FAQThe FAQ section of the landing page
Menu User guide, footer User guide, More in the user guide under the videoThe Help pages, address /help
Every Book a free auditA pre-filled email to kishore.knaresh07@gmail.com titled "Free RAGGuard audit"
Footer Contact, the email address in the closing sectionAn email to kishore.knaresh07@gmail.com
Footer Privacy, footer TermsThe Privacy Policy (/privacy) and the Terms of Service (/terms). Both are public and are also reachable while you are signed in.

If the server has closed sign-ups, the hero and the closing section show Sign in next to Book a free audit, and the Create account choices disappear everywhere. See Sign in and find your way around.


How to read this guide

  • Bold text is a label exactly as it appears on screen: a button, a menu item, a field, a column heading or a status. Where the menu and the page title differ in capitals (for example the menu item Data Sources and the page title "Data sources"), this guide uses whichever you are looking at.
  • Code style is a value you type, a column name, an address or a status code.
  • Menu paths such as Settings > General mean: open Settings, then the General tab.
  • Numbers that come from an audit are examples. Your numbers depend on your data.
  • Anything the interface marks Not available yet is described as exactly that. This guide never promises it.
  • The screenshots show structure. They are not a record of your data.

The chapters in order

ChapterRead it to learn...
1. Getting startedWhat RAGGuard is and how to start (this chapter)
2. Concepts you needEvery term, with an example
3. Sign in and find your way aroundThe sign-in screens, the sidebar, the top bar, search and the three states of every page
4. Connect your data sourcesConnecting S3, PostgreSQL and Qdrant
5. OverviewThe dashboard: every card and chart
6. ReconciliationRunning audits and the three-way comparison
7. Integrity issuesReading, filtering and ignoring issues
8. Repair plansPlanning, triggering, verifying and cleaning up
9. Impact analysisWhat a change or a failure would touch
10. DocumentsEvery document and its state
11. Audit historyEvery audit, and the activity log
12. IntegrationsThe repair webhook and API keys
13. AlertsGetting told about new problems
14. SettingsPipeline target, audit policy, security and account
15. End-to-end journeyOne continuous worked example
16. Practical use casesRecipes for common situations
17. Reference: every valueEvery label, status and setting in one place
18. TroubleshootingWhat to do when something looks wrong
19. The video walkthroughThe narrated demo, scene by scene
20. IsolationWhether your index keeps customers apart: the checks, the drill and the evidence report

Suggested reading orders

  • First look (15 minutes): this chapter, the ten-minute quick start, then Overview.
  • Setting up: chapters 1 to 4, then 6 and 14.
  • Fixing a problem: chapters 7, 8 and 10, with Concepts you need open beside them.
  • Automating: chapters 12 and 13.

Tips and common mistakes

  • Connect your source and your index before you expect results. Run audit stays disabled until both are saved. The manifest is optional.
  • Make the hash match. The most common reason every document looks stale is that the hash your pipeline stores is not the one RAGGuard reads from S3. Use the same value in all three places.
  • Check the key pattern. If most documents show as missing or orphaned, the tenant and document id RAGGuard derives from S3 keys probably differ from the ones in your manifest and index. Test connection shows an example key and what RAGGuard made of it.
  • Do not judge a fresh upload. A document changed in the last few minutes shows as Pending, not broken. That is the grace period working.
  • Use a read-only database role and a read-only S3 key. RAGGuard does not need more, and the connection test warns you about a role that can write.
  • Expect empty screens before the first audit. Most pages say "No audit has run yet" and offer a Run audit button.