Skip to content

Independent verification for RAGFree during beta

Know when your RAG index is wrong, before your users do.

RAGGuard compares your source documents with your vector index, in Qdrant or pgvector, then shows exactly what drifted, how serious it is and how to fix it safely. No processing manifest needed; connect one for extra checks.

Read-only access to your systems. Credentials are encrypted and never shown again.

SOURCEAmazon S3MANIFESTPostgreSQL (optional)INDEXQdrant or pgvectorno vectorsold versiondeleteddeletedstill searchable
CriticalExample finding

Document has no embeddings in the index

The pipeline reported success, but nothing about this document is searchable.

Expected

All chunks indexed

Actual

0 vectors found

The problem

Your pipeline says success. Your users get the wrong answer.

A RAG system can fail silently while every background job reports success. The app keeps running, and nobody finds out until an answer is wrong.

  • Documents that were never embedded

    The file sits in storage, but nothing about it is searchable.

  • Updated documents with old vectors

    The source changed. The index keeps serving the previous version with full confidence.

  • Deleted documents that stay searchable

    Withdrawn or deleted content can still be returned to your users.

  • Half-indexed documents

    Only some chunks made it in, so answers are quietly incomplete.

  • Duplicates and empty vectors

    Retries double-count chunks. Zero vectors can never be retrieved.

  • Drift nobody planned

    The embedding model, the chunking or the permissions changed. Old documents did not.

How it works

Your documents and your index, one version of the truth

Your documents and your index should agree, and so should your manifest if you keep one. RAGGuard checks that they do, continuously.

  1. 1

    Connect, read-only

    Point RAGGuard at your S3 bucket and your vector index, in Qdrant or pgvector. Add the manifest table your pipeline writes to, if you have one, for extra checks. It only ever lists, reads and queries.

  2. 2

    Reconcile

    Every document is compared across the systems you connected, chunk by chunk. Retrieval probes check whether bad content can actually be returned to a user.

  3. 3

    Repair and verify

    Plans go to your own worker as signed requests. RAGGuard then checks the real index again. Old vectors are removed only after you approve.

See it working

A six-minute walkthrough with narration and captions: the Documents page and the Isolation page, on demo data. More in the user guide.

What it finds

11 kinds of silent failure, each with evidence

Every finding says what is wrong, why it matters, which systems are involved and what to do about it.

  • Missing embeddings

    The document is in storage, but nothing about it is searchable.

    Amazon S3Vector index

  • Partial index

    Only some chunks were indexed, so answers are quietly incomplete.

    Vector indexManifest

  • Stale document version

    The source changed, but the index still serves the old version.

    Amazon S3Vector index

  • Superseded vectors active

    Old-generation vectors still answer queries next to the new ones.

    Vector index

  • Orphan vectors

    A deleted document is still searchable.

    Amazon S3Vector index

  • Duplicate chunks

    The same chunk was indexed more than once.

    Vector index

  • Metadata mismatch

    Source, manifest and payloads disagree on counts, versions or permissions.

    ManifestVector index

  • Pipeline drift

    Vectors were built with a different parser, chunker or embedding model.

    ManifestVector index

  • Empty embeddings

    Points with a missing or all-zero vector can never be retrieved.

    Vector index

  • Tenant mismatch

    Vectors tagged with the wrong tenant can leak one customer's content to another.

    Vector index

  • Orphan manifest row

    The manifest lists a document that exists nowhere else.

    Manifest

Scenarios

13 real situations RAGGuard catches

Each one looks fine from the outside: jobs succeed and dashboards stay green. Pick a group to see what your users would notice, what RAGGuard shows and how it gets fixed.

The document exists, but your users cannot find it, or cannot find all of it.

  • A document nobody ever processed

    The file is in your bucket, but the pipeline never picked it up: no manifest row and nothing in the index.

    What your users notice
    Nobody can find it, and no error is ever shown.
    What RAGGuard shows
    HighMissing embeddings
    How it gets fixed
    Re-run it through your pipeline. A repair plan sends it to your worker.
  • The pipeline said success. The index is empty.

    The manifest records success and a chunk count, but the index holds nothing for the document.

    What your users notice
    Dashboards stay green while the document is invisible. This is the most dangerous case.
    What RAGGuard shows
    CriticalMissing embeddings
    How it gets fixed
    Re-index it, then find out why the pipeline reports success before the vectors are written.
  • Half indexed: the worker died part way

    Only some of the chunks reached the index, while the manifest claims all of them.

    What your users notice
    Answers about the missing parts are wrong or empty.
    What RAGGuard shows
    CriticalPartial index
    How it gets fixed
    Re-index only the missing chunks.
  • Vectors that can never match

    Some points exist, but their vector is empty or all zeros.

    What your users notice
    Parts of the document can never be found.
    What RAGGuard shows
    HighEmpty embeddings
    How it gets fixed
    Re-index the document.

The ratings shown are the usual ones. RAGGuard sets severity from the evidence it reads: for example, a gap the pipeline reported as a success is rated higher, and a deleted document that a test search can really return is rated critical.

Use cases

Where RAGGuard earns its place

Run it once to see where you stand, then keep it running so the next silent failure is caught the day it happens.

  • After every ingestion run

    Run an audit from CI or a scheduler with an API key, and get a Slack-compatible alert when the index has drifted.

    For Platform and data engineers

  • Before you change the embedding model or parser

    See which documents the change would touch and what re-embedding would cost, then check afterwards that nothing was left on the old version.

    For AI and platform teams

  • When content is deleted or withdrawn

    Check that a removed document is really gone from search, not only from storage.

    For Content owners and compliance teams

  • When you serve more than one customer

    Check that no customer's content carries another customer's label, and see the result for each customer on the Isolation page.

    For Security and SaaS teams

  • When a user says the answer is wrong

    Look up the document, see whether the source, the manifest and the index agree, and inspect the live pieces to find out why.

    For Support and AI product teams

  • When someone asks for proof

    Share a health score, the history of every audit and exported results, so you can show what was checked and when.

    For Product owners and leadership

What you get

Everything you need to act on it

From the first audit to a verified fix, in one place.

  • A health score you can trust

    The share of documents whose index matches the latest source exactly, tracked after every audit.

  • Evidence, not guesses

    Each finding shows expected versus actual, the values it read, and whether the bad content can really be retrieved.

  • Know the cost before you fix

    Estimated re-embedding work and cost, compared with rebuilding everything, before you commit to a repair.

  • Blast radius

    Which documents, tenants and queries a pipeline change or an ingestion failure would touch.

  • Safe, verified repairs, in bulk

    One plan can cover every broken document. Send it to your own worker as one signed request, or email it as a readable summary with a spreadsheet. RAGGuard then checks the real index, and nothing is removed without your approval.

  • Alerts and automation

    Slack-compatible webhook alerts, and API keys to run audits from CI or a scheduler.

Security

Safe to point at production

You are giving a verifier access to the systems behind your product. It is built to ask for as little as it can.

  • Read-only by design

    RAGGuard lists, reads and queries. It never writes to your S3 bucket, your manifest or your index.

  • Credentials stay secret

    Encrypted before they are stored, used only to read your systems, and never shown again after you save them.

  • Workspaces are isolated

    Every query is scoped to your workspace, so one account can never see another's data.

  • Private networks blocked

    The hosted service only connects to public addresses, so it cannot be pointed at internal services.

  • Signed repairs, approved removals

    Repair requests are signed with HMAC-SHA256, and vectors are removed only after you approve.

  • Hardened sessions

    HttpOnly cookies, rate-limited sign-in, scoped API keys and a strict content security policy.

Works with

The tools you already run

No agents to install and no pipeline to change. Two connections are enough to start: your documents and your index.

  • Source documents

    Amazon S3

    And S3-compatible stores such as MinIO, Cloudflare R2 and Supabase Storage.

  • Vector index

    Qdrant or pgvector

    Qdrant, self-hosted or Cloud. Or a pgvector table in any PostgreSQL: Supabase, Neon or AWS RDS, including LangChain and LlamaIndex tables. Your own field and column names.

  • Processing manifest (optional)

    PostgreSQL

    Not needed to start. If your pipeline records what it indexed, connect that table, with your own column names, for extra checks.

FAQ

Questions, answered

The usual worries, answered plainly.

Does RAGGuard change my data?

No. It only lists, reads and queries your S3 bucket, your vector index and, if you connect one, your manifest table. Fixes are carried out by your own worker, which RAGGuard asks with a signed request. RAGGuard then checks the real index again.

What do I need to connect?

Two things: an S3 bucket with your source documents, and your vector index, either a Qdrant collection or a pgvector table in PostgreSQL. A manifest table, where your ingestion pipeline records what it indexed, is optional and adds extra checks. The Data Sources page tests each connection and tells you exactly what to change.

How does it know what my index should contain?

It compares the content hash of each file in your source with the hash stored on its vectors. If you connect a manifest, it also compares the chunk counts your pipeline recorded with what is actually indexed. It can also run a search to check whether bad content can really be returned.

How much does it cost?

Nothing while RAGGuard is in beta. Paid plans will come later.

What is a free audit?

Email us at kishore.knaresh07@gmail.com and we will set a time. Together we connect RAGGuard to your systems, read-only, run your first audit and go through what it found. You keep the results.

Can I run audits from CI or on a schedule?

Yes. Create an API key and call the audit endpoint from GitHub Actions, cron or any scheduler. A key can run audits and read results, but it cannot read your settings or create other keys.

How are my credentials protected?

They are encrypted before they are stored, used only from the server to read your systems, and never shown again once saved. The hosted service refuses private and localhost addresses.

See what your index is really serving.

Connect your documents and your index, and run your first audit in minutes. Free during beta.

Questions first? Read the FAQ or email kishore.knaresh07@gmail.com.