# What belongs in a private AI knowledge base (and what should stay out)

> Decide what goes into a private AI knowledge base: approved procedures, runbooks and product docs in; PII, HR files, drafts and secrets out. A CTO checklist.

- URL: https://www.rebels.software/guide/what-belongs-in-private-ai-knowledge-base
- Published: 2026-11-06
- Author: Bart Szczepański
- Tags: Private AI, RAG, Knowledge base

A private AI knowledge base should hold the documents people already trust when they answer the same questions by hand: current procedures, product manuals, runbooks and closed tickets with a clear outcome. Keep out anything that creates a leak or a wrong answer if the model quotes it - raw personal data, HR and payroll files, unsigned drafts, secrets and folders nobody owns. Scope the first index around one team and one kind of question, then grow it only after search is reliable.

This guide is about the decision of what goes in. The companion guide on [how to prepare company documents for private AI search](/guide/prepare-documents-for-private-ai-search) covers cleaning, OCR, metadata and permissions once that list exists.

## Why the boundary matters more than the model

Private AI search, usually built as RAG, answers from whatever you put in the index. If the index holds salary sheets next to service manuals, a clever prompt or a loose permission can surface the wrong file. OWASP lists [sensitive information disclosure](https://genai.owasp.org/llmrisk/llm022025-sensitive-information-disclosure/) and [vector and embedding weaknesses](https://genai.owasp.org/llmrisk/llm082025-vector-and-embedding-weaknesses/) among the top risks for LLM apps, including leaks between groups that share one index.

The GDPR [data minimisation principle](https://eur-lex.europa.eu/eli/reg/2016/679/oj) (Article 5(1)(c)) points the same way: process only what is necessary for the purpose. A smaller, intentional corpus is easier to permission, easier to test and less likely to invent answers from stale drafts.

## What usually belongs

Use this as a starting checklist for a first knowledge base aimed at internal Q&A or a field team. Adjust to your industry, but do not skip the "stay out" list below.

1. **Approved, current procedures and SOPs.** The version that is in force today, with an owner and a status field. Older revisions stay in the archive, not in the index.
2. **Product and service documentation.** Manuals, datasheets, warranty rules and installation guides for the lines people ask about every week.
3. **Operational runbooks.** How to restart a line, handle a known fault or escalate an incident. Prefer short, dated runbooks over long slide decks.
4. **Closed tickets and known fixes.** Tickets with a confirmed resolution from a recent window (for example the last 12-24 months), tagged by product or site. Open or half-written tickets confuse the model.
5. **Internal FAQs and decision notes that a named owner still stands behind.** Meeting minutes only when they record a decision, not the whole discussion.
6. **Contracts and commercial templates you deliberately expose to a narrow group.** Master service agreements, standard terms or price lists that sales or legal may search - never the whole contract share by default.

## What should stay out (or sit in a separate, tightly gated index)

1. **Direct personal data you do not need for the use case.** National ID numbers, private addresses, health data, children's data. If a document mixes procedure text with personal data, redact or split it before indexing.
2. **HR and payroll.** Employment contracts, performance reviews, salary sheets, recruitment notes. These almost never belong in a general knowledge base.
3. **Secrets and credentials.** API keys, passwords, VPN configs, private keys, unreleased vulnerability notes. A search hit that quotes a secret is an incident.
4. **Unsigned drafts and "final_v3_NEW" files.** Until someone marks a document as approved, it should not answer customer or technician questions.
5. **Personal mailboxes and chat dumps.** They are full of side context, opinions and third-party data you did not mean to publish inside the company.
6. **Legal holds, disputes and M&A materials.** Keep them in the systems built for them. A helpful answer that cites a dispute memo is still a leak.
7. **Anything your source permissions already overshare.** If a folder is open to "Everyone", fix that at the source before the index copies the mistake. Microsoft, for example, states that Copilot ["only surfaces organizational data to which individual users have at least view permissions"](https://learn.microsoft.com/en-us/copilot/microsoft-365/microsoft-365-copilot-privacy) and asks customers to get permission models right - a private system you build yourself has the same constraint.

## Step by step: draw the boundary for one team

1. **Write the use case in one sentence.** Example: "Service technicians ask how to diagnose fault codes on product family X." Everything else waits.
2. **List candidate sources.** Network shares, SharePoint, the ERP, the ticket system, shared mailboxes. Name an owner for each.
3. **Mark each source include, exclude or later.** Use the two lists above. When unsure, choose exclude for the first round.
4. **For every include, write who may see it.** Carry those groups into the index and filter before the model runs. One service account that reads everything is a common failure mode.
5. **Define "current".** A date cut-off, a status field or both. Without it, old procedures win on keyword overlap.
6. **Hand the include list to the preparation pipeline.** Deduplicate, OCR scans, add metadata and test on real questions - the steps in [preparing documents for private AI search](/guide/prepare-documents-for-private-ai-search).
7. **Review the boundary when you add a second team.** New sources get the same include/exclude pass. Do not paste another file server into the live index.

## Common mistakes

- **Indexing the whole file server "to be safe".** You import every overshared folder and every draft. Answers become hard to trust.
- **Treating "we have a VPN" as a content policy.** Network location is not a substitute for deciding what the model may quote.
- **Putting contracts in for everyone because legal might need them someday.** Give legal a separate corpus with their groups, or leave contracts out of the first index.
- **Leaving PII inside otherwise useful PDFs.** Redact or split. Minimisation is cheaper than explaining a leak.
- **No owner for the include list.** When a document is withdrawn, nobody removes it from the index.

## Next step

Pick one team and one recurring question. On one page, list the sources that answer it, mark each include or exclude, and write the access groups next to every include. That page is the scope of your first private AI knowledge base.

Rebels Software builds private AI on infrastructure you control, as part of the platforms and applications we deliver. In a Discovery workshop we map sources, permissions and the include/exclude boundary with you, then prove one case as a PoC on your real documents inside your network. After deployment, Rebels Care keeps the corpus and the index aligned with what your teams actually use. If that matches where you are, book a call on this page or start from the [Rebels Software homepage](/).

We start with your biggest problem and the data behind it. [Book a free call](https://calendly.com/bartlomiej-jacyno/intro)
