ippadmin/itpp-infrastructure
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

rebrand Scirium docs (Wall-O -> Scirium) + add v1/v2 documentation package

Browse Source
This commit is contained in:
root
2026-08-16 09:24:58 -04:00
parent 0424b2954a
commit 64cf1043e7
11 changed files with 453 additions and 82 deletions
Download Patch File Download Diff File Expand all files Collapse all files
projects/scirium/docs/01-overview.md
+75
View File
@@ -0,0 +1,75 @@
# Scirium: Product Overview
Status: PLANNED
Audience: everyone (technical and business). Read this first.
## 1. One-liner
Ask your company anything. Get a cited answer from your own documents, scoped to the exact department you are talking to.
## 2. What Scirium Is
Scirium is a self-hosted, white-label, multi-tenant knowledge assistant for internal staff. A company points Scirium at its own document libraries (SharePoint, OneDrive, employee handbooks, policies, procedures). Staff ask questions in a chat window. Scirium answers using only that company's documents, and every answer carries inline citations pointing to the source.
Scirium is pronounced "sigh-ree-um", from the Latin "scire" (to know) plus the -ium element suffix.
## 3. Who It Is For
| Audience | What they need from this product |
|---|---|
| Staff (end users) | Correct, cited answers to policy, procedure, and how-to questions without interrupting a human. |
| Practice / business admins | A way to turn existing documents into a scoped, auditable Q&A assistant per department. |
| MSP / reseller partners | A white-label product they sell under their own brand on infrastructure they control. |
| IT Pro Partner (operator) | A multi-tenant service that runs on existing netcup infrastructure with high gross margin. |
## 4. The Core Invariant
One channel equals one knowledge domain plus one agent plus one scoped source. This is a hard rule, enforced in the data model, not a policy.
| Bound item | Meaning |
|---|---|
| One knowledge domain | Employee Resources, Billing, IT Help, or similar. |
| One agent | A domain-tuned persona that answers only within that domain. |
| One scoped source | A set of M365 document libraries mapped to one vector namespace. |
A channel cannot span two domains, and an agent cannot serve two channels. If a business needs a second domain, it creates a second channel, a second agent, and a second scope.
## 5. Why This Is Not Just Another Chatbot
A regular chatbot is one undifferentiated model with no grounding in your documents, no audit trail, and no scoping. Scirium is the opposite on every axis a business cares about.
| Axis | Regular chat | Scirium |
|---|---|---|
| Grounding | May hallucinate, no source | Every answer cites its source documents inline with a Sources footer |
| Scoping | One model answers everything | One channel equals one domain plus one agent plus one scoped source. Billing answers billing, HR answers HR. |
| Data residency | Sent to a vendor's cloud | Self-hosted, white-label, data stays in the company's M365 and your infra |
| Audit trail | None | Every question and answer logged with citations, model, tokens, latency |
| Identity | Anonymous | Each domain is a named agent persona (for example "Scirium Billing") |
| Failure mode | Confident wrong answers | If the answer is not in scope, it says so instead of guessing |
| Tenancy | N/A | One deployment serves many isolated tenants |
The difference is trust. A regular chatbot is a tool. Scirium is a scoped, grounded, audited answer layer over documents the company already owns.
## 6. Architecture at a Glance
| Layer | Responsibility |
|---|---|
| Rocket.Chat | Chat transport only. One workspace per tenant. Zero intelligence. |
| FastAPI orchestrator | All intelligence. Tenancy, agents, scopes, M365 connector, retrieval, LLM. |
| Postgres + pgvector | State and vectors. Tenants, channels, agents, scopes, documents, chunks, messages. |
| admin-ai | LLM. DeepSeek V4 Pro primary with a configured fallback chain. |
| Wasabi S3 | Object storage. Sync staging, backups, agent assets, audit exports. |
This split is fixed. Rocket.Chat never holds intelligence; the orchestrator never renders chat UI.
## 7. Hard Scope
Internal staff documentation only. No patient records, no PHI, no HIPAA scope, no medical or vision AI. This boundary is permanent and is not a roadmap item.
## 8. Related Documents
- 02-v1-scope.md: exactly what v1 ships.
- 03-v2-scope.md: what v2 adds.
- 04-admin-guide.md: provisioning and go-live.
- 05-user-guide.md: how staff use it.
- 06-roadmap.md: v1 through v4 and the autonomy curve.
projects/scirium/docs/02-v1-scope.md
+67
View File
@@ -0,0 +1,67 @@
# Scirium v1 Scope
Status: PLANNED
Applies to: v1 (grounded cited Q&A only)
## 1. What v1 Ships
v1 is a read-only, grounded, cited question answering assistant. A staff member asks a question in chat. Scirium retrieves the relevant passages from that channel's scoped M365 document libraries and returns an answer with inline citations to the source documents.
v1 ships exactly one capability: grounded, cited Q&A. Nothing else.
| Capability | In v1? | Notes |
|---|---|---|
| Grounded, cited Q&A | Yes | The core and only capability. |
| Read-only retrieval over scoped M365 libraries | Yes | SharePoint and OneDrive libraries named in the channel's scope. |
| Autonomous actions (send, write, create, update) | No | Zero. No writes to M365, no message posting beyond the answer, no integrations. |
| Report generation, content drafts, workflows | No | Deferred to v2 and later. |
| Patient records or PHI | No | Permanently out of scope, every version. |
## 2. The v1 Contract
- Every answer cites its sources inline and in a Sources footer.
- If the answer is not in the knowledge base, Scirium says so instead of guessing.
- Scirium never writes to the source systems. The M365 connector holds read permission only.
- Every question and answer is logged for audit.
## 3. The 6-Step Message Flow
1. Staff post a question, either by @mentioning the agent in a channel or by DMing the bot.
2. Rocket.Chat fires an outgoing webhook to the orchestrator (channel path) or the bot listener delivers a DM event (DM path).
3. The orchestrator authenticates the request, then resolves tenant, channel, agent, and scope from the room and webhook path.
4. Retrieval: a pgvector semantic search over the channel's vector namespace returns candidate chunks, reranked to the top set. A below-threshold result falls back to Microsoft Graph search over the scope's libraries.
5. The orchestrator builds a prompt from the agent persona, the retrieved context with source markers, and the citation instruction, then calls admin-ai (DeepSeek V4 Pro primary with fallback).
6. The answer is posted back to Rocket.Chat as the agent bot, with inline citations and a Sources footer.
## 4. Retrieval Details
| Step | Behavior |
|---|---|
| Primary | pgvector cosine search filtered by tenant_id and kb_scope_id in the same query. |
| Candidate set | Top 20 by score. |
| Rerank | Top 5 passed to the prompt. |
| Fallback | Below threshold or empty: Microsoft Graph /search/query over the scope's document_libraries. |
| Isolation | tenant_id and kb_scope_id filters are mandatory, so a chunk can never leak across channels or tenants. |
## 5. Error Handling
| Condition | v1 behavior |
|---|---|
| Retrieval empty or below threshold | Reply "I could not find an answer in the knowledge base for this question", no sources, never guess. |
| LLM timeout or error | Retry once on the fallback model, then a canned "could not reach the model" reply. |
| Channel has no agent or no scope | Configuration error reply (or silent no-op, per tenant config). |
| Webhook retry (duplicate) | Idempotency key deduplicates; the duplicate returns 200 and is dropped. |
| Post failure | Retry with backoff, then mark failed and surface a tenant alert. |
## 6. Explicit v1 Exclusions
v1 does none of the following, and this is intentional:
- No actions. v1 cannot send email, create calendar items, update records, or write to any system.
- No writes to M365. The connector is read-only.
- No patient records or PHI, ever.
- No multi-domain answers. A channel answers only its own scope.
- No proactive messages. The bot replies only to a question addressed to it.
- No role escalation. An answer cannot grant itself new capabilities.
Everything on this list is either permanently excluded (PHI) or deferred to a later version behind the propose-approve-audit gate.
projects/scirium/docs/03-v2-scope.md
+62
View File
@@ -0,0 +1,62 @@
# Scirium v2 Scope
Status: PLANNED
Applies to: v2 (risk-tiered actions on top of v1)
## 1. What v2 Adds
v2 keeps everything in v1 (grounded, cited, read-only Q&A) and adds a single new capability class: risk-tiered actions. v2 never removes the v1 grounding rails.
## 2. Risk-Tiered Actions
Actions are grouped by risk and ship in that order. Reporting first, low-risk writes second, and never anything above the approved tier.
| Tier | Examples | Ships | Autonomy |
|---|---|---|---|
| 0 | Q&A only (v1) | v1 | None |
| 1 | Reporting: summaries, compliance checklists, document search reports | v2 first | None beyond read |
| 2 | Low-risk writes: draft a policy section, draft an email reply, update an internal FAQ | v2 second | Propose, then approve, then narrow audited write |
| 3 | Higher-risk writes and integrations | v3 | Still gated, broader by tier |
## 3. The Propose-Approve-Audit Gate
Every action in v2 and later passes through three steps. This gate is permanent.
1. Propose. The agent drafts the action and shows exactly what it would do: the target system, the record, and the exact change.
2. Approve. A human with the right role approves or rejects. No action executes without an explicit approval.
3. Audit. The executed action is written narrowly (only the approved change), and the full record is logged: who proposed, who approved, what changed, when.
| Gate step | What it enforces |
|---|---|
| Propose | The agent never acts silently; it always shows the plan first. |
| Approve | A human is in the loop for every action, every time. |
| Audit | Every executed action is logged and replayable from the audit trail. |
## 4. Reporting First, Writes Second
v2 ships reporting before writes because reporting is read-only and low risk. Reporting proves the pipeline end to end (propose, approve, audit) before the product is allowed to touch any system.
Low-risk write examples (tier 2, all gated):
- Draft a section of an employee handbook for an admin to review.
- Draft a reply to a customer email that a human then sends.
- Update an internal FAQ entry with an approved wording change.
Each example follows propose, approve, narrow write, audit. None of them are autonomous.
## 5. Permanent Autonomy Guardrails
These hold for v2 and every later version:
- The propose-approve-audit gate is never removed or bypassed.
- Writes are always narrow: only the approved field or record, nothing else.
- Risk tier is a hard ceiling. A channel cannot act above its assigned tier.
- Patient records and PHI remain permanently out of scope.
- Every action is logged in the messages and audit tables with the approver identity.
## 6. What v2 Does Not Do
- No free-roaming agents that act across channels or systems.
- No writes without a prior human approval.
- No patient or PHI content, regardless of tier.
- No removal of the v1 grounding and citation rails.
projects/scirium/docs/04-admin-guide.md
+76
View File
@@ -0,0 +1,76 @@
# Scirium Admin Guide
Status: PLANNED
Audience: tenant admin and Scirium operator
Purpose: provision a tenant and take a channel live end to end.
## 1. Prerequisites
- A Microsoft 365 tenant with the Scirium Entra app registered and holding the Sites.Selected application permission.
- A Rocket.Chat workspace provisioned for the tenant (one workspace per tenant).
- Vaultwarden access for secrets (M365 credentials, bot tokens, webhook secret).
- Admin role on the orchestrator.
## 2. Provision a Tenant
1. Create the tenants row: slug, name, branding (logo_url, accent_color), M365 tenant id and client id.
2. Store the M365 client secret in Vaultwarden; put only the reference in m365_credential_ref.
3. Record the Rocket.Chat workspace root URL in rocket_chat_url.
4. Generate a webhook signing secret, store it in Vaultwarden, and set webhook_secret_ref.
5. Set status to provisioning until the workspace and connector are confirmed.
The tenant is now the root of the tenancy tree. Every downstream object hangs off it.
## 3. Create a Channel
1. Insert a channels row: name (the knowledge domain, for example "Billing"), slug (unique per tenant), description.
2. Optionally create the matching Rocket.Chat room and store rocket_chat_room_id and rocket_chat_room_name.
3. Leave status = draft. The channel stays draft until the scope is bound and indexed.
## 4. Attach an Agent
1. Insert an agents row: name (for example "Scirium Billing"), system_prompt (the domain persona), model_name (deepseek-v4-pro) and fallback_model, rocket_chat_bot_username.
2. Provision the Rocket.Chat bot user (POST /api/v1/users.create with roles bot), create a personal access token, and store the token reference and bot user id.
3. Invite the bot to the room (POST /api/v1/channels.invite).
4. Set channels.agent_id to the new agent.
## 5. Bind a Scope
1. Insert a kb_scopes row: vector_namespace (tenant_{tenant_id}__scope_{kb_scope_id}), document_libraries (the allowlisted M365 site and drive ids), embedding_model and embedding_dim, chunk_size (512) and chunk_overlap (64).
2. Set channels.kb_scope_id to the new scope.
The scope is the allowlist. Only the libraries named here are ever indexed or searched for this channel.
## 6. Run the Initial Sync
1. Trigger the M365 connector for the scope. It pulls the named document libraries, extracts text, chunks, embeds, and writes documents and chunks rows under the namespace.
2. Monitor document status: pending to extracting to indexing to indexed.
3. Confirm nonzero indexed document count and nonzero chunk count in the scope's namespace.
4. The channel stays in draft until this succeeds.
This is the only long-running step. Everything before it is fast and idempotent.
## 7. Go Live
1. Set channels.status = active.
2. Configure the outgoing webhook on the channel with the trigger word set to the bot username, pointing at the orchestrator webhook URL.
3. Optionally register a slash command such as /ask.
4. Run an end-to-end smoke test: ask a question in the channel, confirm the agent bot posts a cited answer, and confirm the answer's sources resolve to documents inside the scope.
5. Verify a below-threshold question returns "I could not find an answer in the knowledge base for this question" instead of guessing.
## 8. Operate
| Task | Where |
|---|---|
| Delta sync schedule | kb_scopes.sync_policy |
| Document failure triage | documents.status and documents.error |
| Audit review | messages table (question, answer, citations, model, latency) |
| Secret rotation | Vaultwarden; update only the reference in Postgres |
| Suspend a tenant | tenants.status = suspended |
## 9. Rules
- Never inline a secret in Postgres or config. References only.
- Never index a library outside the scope allowlist.
- Never go live on an empty scope. Indexed document count must be nonzero.
- Never grant the connector write permission. Read only.
projects/scirium/docs/05-user-guide.md
+53
View File
@@ -0,0 +1,53 @@
# Scirium User Guide
Status: PLANNED
Audience: staff (end users)
## 1. How to Ask
You can reach an agent two ways:
| Mode | Use it for | How |
|---|---|---|
| Channel @mention | Questions the whole team benefits from; answers stay searchable in the room | Type @ followed by the agent name, then your question. |
| DM the bot | Private follow-up, clarification, or a 1:1 thread | Open a direct message with the agent bot and type your question. |
Both paths produce the same cited answer format. Only the destination differs.
## 2. What a Good Question Looks Like
Ask a complete question about a specific topic in the channel's domain. For example, in the Billing channel, ask "What is our refund policy on a disputed charge?" rather than "refund?".
## 3. How Citations Work
Every answer cites its sources in two places:
1. Inline markers, such as [1], [2], next to the sentences they support.
2. A Sources footer at the end of the answer listing each cited document title and a link.
Click the source link to open the document and verify the answer yourself. This is the point of the product: you can always check the source.
## 4. When the Answer Says It Is Not in the Knowledge Base
If the answer says "I could not find an answer in the knowledge base for this question", it means one of:
| Reason | What to do |
|---|---|
| The document is not in this channel's scope | Ask in the right channel, or ask an admin to add the document to the scope. |
| The question is outside this channel's domain | Ask the matching channel (for example HR questions go to the Employee Resources channel). |
| The answer genuinely is not documented | Ask a human, or ask an admin to add the missing document. |
Do not rephrase and expect a guess. Scirium is designed to say it does not know rather than invent an answer.
## 5. What Scirium Cannot Do
- It cannot answer from documents outside the channel's scope.
- It cannot answer questions about patient records or personal health information.
- It cannot take actions on your behalf (v1). It answers questions only.
- It cannot invent an answer when the source is missing.
## 6. Tips
- Keep a question to one topic for the clearest citations.
- If the answer is close but not quite right, DM the bot a follow-up with more detail.
- If a source link does not open, report it to an admin; the document may have moved or been removed from the scope.
projects/scirium/docs/06-roadmap.md
+38
View File
@@ -0,0 +1,38 @@
# Scirium Roadmap
Status: PLANNED
Audience: technical and business
## 1. Version Curve
| Version | Capability | Autonomy |
|---|---|---|
| v1 | Grounded, cited Q&A. Read-only retrieval over scoped M365 libraries. No actions. | Zero writes, answer only |
| v2 | Risk-tiered actions. Reporting first, then low-risk writes behind propose-approve-audit. | Conditional, human approved |
| v3 | Broader autonomous actions by risk tier, deeper connectors, granular per-channel permissions. | Tiered autonomy |
| v4 | Multi-connector breadth, analytics, admin self-service at scale. | Mature, governed |
## 2. What Changes Between Versions
Later versions add autonomy, not accuracy. v1 is already fully grounded and cited. v2 through v4 add the ability to act, and only within an approved risk tier.
| Version | Adds | Does not change |
|---|---|---|
| v1 | Cited Q&A | Grounding, scoping, isolation |
| v2 | Reporting and low-risk writes | Grounding rails, approval gate |
| v3 | More connectors and tiers | One channel one domain invariant |
| v4 | Analytics and self-service | Hard scope (no PHI) |
## 3. The Permanent Gate
The propose-approve-audit gate is permanent across all versions. Later versions add autonomy but never remove or bypass this gate.
1. Propose: the agent shows exactly what it would do.
2. Approve: a human approves or rejects. Nothing executes without approval.
3. Audit: the narrow, approved write executes and is logged.
No version of Scirium becomes a free-roaming agent. Autonomy expands by risk tier, and the gate stays.
## 4. Hard Scope, All Versions
Internal staff documentation only. No patient records, no PHI, no HIPAA scope, no medical or vision AI. This boundary is not on the roadmap; it is permanent.