Assessor_2.0/docs/REGULATION_MAPPING.md

Regulation mapping — quick command reference

The regulation retrieval feature maps a finding definition to the best-matching regulation units (e.g. ISO 27002 controls) using a Qdrant vector database and an OpenAI-compatible embedding endpoint. It is disabled unless configured and is backend-only.

This is the "ingestion + retrieval" slice (Phase 13D). LLM selection, review workflow and UI are not part of it yet. See docs/IMPLEMENTATION_PLAN.md (Phase 13D) and SECURITY.md (Regulierungssuche-Grenze).

Prerequisites

• Docker (Qdrant runs in a container).
• An OpenAI-compatible embedding endpoint serving bge-m3 (multilingual, 1024 dimensions, no prefixes). Either:
  • llama.cpp: download bge-m3-FP16.gguf (or bge-m3-Q8_0.gguf) from gpustack/bge-m3-GGUF, then llama-server -m bge-m3-FP16.gguf --embedding --pooling cls -c 8192 -ub 8192 --port 8080 → base URL http://127.0.0.1:8080/v1.
  • Ollama: ollama pull bge-m3 → base URL http://127.0.0.1:11434/v1.
• The catalog file iso27002-controls-embed.jsonl — it contains licensed text and must stay outside the repository (it lives under Mapping-test/, which is git-ignored).

One-time setup

# 1. Start Qdrant (reachable at http://127.0.0.1:6333)
npm run qdrant:up

# 2. Start the embedding model (separate terminal / service)
#    llama.cpp:
llama-server -m bge-m3-FP16.gguf --embedding --pooling cls -c 8192 -ub 8192 --port 8080
#    or Ollama:  ollama pull bge-m3 && ollama serve   (port 11434)

Add to .env.postgres (create it from .env.postgres.example if needed):

ASSESSOR_QDRANT_URL=http://127.0.0.1:6333
ASSESSOR_QDRANT_COLLECTION=regulation_catalog_v1
ASSESSOR_EMBEDDING_BASE_URL=http://127.0.0.1:8080/v1
# ^ llama.cpp on :8080; for Ollama use http://127.0.0.1:11434/v1
ASSESSOR_EMBEDDING_MODEL=bge-m3
ASSESSOR_EMBEDDING_DIMENSIONS=1024
ASSESSOR_REGMAP_CATALOG_VERSION=DIN EN ISO/IEC 27002:2024-01
# For multilingual-e5-* instead of bge-m3, the prefixes must end with a space:
# ASSESSOR_EMBEDDING_QUERY_PREFIX=query: 
# ASSESSOR_EMBEDDING_PASSAGE_PREFIX=passage: 

# 3. Embed the catalog into Qdrant (idempotent — safe to re-run)
npm run regmap:ingest "Mapping-test/test/iso27002-controls-embed.jsonl"

Run the app

# Development (client on :5173, backend API on :4173, reads .env.postgres)
npm run dev

# Production-like (build, then serve everything on :4173)
npm run serve:local

Verify

# Capability should report "ready" once Qdrant + embeddings + collection are set
curl http://127.0.0.1:4173/api/v1/regulation-mapping/capability

# Health also carries the capability
curl http://127.0.0.1:4173/api/v1/health

# Retrieve candidates for a finding (POST needs the same-origin mutation header)
curl -X POST http://127.0.0.1:4173/api/v1/regulation-mapping/retrieve -H "Content-Type: application/json" -H "X-Assessor-Request: 1" -d '{"definition":{"title":"RC4 encryption enabled on Domain Controller","description":"Weak cipher permitted on DC"}}'

Capability states: disabled (no embedding URL) · misconfigured (collection missing, dimension/model mismatch, or endpoint unreachable) · ready.

Command reference

Command	What it does
npm run qdrant:up	Start the Qdrant container (detached).
npm run qdrant:down	Stop and remove the Qdrant container.
npm run qdrant:logs	Follow Qdrant logs.
npm run regmap:ingest <file.jsonl>	Embed the catalog and upsert into Qdrant.
npm run dev	Dev stack: client :5173, backend :4173 (loads .env.postgres).
npm run serve:local	Build and serve the whole app on :4173.
npm run start:local	Serve an already-built app on :4173.
npm run db:up / db:down	Start / stop the local PostgreSQL container.
npm test / npm run build	Test suite / production build.

Notes

• Qdrant's data persists in the assessor-qdrant-data Docker volume across qdrant:down / qdrant:up. To wipe it: docker volume rm assessor_assessor-qdrant-data.
• ASSESSOR_EMBEDDING_DIMENSIONS must match the embedding model's output size, or the collection validation fails as misconfigured. Changing models means re-ingesting (delete the collection or use a new ASSESSOR_QDRANT_COLLECTION).
• On this machine, the dev Qdrant compose publishes port 6333 on the WSL distro interface so Windows 127.0.0.1:6333 reaches it (WSL2 does not forward a 127.0.0.1-bound container port). It stays isolated from the LAN in WSL2 NAT mode.