diff --git a/AGENTS.md b/AGENTS.md index d1eb2fb..3480bba 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -52,7 +52,7 @@ ## Core Engine and engines - Explicit user decision 2026-10-01: ASSESSOR with all its existing functions is the **Core Engine**. The context enrichment (Phase 13E, "context engine"/"Kontext-Engine" in earlier text) is the **Context Engine**. The read-only Report Engine (Phase 7, `src/reporting/`, `server/reporting/`) is an engine in the same sense. Future features arrive as further engines that provide a capability without changing the Core. The UI labels `Context`/`Kontext` stay unchanged. -- Explicit user decision 2026-10-05: the regulatory control mapping (Phase 13F, ISO mapping) is the **Compliance Engine** (`src/compliance/`, `server/compliance/`, `tests/compliance/`). It is framework-neutral by intent: ISO/IEC 27001/27002 is its first corpus, later frameworks such as BSI or DORA arrive inside this engine without a Core change. It reads the Core only through `ComplianceReadPort` (single Result DB row by project, filtered Result DB pages, project, assessment and scan-run lists; explicit user decision 2026-10-05: additionally the internal-analysis `policy-evaluation` rows of the same project and timeslot, used solely to read the GPO technical setting ID that the Result DB withholds, so a GPO setting counts once like its rolled-up Work record and the Work ISO facets match it; that ID is never displayed or exported and no other internal field is read) and stores mapping runs, accept/reject decisions, batches and AI overview runs only through its own `ComplianceStore`. Explicit user decision 2026-10-05: all AI integration belongs to this engine. The AI overview (service, model, Client Info panel) moved from the Core into it, and the Result DB `Export JSON` is the engine's 1:1 copy of what its AI features receive: the exact AI overview input built by the same function the overview sends to the model, and every distinct ISO mapping query with the Result DB rows that produce it (the retrieved candidate controls depend on the index and are not exported). The shared model client `server/ai/litellm-client.ts` stays Core platform code. The existing tables of migrations 025 (AI overview runs) and 026 keep their names and data and are owned by the engine; its writes never advance the Core repository revision. Because the Core deletes scan runs and assessments without knowing the engine, the engine removes AI overview runs of a deleted timeslot from its own table on start and on read. The Core project export receives the engine data, including succeeded AI overview runs, beside the Core export at the HTTP route. Its extension points are the Finding Reader `Map ISO controls` action and its window, the Finding Reader `ISO controls` section, the Result DB `Map all ISO controls` batch action and `Export JSON` action, the Data Hub ISO corpus tile, the Client Info `Compliance` row (formerly the `ISO controls` section) and `AI Overview` tab, and the Work `ISO control` and `ISO theme` facets. Switched off, the engine also removes the AI overview tab and its requests and the `Export JSON` action. Explicit user decision 2026-10-05: `Map ISO controls` opens a window like `Accepted Risk` with Accept and Reject per control (no third decision value). The Finding Reader lists the controls whose latest decision is accepted in an `ISO controls` section read like `Description`, after the summary fields. Client Info shows the accepted controls of the selected timeslot directly below the Context heatmap in the same tile layout without severity colour, each with its ISO 27002 theme and the number of deviation findings (a GPO setting counts once); a tile opens Work with the `Deviation` status and that control selected in the `ISO control` facet, like a Context heatmap cell. Explicit user decision 2026-10-06 supersedes only the tile layout of that Client Info section, after feedback that it grows too large: the accepted controls are grouped by ISO 27002 theme in ISO order (Organizational, People, Physical, Technological; a theme without hits is omitted). Each full-width theme heading states its number of controls and of distinct findings (a finding counts once per theme) and opens Work with the `Deviation` status and that theme selected in the `ISO theme` facet. Below it, its controls flow in columns of twice the compact tile width, one compact row per control sorted by finding count, showing the Annex A ID, the complete title (wrapped, never truncated) and the authoritative count, without an amount bar (removed the same day because it cost title width); a row opens Work exactly like the former tile. A theme shows its top six controls; a keyboard-accessible `Show all N controls` button below expands that theme and `Show top 6` collapses it again, and every theme starts collapsed for a new timeslot. The theme heading still opens all of its controls in Work. Explicit user decision 2026-10-06, later the same day, supersedes the in-page list and its top-six cap: Client Info shows a `Compliance` row below the Context heatmap with one tile per framework, in the exact environment tile geometry and fact grid (its own `client-compliance-grid` with the same tile rules; the environment row keeps exactly its six tiles), so later frameworks such as BSI or DORA add a tile instead of a section. The `ISO 27001/27002` tile shows `Controls` (accepted controls), `Findings` (distinct findings) and the two themes with the most findings, each with its finding count (fewer when fewer themes have hits; an em dash while loading, never zero). Its `Results` button opens the shared Results window with the complete grouped list described above, without a cap; a theme heading or row there closes the window and opens Work as described. Explicit user decision 2026-10-06, again later the same day, supersedes the two-theme selection of the tile: after `Controls` and `Findings` the tile shows all four themes in ISO order (Organizational, People, Physical, Technological), each with its finding count (`0 findings` without hits, an em dash while loading). A theme with hits is one click target over its whole fact cell and opens Work with the `Deviation` status and that theme selected in the `ISO theme` facet; a theme without hits is not clickable, because the facet offers only values with hits. Hover changes only the cell surface; keyboard focus shows the blue focus ring. All six facts stay visible in this tile at every width, unlike the environment tiles, which show two facts below their measured four-fact width. Explicit user decision 2026-10-05: Work offers the Compliance Engine facets `ISO control` (one value per accepted control, `A.8.20 · Networks security`) and `ISO theme` (Organizational, People, Physical, Technological) in mixed Work and in both GPO entry paths, directly before the permanent final `Source: Scan`; this extends the GPO-only facet list. They read the accepted controls of every mappable finding of the timeslot (scanner findings by occurrence, GPO settings by scope and technical identity), offer only values with hits, count against the other active filters, keep an active selection while loading, and disappear with their filter cleared when the engine is off. Decisions stay global per scanner rule. Provisional explicit user decision 2026-10-05 (may change later): after a mapping succeeds, single or batch, the engine accepts its first two proposals in the model's order automatically, only the one if there is a single proposal, with the comment `Accepted automatically: one of the first two proposals of the mapping.`; a control that already has a decision keeps it, so a manual reject is never overridden, and a failure of this step never fails the mapping. +- Explicit user decision 2026-10-05: the regulatory control mapping (Phase 13F, ISO mapping) is the **Compliance Engine** (`src/compliance/`, `server/compliance/`, `tests/compliance/`). It is framework-neutral by intent: ISO/IEC 27001/27002 is its first corpus, later frameworks such as BSI or DORA arrive inside this engine without a Core change. It reads the Core only through `ComplianceReadPort` (single Result DB row by project, filtered Result DB pages, project, assessment and scan-run lists; explicit user decision 2026-10-05: additionally the internal-analysis `policy-evaluation` rows of the same project and timeslot, used solely to read the GPO technical setting ID that the Result DB withholds, so a GPO setting counts once like its rolled-up Work record and the Work ISO facets match it; that ID is never displayed or exported and no other internal field is read) and stores mapping runs, accept/reject decisions, batches and AI overview runs only through its own `ComplianceStore`. Explicit user decision 2026-10-05: all AI integration belongs to this engine. The AI overview (service, model, Client Info panel) moved from the Core into it, and the Result DB `Export JSON` is the engine's 1:1 copy of what its AI features receive: the exact AI overview input built by the same function the overview sends to the model, and every distinct ISO mapping query with the Result DB rows that produce it (the retrieved candidate controls depend on the index and are not exported). The shared model client `server/ai/litellm-client.ts` stays Core platform code. The existing tables of migrations 025 (AI overview runs) and 026 keep their names and data and are owned by the engine; its writes never advance the Core repository revision. Because the Core deletes scan runs and assessments without knowing the engine, the engine removes AI overview runs of a deleted timeslot from its own table on start and on read. The Core project export receives the engine data, including succeeded AI overview runs, beside the Core export at the HTTP route. Its extension points are the Finding Reader `Map ISO controls` action and its window, the Finding Reader `ISO controls` section, the Result DB `Map all ISO controls` batch action and `Export JSON` action, the Data Hub ISO corpus tile, the Client Info `Compliance` row (formerly the `ISO controls` section) and `AI Overview` tab, and the Work `ISO control` and `ISO theme` facets. Switched off, the engine also removes the AI overview tab and its requests and the `Export JSON` action. Explicit user decision 2026-10-05: `Map ISO controls` opens a window like `Accepted Risk` with Accept and Reject per control (no third decision value). The Finding Reader lists the controls whose latest decision is accepted in an `ISO controls` section read like `Description`, after the summary fields. Client Info shows the accepted controls of the selected timeslot directly below the Context heatmap in the same tile layout without severity colour, each with its ISO 27002 theme and the number of deviation findings (a GPO setting counts once); a tile opens Work with the `Deviation` status and that control selected in the `ISO control` facet, like a Context heatmap cell. Explicit user decision 2026-10-06 supersedes only the tile layout of that Client Info section, after feedback that it grows too large: the accepted controls are grouped by ISO 27002 theme in ISO order (Organizational, People, Physical, Technological; a theme without hits is omitted). Each full-width theme heading states its number of controls and of distinct findings (a finding counts once per theme) and opens Work with the `Deviation` status and that theme selected in the `ISO theme` facet. Below it, its controls flow in columns of twice the compact tile width, one compact row per control sorted by finding count, showing the Annex A ID, the complete title (wrapped, never truncated) and the authoritative count, without an amount bar (removed the same day because it cost title width); a row opens Work exactly like the former tile. A theme shows its top six controls; a keyboard-accessible `Show all N controls` button below expands that theme and `Show top 6` collapses it again, and every theme starts collapsed for a new timeslot. The theme heading still opens all of its controls in Work. Explicit user decision 2026-10-06, later the same day, supersedes the in-page list and its top-six cap: Client Info shows a `Compliance` row below the Context heatmap with one tile per framework, in the exact environment tile geometry and fact grid (its own `client-compliance-grid` with the same tile rules; the environment row keeps exactly its six tiles), so later frameworks such as BSI or DORA add a tile instead of a section. The `ISO 27001/27002` tile shows `Controls` (accepted controls), `Findings` (distinct findings) and the two themes with the most findings, each with its finding count (fewer when fewer themes have hits; an em dash while loading, never zero). Its `Results` button opens the shared Results window with the complete grouped list described above, without a cap; a theme heading or row there closes the window and opens Work as described. Explicit user decision 2026-10-06, again later the same day, supersedes the two-theme selection of the tile: after `Controls` and `Findings` the tile shows all four themes in ISO order (Organizational, People, Physical, Technological), each with its finding count (`0 findings` without hits, an em dash while loading). A theme with hits is one click target over its whole fact cell and opens Work with the `Deviation` status and that theme selected in the `ISO theme` facet; a theme without hits is not clickable, because the facet offers only values with hits. Hover changes only the cell surface; keyboard focus shows the blue focus ring. All six facts stay visible in this tile at every width, unlike the environment tiles, which show two facts below their measured four-fact width. Explicit user decision 2026-10-05: Work offers the Compliance Engine facets `ISO control` (one value per accepted control, `A.8.20 · Networks security`) and `ISO theme` (Organizational, People, Physical, Technological) in mixed Work and in both GPO entry paths, directly before the permanent final `Source: Scan`; this extends the GPO-only facet list. They read the accepted controls of every mappable finding of the timeslot (scanner findings by occurrence, GPO settings by scope and technical identity), offer only values with hits, count against the other active filters, keep an active selection while loading, and disappear with their filter cleared when the engine is off. Decisions stay global per scanner rule. Provisional explicit user decision 2026-10-05 (may change later): after a mapping succeeds, single or batch, the engine accepts its first two proposals in the model's order automatically, only the one if there is a single proposal, with the comment `Accepted automatically: one of the first two proposals of the mapping.`; a control that already has a decision keeps it, so a manual reject is never overridden, and a failure of this step never fails the mapping. Explicit user decision 2026-10-08: the Data Hub ISO corpus tile carries the switch `Map new scans automatically` (default off), stored only in the engine's own settings file `regulatory-corpora/auto-mapping.json`, never in a Core table. While it is on, an engine-side backend watcher (`server/compliance/regulatory-auto-mapping.ts`) checks every 10 seconds through `ComplianceReadPort` only: the active (not superseded) scan runs and, because GPO backups create no scan run, the GPO deviation rows of every timeslot as filtered Result DB pages. For each new scan run it starts the existing ISO batch over that run's deviations, and for each timeslot with new GPO deviation rows a batch over that timeslot's GPO deviations, each with a stable idempotency key. Rules with a succeeded or active mapping of the current pipeline versions are reused, so only new rules reach the model; failed mappings are retried, and the provisional auto-acceptance applies unchanged. What exists when the watcher starts or is switched on is the baseline and is not mapped retroactively; new imports wait while mapping is unavailable. Switching the Compliance Engine off in the Engines popup also switches this server-wide setting off (once, from any view); switching the engine on again leaves it off. Already started batches are never cancelled. - Engines may read the Core without restriction but never write to, change or override it. They read only through a declared read-only port and the read-only database access, never through Core tables or internal modules directly. What an engine shows or exports still obeys the Core's privacy and Result DB disclosure rules. - Dependencies point one way: an engine depends on the Core, the Core never depends on an engine. The Core works unchanged with every engine switched off; each engine keeps a permanent on/off invariance test like the Context layer's. Engines do not build on each other except through another engine's published results. - An engine that needs storage gets its own schema or tables after separate approval with backup and migration, never columns in Core tables. An engine cannot create findings: new scanner profiles, importers and parsers write Core data and are always Core changes. @@ -75,7 +75,7 @@ ## Context layer (Phase 13E) -- Latest explicit user decision 2026-10-01 supersedes the preceding permanent-Context decision: the Work toolbar has an `Engines` button at the former Context-button position. It stays marked blue, including on hover and while the popup is closed (an explicit colour-role exception for this control only). The shared popup lists Core Engine, Context Engine, Report Engine and, since the explicit user decision of 2026-10-05, Compliance Engine with checkboxes. All default to active; Core Engine is always checked and cannot be disabled. User choices for optional engines are stored locally under the shared Core-owned engine-activation contract. Disabling Context removes its UI, stops new context requests and clears the active Context Cluster filter; disabling Report blocks new report generation from the UI; disabling Compliance removes the Finding Reader ISO action and panel, the Result DB ISO batch action and `Export JSON`, the Client Info `AI Overview` tab and the Data Hub ISO corpus tile and stops their requests, and re-enabling starts the Finding Reader ISO panel closed. Already submitted backend work is not cancelled by this local preference; its late UI output is suppressed. Cluster badges still select the exact Work filter, and the Context detail section still starts collapsed. Engine independence and Core invariance remain binding. +- Latest explicit user decision 2026-10-01 supersedes the preceding permanent-Context decision: the Work toolbar has an `Engines` button at the former Context-button position. It stays marked blue, including on hover and while the popup is closed (an explicit colour-role exception for this control only). The shared popup lists Core Engine, Context Engine, Report Engine and, since the explicit user decision of 2026-10-05, Compliance Engine with checkboxes. All default to active; Core Engine is always checked and cannot be disabled. User choices for optional engines are stored locally under the shared Core-owned engine-activation contract. Disabling Context removes its UI, stops new context requests and clears the active Context Cluster filter; disabling Report blocks new report generation from the UI; disabling Compliance removes the Finding Reader ISO action and panel, the Result DB ISO batch action and `Export JSON`, the Client Info `AI Overview` tab and the Data Hub ISO corpus tile and stops their requests, and re-enabling starts the Finding Reader ISO panel closed. As the only server-side effect of this local preference (explicit user decision 2026-10-08), disabling Compliance also switches the automatic ISO mapping of new uploads off for the whole server; re-enabling does not switch it on again. Already submitted backend work is not cancelled by this local preference; its late UI output is suppressed. Cluster badges still select the exact Work filter, and the Context detail section still starts collapsed. Engine independence and Core invariance remain binding. - Explicit user decision 2026-10-01 supersedes the earlier Context on/off, default-off, locally saved switch, hover-button and badge-scroll agreements: Context is permanently active and the global toolbar Context button is removed. Ignore obsolete locally saved on/off preferences. Clicking a Finding Reader Context cluster badge selects that exact `Context Cluster` Work facet value, replacing the previous cluster selection while preserving other filters. The Context detail section still starts collapsed for each finding and opens through its own summary. All read-only, provenance and verdict/score boundaries remain binding; only an explicit cluster-filter selection restricts the Work record set. diff --git a/docs/IMPLEMENTATION_PLAN.md b/docs/IMPLEMENTATION_PLAN.md index 9bffb8e..74c900c 100644 --- a/docs/IMPLEMENTATION_PLAN.md +++ b/docs/IMPLEMENTATION_PLAN.md @@ -4210,6 +4210,8 @@ Festgelegtes Betriebsrisiko vom 2026-09-06: Verzeichnisabhaengige Compose-Projek ## 12. Änderungsprotokoll 2026-10-05 - Phase 7 / Management Report (Version 2.16.6): Zahlen an Client Info angeglichen (Report-Engine-Arbeit mit Core-Freigabe Evaluation, Core Tests und Governance; per Test und am Dummy-Client verifiziert). Nutzerentscheidung: Datengrundlage der Reports ist Raw Data der Work-Eintraege, die in Client Info angezeigten Zahlen sind verbindlich (neuer Abschnitt `Report numbers` in AGENTS.md). Behoben: Die Tabelle `Technische Domain-Auswertung` zaehlte Ergebniszeilen (Dummy: 998 bewertet/579 abweichend) statt Work-Eintraege; sie zaehlt jetzt mit demselben Klassifikator und denselben Ergebniszustaenden wie Client Info auf dem vollstaendigen Raw-Data-Blatt. Der Report-Datensatz liest dafuer dieselben aktuellen GPO-Bewertungen wie Client Info (`listCurrentPolicyEvaluationsForMetrics`). Dummy: 759/477 und je Domain 112/54/19/245/7/40 Abweichungen, identisch mit Client Info. Die vier Themen stehen in Summary, Uebersicht, Diagramm und Themenblaettern einheitlich nach Anzahl absteigend. Heatmap an der Basis behoben (Context Engine 1.11.1, Engine-Arbeit): GPO-Einstellungen mit katalogisierter Wertsemantik (Dummy: MinimumPasswordLength, PasswordComplexity, LockoutBadCount) waren ihrem Problemfeld zugeordnet, ihr ASSESSOR-Urteil `Fail` wurde aber nicht gelesen; die Heatmap zaehlte deshalb 474 statt 477. Die Engine fuehrt diese Urteile jetzt unveraendert als `declaredDeviations`; Heatmap, Client Info und Report zeigen 477 (Kennwortrichtlinie 22), `Ohne Problemkategorie` entfaellt. Synthetischer Regressionstest in `tests/assessment-context/context-engine.test.ts`. Zweite Pruefung am Dummy-Client (gleicher Release-Stand): Zuordnung in ASSESSOR vollstaendig (477 Abweichungen je genau ein Problemfeld und ein Thema, Domains gleich Client Info). Im Report korrigiert: nicht ausgefuehrt/Fehler/nicht anwendbar/unbekannt nach Work-Zustaenden (171/2/3/19 statt 170/2/3/0), Erhebungen nur Scan- und GPO-Erhebungslaeufe mit Timeslot-Datum der Quelldateien (5 im Timeslot 22.7.2026 statt 38 Verarbeitungsschritte), Risikoregister wie Client Info (1 erfasst, 0 aktuell gueltig statt 'dokumentiert'). Offen fuer Core: CIS-CAT `Manual` ist normalisiert `informational`, in Work aber `not run`. Versionsentscheidung: Patch 2.16.5 -> 2.16.6 (rueckwaertskompatible Korrekturen an Report- und Heatmap-Zahlen); Versionstests und beide Produktions-Builds bestanden. +2026-10-08 - Phase 13F / Automatische ISO-Zuordnung neuer Uploads (Nutzerauftrag; Engine-Code, dazu Core-Freigabe Platform, Core Tests; Produktversion 2.20.0): Die Compliance Engine ordnet neu hochgeladene Scans und GPO-Auswertungen automatisch ISO-Kontrollen zu, solange der neue Schalter `Map new scans automatically` in der Data-Hub-Kachel `ISO Control Corpus` aktiv ist (Standard aus). `server/compliance/regulatory-auto-mapping.ts` liest den Core ausschliesslich ueber `ComplianceReadPort`: alle 10 s die Projekt-, Timeslot- und Scanlauflisten sowie je Timeslot die GPO-Abweichungszeilen als gefilterte Result-DB-Seiten (`audience: result-db`, `sourceKind: policy-evaluation`, `deviation: true`), weil GPO-Backups keinen Scanlauf erzeugen. Je neuem, nicht abgeloesten Scanlauf startet der bestehende Batch ueber dessen Abweichungen; je Timeslot mit neuen GPO-Abweichungszeilen ein Batch ueber dessen GPO-Abweichungen. Bereits zugeordnete Regeln werden wiederverwendet, sodass nur neue Regeln das Modell erreichen; die vorlaeufige automatische Annahme der ersten zwei Vorschlaege gilt unveraendert. Was beim Start oder Einschalten existiert, ist die Basislinie und wird nicht nachtraeglich zugeordnet; ohne aktiven Korpus warten neue Uploads, bis die Zuordnung verfuegbar ist. Idempotenzschluessel je Scanlauf bzw. je Menge neuer GPO-Zeilen verhindern doppelte Batches. Der Schalter liegt in der Engine-eigenen Datei `regulatory-corpora/auto-mapping.json` (keine Tabelle, keine Migration) und wird im Corpus-Status als `autoMapping` geliefert. Core-Aenderungen (Platform): Route `PUT /reference-data/iso-corpus/auto-mapping` und Methode `setIsoAutoMapping` im API-Client. Synthetische Tests: Watcher (Standard aus ohne Core-Lesezugriff, Basislinie, neuer Scan, neue GPO-Zeilen mit getrennten Batches, abgeloeste Laeufe, Warten auf Verfuegbarkeit, Persistenz und Ausschalten), HTTP-Route und Klickinteraktion der Kachel. `npm test` mit 2.138 bestandenen und 18 uebersprungenen Tests, beide Produktionsbuilds erfolgreich. Versionsentscheidung: rueckwaertskompatible neue Faehigkeit, Minor 2.19.1 -> 2.20.0. Offen: Der Engine-Abschnitt in `AGENTS.md` nennt den Schalter noch nicht (Governance nicht freigegeben); Darstellung in beiden Themes nicht live geprueft, weil die laufende Anwendung reale Daten enthaelt. Kein Phasenstatus geaendert, kein Reimport. Nachtrag desselben Tages (Nutzerauftrag mit Core-Freigabe Presentation): Das Abschalten der Compliance Engine im Engines-Popup schaltet die automatische Zuordnung serverseitig aus. `src/app/engine-preferences.ts` exportiert dafuer `subscribeEnginePreferences`; das Engine-Modul `src/compliance/ui/auto-mapping-engine-link.ts` wird mit der Data-Hub-Kachel beim App-Start geladen und sendet beim Wechsel von an nach aus einmal `setIsoAutoMapping(false)`. Erneutes Einschalten laesst die Zuordnung aus; laufende Batches werden nicht abgebrochen. Die lokale Engine-Wahl wirkt damit auf eine serverweite Einstellung (bei gemeinsamem Server fuer alle Nutzer). Dauerhafter Interaktionstest ueber das Engines-Popup (aus sendet einmal, an sendet nichts, andere Engine sendet nichts); `npm test` mit 2.140 bestandenen und 18 uebersprungenen Tests, beide Produktionsbuilds erfolgreich; Version bleibt 2.20.0 (derselbe noch nicht festgeschriebene Stand). Zweiter Nachtrag (Core-Freigabe Governance): `AGENTS.md` haelt Schalter, Watcher, Basislinie, Wiederverwendung und die Kopplung an das Abschalten der Compliance Engine als Nutzerentscheidung vom 2026-10-08 fest; der offene Punkt dazu ist erledigt. + 2026-10-07 - Ingest / PingCastle-HTML-Gesamtstruktur aus Configuration-DN (Nutzerentscheidung Option 2; Core-Freigabe Ingest, Normalization, Core Tests; Produktversion 2.19.1): Der PingCastle-HTML-Bericht enthaelt kein Feld fuer den Gesamtstrukturnamen, nur die Gesamtstruktur-Funktionsebene. Vollstaendige DNs `CN=Configuration,DC=...` stehen nur in PingCastles eingebetteten Graphdaten (`