2.20.0: Automatic ISO mapping of new scans and GPO imports
The Compliance Engine maps newly uploaded scans and GPO evaluations to ISO controls while the new Data Hub switch "Map new scans automatically" is on (default off). An engine-side watcher reads only through ComplianceReadPort: active scan runs and, for GPOs, the deviation rows of each timeslot as filtered Result DB pages. It starts the existing ISO batch per new scan run or per timeslot with new GPO rows; already mapped rules are reused, so only new rules reach the model. Existing data forms the baseline and is not mapped retroactively. The switch lives in the engine's own settings file, not a Core table. Switching the Compliance Engine off in the Engines popup also switches automatic mapping off server-wide. Core changes (released Platform, Core Tests, Presentation, Governance): PUT /reference-data/iso-corpus/auto-mapping route, setIsoAutoMapping API client method, subscribeEnginePreferences export, AGENTS.md decision and product version 2.19.1 -> 2.20.0.
This commit is contained in:
parent
f35b1795a1
commit
e9ad4e9372
|
|
@ -3404,6 +3404,8 @@ Festgelegtes Betriebsrisiko vom 2026-09-06: Verzeichnisabhaengige Compose-Projek
|
||||||
|
|
||||||
## 12. Änderungsprotokoll
|
## 12. Änderungsprotokoll
|
||||||
|
|
||||||
|
2026-10-08 - Phase 13F / Automatische ISO-Zuordnung neuer Uploads (Nutzerauftrag; Engine-Code, dazu Core-Freigabe Platform, Core Tests; Produktversion 2.20.0): Die Compliance Engine ordnet neu hochgeladene Scans und GPO-Auswertungen automatisch ISO-Kontrollen zu, solange der neue Schalter `Map new scans automatically` in der Data-Hub-Kachel `ISO Control Corpus` aktiv ist (Standard aus). `server/compliance/regulatory-auto-mapping.ts` liest den Core ausschliesslich ueber `ComplianceReadPort`: alle 10 s die Projekt-, Timeslot- und Scanlauflisten sowie je Timeslot die GPO-Abweichungszeilen als gefilterte Result-DB-Seiten (`audience: result-db`, `sourceKind: policy-evaluation`, `deviation: true`), weil GPO-Backups keinen Scanlauf erzeugen. Je neuem, nicht abgeloesten Scanlauf startet der bestehende Batch ueber dessen Abweichungen; je Timeslot mit neuen GPO-Abweichungszeilen ein Batch ueber dessen GPO-Abweichungen. Bereits zugeordnete Regeln werden wiederverwendet, sodass nur neue Regeln das Modell erreichen; die vorlaeufige automatische Annahme der ersten zwei Vorschlaege gilt unveraendert. Was beim Start oder Einschalten existiert, ist die Basislinie und wird nicht nachtraeglich zugeordnet; ohne aktiven Korpus warten neue Uploads, bis die Zuordnung verfuegbar ist. Idempotenzschluessel je Scanlauf bzw. je Menge neuer GPO-Zeilen verhindern doppelte Batches. Der Schalter liegt in der Engine-eigenen Datei `regulatory-corpora/auto-mapping.json` (keine Tabelle, keine Migration) und wird im Corpus-Status als `autoMapping` geliefert. Core-Aenderungen (Platform): Route `PUT /reference-data/iso-corpus/auto-mapping` und Methode `setIsoAutoMapping` im API-Client. Synthetische Tests: Watcher (Standard aus ohne Core-Lesezugriff, Basislinie, neuer Scan, neue GPO-Zeilen mit getrennten Batches, abgeloeste Laeufe, Warten auf Verfuegbarkeit, Persistenz und Ausschalten), HTTP-Route und Klickinteraktion der Kachel. `npm test` mit 2.138 bestandenen und 18 uebersprungenen Tests, beide Produktionsbuilds erfolgreich. Versionsentscheidung: rueckwaertskompatible neue Faehigkeit, Minor 2.19.1 -> 2.20.0. Offen: Der Engine-Abschnitt in `AGENTS.md` nennt den Schalter noch nicht (Governance nicht freigegeben); Darstellung in beiden Themes nicht live geprueft, weil die laufende Anwendung reale Daten enthaelt. Kein Phasenstatus geaendert, kein Reimport. Nachtrag desselben Tages (Nutzerauftrag mit Core-Freigabe Presentation): Das Abschalten der Compliance Engine im Engines-Popup schaltet die automatische Zuordnung serverseitig aus. `src/app/engine-preferences.ts` exportiert dafuer `subscribeEnginePreferences`; das Engine-Modul `src/compliance/ui/auto-mapping-engine-link.ts` wird mit der Data-Hub-Kachel beim App-Start geladen und sendet beim Wechsel von an nach aus einmal `setIsoAutoMapping(false)`. Erneutes Einschalten laesst die Zuordnung aus; laufende Batches werden nicht abgebrochen. Die lokale Engine-Wahl wirkt damit auf eine serverweite Einstellung (bei gemeinsamem Server fuer alle Nutzer). Dauerhafter Interaktionstest ueber das Engines-Popup (aus sendet einmal, an sendet nichts, andere Engine sendet nichts); `npm test` mit 2.140 bestandenen und 18 uebersprungenen Tests, beide Produktionsbuilds erfolgreich; Version bleibt 2.20.0 (derselbe noch nicht festgeschriebene Stand). Zweiter Nachtrag (Core-Freigabe Governance): `AGENTS.md` haelt Schalter, Watcher, Basislinie, Wiederverwendung und die Kopplung an das Abschalten der Compliance Engine als Nutzerentscheidung vom 2026-10-08 fest; der offene Punkt dazu ist erledigt.
|
||||||
|
|
||||||
2026-10-07 - Ingest / PingCastle-HTML-Gesamtstruktur aus Configuration-DN (Nutzerentscheidung Option 2; Core-Freigabe Ingest, Normalization, Core Tests; Produktversion 2.19.1): Der PingCastle-HTML-Bericht enthaelt kein Feld fuer den Gesamtstrukturnamen, nur die Gesamtstruktur-Funktionsebene. Vollstaendige DNs `CN=Configuration,DC=...` stehen nur in PingCastles eingebetteten Graphdaten (`<script type="application/json" data-pingcastle-selector=...>`); im sichtbaren Text ist der DC-Teil abgeschnitten. Extraktor 2.4.0 liest diese Bloecke als inerte Daten (hoechstens 64 Bloecke, je 5 Mio. Zeichen, nur `JSON.parse`, nur Zeichenketten `nodes[].name`), nimmt jeden DN der Form `[CN=...,]CN=Configuration,DC=...` und leitet daraus die Gesamtstruktur-Stammdomaene ab, weil der Configuration-Namenskontext stets unter ihr liegt. `forest.name` entsteht nur, wenn alle Fundstellen dieselbe Stammdomaene nennen; es traegt `derivation: configuration-partition-dn`, die Bezeichnung `Derived from the configuration partition path`, die Fundstellenzahl und den Quellpfad der ersten Fundstelle. Widersprechende Stammdomaenen ergeben die Warnung `pingcastle-forest-root-conflict` und keinen Wert; fehlende, abgeschnittene, nicht-JSON- oder unlesbare Daten ergeben keinen Wert. Nie aus dem Domaenennamen geraten. Profil PingCastle 1.7.0 mit append-only Manifest (`reimportRequired`, `environment-context`). Nachtrag mit Core-Freigabe Presentation: Client Info (Domain Information und Directory) waehlt fuer `Forest` zuerst eine native Beobachtung ohne Ableitung (PingCastle-XML, Purple Knight) und erst sonst die abgeleitete; eine abgeleitete zeigt nach dem Wert gedaempft `(derived from configuration path)` ueber das neue optionale Feld `note` von `DashboardFact` und die Klasse `.client-fact-note` (vorhandene Tokens `--muted`, `--fw-regular`). Synthetische Renderpruefung: nur abgeleitet mit Hinweis, nativ plus abgeleitet nur nativ, ohne Beobachtung Gedankenstrich. Nachtrag mit Core-Freigabe Core Tests: dauerhafter Verhaltenstest in `tests/features/dashboard-layout.test.ts` (nur abgeleitet mit Hinweis im gerenderten `dd`, nativ vor abgeleitet in beiden Reihenfolgen, nicht beobachtete native Werte ignoriert, ohne Beobachtung Gedankenstrich, nativ ohne Hinweisklasse); `npm test` mit 2.131 bestandenen und 18 uebersprungenen Tests. Offen: der Management Report (`src/reporting/client-report-projection.ts`, Report Engine) liest `forest.name` weiter ohne Vorrang und Hinweis; beide Themes nicht live geprueft, weil die laufende Anwendung reale Daten enthaelt. Verifikation: synthetischer Test fuer Uebereinstimmung, Konflikt und Nichtableitung; Dummy-Bericht ergibt `sevenkingdoms.local` aus vier Fundstellen ohne Konflikt; `npm test` 2.129 bestanden, ein lastbedingter Zeitfehler in `datahub-metrics-prefetch` besteht einzeln; beide Produktionsbuilds erfolgreich. Kein Phasenstatus geaendert.
|
2026-10-07 - Ingest / PingCastle-HTML-Gesamtstruktur aus Configuration-DN (Nutzerentscheidung Option 2; Core-Freigabe Ingest, Normalization, Core Tests; Produktversion 2.19.1): Der PingCastle-HTML-Bericht enthaelt kein Feld fuer den Gesamtstrukturnamen, nur die Gesamtstruktur-Funktionsebene. Vollstaendige DNs `CN=Configuration,DC=...` stehen nur in PingCastles eingebetteten Graphdaten (`<script type="application/json" data-pingcastle-selector=...>`); im sichtbaren Text ist der DC-Teil abgeschnitten. Extraktor 2.4.0 liest diese Bloecke als inerte Daten (hoechstens 64 Bloecke, je 5 Mio. Zeichen, nur `JSON.parse`, nur Zeichenketten `nodes[].name`), nimmt jeden DN der Form `[CN=...,]CN=Configuration,DC=...` und leitet daraus die Gesamtstruktur-Stammdomaene ab, weil der Configuration-Namenskontext stets unter ihr liegt. `forest.name` entsteht nur, wenn alle Fundstellen dieselbe Stammdomaene nennen; es traegt `derivation: configuration-partition-dn`, die Bezeichnung `Derived from the configuration partition path`, die Fundstellenzahl und den Quellpfad der ersten Fundstelle. Widersprechende Stammdomaenen ergeben die Warnung `pingcastle-forest-root-conflict` und keinen Wert; fehlende, abgeschnittene, nicht-JSON- oder unlesbare Daten ergeben keinen Wert. Nie aus dem Domaenennamen geraten. Profil PingCastle 1.7.0 mit append-only Manifest (`reimportRequired`, `environment-context`). Nachtrag mit Core-Freigabe Presentation: Client Info (Domain Information und Directory) waehlt fuer `Forest` zuerst eine native Beobachtung ohne Ableitung (PingCastle-XML, Purple Knight) und erst sonst die abgeleitete; eine abgeleitete zeigt nach dem Wert gedaempft `(derived from configuration path)` ueber das neue optionale Feld `note` von `DashboardFact` und die Klasse `.client-fact-note` (vorhandene Tokens `--muted`, `--fw-regular`). Synthetische Renderpruefung: nur abgeleitet mit Hinweis, nativ plus abgeleitet nur nativ, ohne Beobachtung Gedankenstrich. Nachtrag mit Core-Freigabe Core Tests: dauerhafter Verhaltenstest in `tests/features/dashboard-layout.test.ts` (nur abgeleitet mit Hinweis im gerenderten `dd`, nativ vor abgeleitet in beiden Reihenfolgen, nicht beobachtete native Werte ignoriert, ohne Beobachtung Gedankenstrich, nativ ohne Hinweisklasse); `npm test` mit 2.131 bestandenen und 18 uebersprungenen Tests. Offen: der Management Report (`src/reporting/client-report-projection.ts`, Report Engine) liest `forest.name` weiter ohne Vorrang und Hinweis; beide Themes nicht live geprueft, weil die laufende Anwendung reale Daten enthaelt. Verifikation: synthetischer Test fuer Uebereinstimmung, Konflikt und Nichtableitung; Dummy-Bericht ergibt `sevenkingdoms.local` aus vier Fundstellen ohne Konflikt; `npm test` 2.129 bestanden, ein lastbedingter Zeitfehler in `datahub-metrics-prefetch` besteht einzeln; beide Produktionsbuilds erfolgreich. Kein Phasenstatus geaendert.
|
||||||
|
|
||||||
2026-10-07 - Ingest / PingCastle-XML-Domaenenfakten (Core-Freigabe Ingest, Core Tests; danach Ingest, Normalization, Platform; Produktversion 2.19.1): Der XML-Import legte jedes `HealthcheckData`-Element nur unter einer pfadbasierten Eigenschaft `pingcastle.xml.*` ab; Client Info, Data Hub und Report fanden deshalb weder Domaene noch die uebrigen Domaenenfakten (nur das Scoring las `DomainFQDN` ueber einen Leseadapter). Extraktor 2.3.0 ordnet jetzt ausschliesslich die exakten Top-Level-Blaetter `DomainFQDN`, `NetBIOSName`, `ForestFQDN`, `DomainCreation`, `DomainFunctionalLevel`, `ForestFunctionalLevel`, `SchemaVersion`, `IsRecycleBinEnabled`, `NumberOfDC`, `GenerationDate` und `EngineVersion` denselben Eigenschaften wie der HTML-Import zu (`domain.fqdn`, `domain.netbios_name`, `forest.name`, ...). Originalwert und XML-Pfad bleiben erhalten; Funktionsebenen bleiben die numerischen XML-Codes und werden nicht uebersetzt; `DomainSid` und verschachtelte gleichnamige Elemente erhalten keine semantische Eigenschaft. Profil PingCastle 1.6.0 mit neuem, append-only Manifest (`reimportRequired`, Dimension `environment-context`); da Manifeste je Profil gelten, empfiehlt Data Hub den Reimport auch fuer HTML-Importe, die dadurch nichts gewinnen. Synthetischer Test fuer Zuordnung, ausgenommene SID und verschachtelte Elemente; Pruefung gegen den freigegebenen Dummy-Export. Nachtrag mit Core-Freigabe Core Tests: Die Tests, die PingCastle 1.5.0 als aktuelles Profil pinnten, sind auf 1.6.0 angeglichen; 1.5.0 ist als reimportpflichtig abgesichert. `npm test` mit 2.129 bestandenen und 18 uebersprungenen Tests sowie beide Produktionsbuilds erfolgreich. Versionsentscheidung: rueckwaertskompatible Korrektur, faellt in den noch nicht festgeschriebenen Patch 2.19.1. Kein Phasenstatus geaendert.
|
2026-10-07 - Ingest / PingCastle-XML-Domaenenfakten (Core-Freigabe Ingest, Core Tests; danach Ingest, Normalization, Platform; Produktversion 2.19.1): Der XML-Import legte jedes `HealthcheckData`-Element nur unter einer pfadbasierten Eigenschaft `pingcastle.xml.*` ab; Client Info, Data Hub und Report fanden deshalb weder Domaene noch die uebrigen Domaenenfakten (nur das Scoring las `DomainFQDN` ueber einen Leseadapter). Extraktor 2.3.0 ordnet jetzt ausschliesslich die exakten Top-Level-Blaetter `DomainFQDN`, `NetBIOSName`, `ForestFQDN`, `DomainCreation`, `DomainFunctionalLevel`, `ForestFunctionalLevel`, `SchemaVersion`, `IsRecycleBinEnabled`, `NumberOfDC`, `GenerationDate` und `EngineVersion` denselben Eigenschaften wie der HTML-Import zu (`domain.fqdn`, `domain.netbios_name`, `forest.name`, ...). Originalwert und XML-Pfad bleiben erhalten; Funktionsebenen bleiben die numerischen XML-Codes und werden nicht uebersetzt; `DomainSid` und verschachtelte gleichnamige Elemente erhalten keine semantische Eigenschaft. Profil PingCastle 1.6.0 mit neuem, append-only Manifest (`reimportRequired`, Dimension `environment-context`); da Manifeste je Profil gelten, empfiehlt Data Hub den Reimport auch fuer HTML-Importe, die dadurch nichts gewinnen. Synthetischer Test fuer Zuordnung, ausgenommene SID und verschachtelte Elemente; Pruefung gegen den freigegebenen Dummy-Export. Nachtrag mit Core-Freigabe Core Tests: Die Tests, die PingCastle 1.5.0 als aktuelles Profil pinnten, sind auf 1.6.0 angeglichen; 1.5.0 ist als reimportpflichtig abgesichert. `npm test` mit 2.129 bestandenen und 18 uebersprungenen Tests sowie beide Produktionsbuilds erfolgreich. Versionsentscheidung: rueckwaertskompatible Korrektur, faellt in den noch nicht festgeschriebenen Patch 2.19.1. Kein Phasenstatus geaendert.
|
||||||
|
|
|
||||||
|
|
@ -1,12 +1,12 @@
|
||||||
{
|
{
|
||||||
"name": "assessor",
|
"name": "assessor",
|
||||||
"version": "2.19.1",
|
"version": "2.20.0",
|
||||||
"lockfileVersion": 3,
|
"lockfileVersion": 3,
|
||||||
"requires": true,
|
"requires": true,
|
||||||
"packages": {
|
"packages": {
|
||||||
"": {
|
"": {
|
||||||
"name": "assessor",
|
"name": "assessor",
|
||||||
"version": "2.19.1",
|
"version": "2.20.0",
|
||||||
"dependencies": {
|
"dependencies": {
|
||||||
"@fontsource-variable/inter": "^5.3.0",
|
"@fontsource-variable/inter": "^5.3.0",
|
||||||
"jsdom": "^29.1.1",
|
"jsdom": "^29.1.1",
|
||||||
|
|
|
||||||
|
|
@ -1,6 +1,6 @@
|
||||||
{
|
{
|
||||||
"name": "assessor",
|
"name": "assessor",
|
||||||
"version": "2.19.1",
|
"version": "2.20.0",
|
||||||
"private": true,
|
"private": true,
|
||||||
"type": "module",
|
"type": "module",
|
||||||
"scripts": {
|
"scripts": {
|
||||||
|
|
|
||||||
|
|
@ -775,6 +775,12 @@ export function createAssessorHttpServer(options: AssessorHttpServerOptions): Se
|
||||||
...corpusStatus, configured: false, disabledReason: 'An ISO upload token of at least 32 characters must be configured.'
|
...corpusStatus, configured: false, disabledReason: 'An ISO upload token of at least 32 characters must be configured.'
|
||||||
}, requestId); return;
|
}, requestId); return;
|
||||||
}
|
}
|
||||||
|
if (segments.join('/') === 'reference-data/iso-corpus/auto-mapping' && method === 'PUT') {
|
||||||
|
if (!options.regulatoryCorpusUploadService) throw new HttpError(503, 'iso_corpus_unavailable', 'ISO corpus management is not configured.');
|
||||||
|
const body = await readJson<{ enabled: boolean }>(request, maximumBodyBytes);
|
||||||
|
if (typeof body?.enabled !== 'boolean') throw new HttpError(400, 'invalid_auto_mapping', 'enabled must be true or false.');
|
||||||
|
status = 200; sendJson(response, status, await options.regulatoryCorpusUploadService.setAutoMapping(body.enabled), requestId); return;
|
||||||
|
}
|
||||||
if (segments.join('/') === 'reference-data/iso-corpus' && method === 'POST') {
|
if (segments.join('/') === 'reference-data/iso-corpus' && method === 'POST') {
|
||||||
if (!options.regulatoryCorpusUploadService) throw new HttpError(503, 'iso_corpus_unavailable', 'ISO corpus management is not configured.');
|
if (!options.regulatoryCorpusUploadService) throw new HttpError(503, 'iso_corpus_unavailable', 'ISO corpus management is not configured.');
|
||||||
verifyIsoUploadToken(request, options.isoUploadToken);
|
verifyIsoUploadToken(request, options.isoUploadToken);
|
||||||
|
|
|
||||||
|
|
@ -9,6 +9,7 @@ import { PostgresComplianceStore } from './postgres-compliance-store';
|
||||||
import { createRegulatoryRagRuntime } from './regulatory-rag-runtime';
|
import { createRegulatoryRagRuntime } from './regulatory-rag-runtime';
|
||||||
import { RegulatoryMappingService } from './regulatory-mapping-service';
|
import { RegulatoryMappingService } from './regulatory-mapping-service';
|
||||||
import { RegulatoryCorpusUploadService } from './regulatory-corpus-upload-service';
|
import { RegulatoryCorpusUploadService } from './regulatory-corpus-upload-service';
|
||||||
|
import { RegulatoryAutoMapping } from './regulatory-auto-mapping';
|
||||||
import { AiOverviewService, type AiOverviewServiceOptions } from './ai-overview-service';
|
import { AiOverviewService, type AiOverviewServiceOptions } from './ai-overview-service';
|
||||||
|
|
||||||
export interface ComplianceEngineRuntime {
|
export interface ComplianceEngineRuntime {
|
||||||
|
|
@ -58,16 +59,20 @@ export async function createComplianceEngineRuntime(options: ComplianceEngineRun
|
||||||
mappingService = new RegulatoryMappingService(read, store, { disabledReason });
|
mappingService = new RegulatoryMappingService(read, store, { disabledReason });
|
||||||
}
|
}
|
||||||
await mappingService.recoverInterruptedRuns();
|
await mappingService.recoverInterruptedRuns();
|
||||||
|
const corpusRoot = join(options.artifactRoot, 'regulatory-corpora');
|
||||||
|
const autoMapping = new RegulatoryAutoMapping({ settingsPath: join(corpusRoot, 'auto-mapping.json'), read, mapping: mappingService });
|
||||||
const corpusUploadService = new RegulatoryCorpusUploadService({
|
const corpusUploadService = new RegulatoryCorpusUploadService({
|
||||||
root: join(options.artifactRoot, 'regulatory-corpora'),
|
root: corpusRoot,
|
||||||
ingestion: rag?.ingestion,
|
ingestion: rag?.ingestion,
|
||||||
countIndexed: rag ? (corpusId, corpusVersion) => rag!.index.count({ corpusId, corpusVersion }) : undefined,
|
countIndexed: rag ? (corpusId, corpusVersion) => rag!.index.count({ corpusId, corpusVersion }) : undefined,
|
||||||
pruneIndexed: rag ? (corpusId, corpusVersion) => rag!.index.deleteOtherCorpusVersions(corpusId, corpusVersion) : undefined,
|
pruneIndexed: rag ? (corpusId, corpusVersion) => rag!.index.deleteOtherCorpusVersions(corpusId, corpusVersion) : undefined,
|
||||||
mapping: mappingService,
|
mapping: mappingService,
|
||||||
|
autoMapping,
|
||||||
embeddingModel: rag?.embeddingModel,
|
embeddingModel: rag?.embeddingModel,
|
||||||
disabledReason
|
disabledReason
|
||||||
});
|
});
|
||||||
await corpusUploadService.initialize();
|
await corpusUploadService.initialize();
|
||||||
|
await autoMapping.start();
|
||||||
const aiOverviewService = new AiOverviewService(read, store, { ...options.aiOverview, client: options.aiClient });
|
const aiOverviewService = new AiOverviewService(read, store, { ...options.aiOverview, client: options.aiClient });
|
||||||
await aiOverviewService.recoverInterruptedRuns();
|
await aiOverviewService.recoverInterruptedRuns();
|
||||||
return {
|
return {
|
||||||
|
|
@ -75,7 +80,7 @@ export async function createComplianceEngineRuntime(options: ComplianceEngineRun
|
||||||
corpusUploadService,
|
corpusUploadService,
|
||||||
aiOverviewService,
|
aiOverviewService,
|
||||||
store,
|
store,
|
||||||
stop: () => Promise.all([corpusUploadService.stop().then(() => mappingService.close()), aiOverviewService.close()]).then(() => undefined),
|
stop: () => Promise.all([autoMapping.stop(), corpusUploadService.stop().then(() => mappingService.close()), aiOverviewService.close()]).then(() => undefined),
|
||||||
close: async () => { await pool?.end(); }
|
close: async () => { await pool?.end(); }
|
||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -0,0 +1,157 @@
|
||||||
|
import { createHash, randomUUID } from 'node:crypto';
|
||||||
|
import { mkdir, readFile, rename, rm, writeFile } from 'node:fs/promises';
|
||||||
|
import { dirname } from 'node:path';
|
||||||
|
import type { ComplianceReadPort } from '../../src/compliance/compliance-store';
|
||||||
|
import type { RegulatoryMappingBatchCreateRequest } from '../../src/compliance/regulatory-mapping';
|
||||||
|
import type { RegulatoryMappingService } from './regulatory-mapping-service';
|
||||||
|
|
||||||
|
const DEFAULT_POLL_INTERVAL_MS = 10_000;
|
||||||
|
const PAGE_SIZE = 500;
|
||||||
|
|
||||||
|
interface AutoMappingSettings { version: 1; enabled: boolean }
|
||||||
|
interface PendingBatch { projectId: string; request: RegulatoryMappingBatchCreateRequest }
|
||||||
|
|
||||||
|
export interface RegulatoryAutoMappingOptions {
|
||||||
|
/** Engine-owned settings file; no Core table or column stores the switch. */
|
||||||
|
settingsPath: string;
|
||||||
|
read: Pick<ComplianceReadPort, 'listProjects' | 'listAssessments' | 'listProjectScanRuns' | 'queryUnifiedResults'>;
|
||||||
|
mapping: Pick<RegulatoryMappingService, 'generationEnabled' | 'startBatch'>;
|
||||||
|
pollIntervalMs?: number;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Maps newly uploaded scans and GPO evaluations automatically while its Data Hub switch is on (default off).
|
||||||
|
* It reads the Core only through the read-only port: committed scan runs, and the GPO deviation rows of each
|
||||||
|
* timeslot as filtered Result DB pages (GPO backups have no scan run). For every new scan run, and every
|
||||||
|
* timeslot with new GPO deviation rows, it starts the existing ISO batch on those deviations; rules mapped
|
||||||
|
* before are reused, so only new rules reach the model. What exists when the watcher starts or is switched
|
||||||
|
* on forms the baseline and is not mapped retroactively.
|
||||||
|
*/
|
||||||
|
export class RegulatoryAutoMapping {
|
||||||
|
private enabled = false;
|
||||||
|
private knownScanRuns: Set<string> | null = null;
|
||||||
|
private knownGpoRows: Set<string> | null = null;
|
||||||
|
private readonly pending = new Map<string, PendingBatch>();
|
||||||
|
private timer?: ReturnType<typeof setInterval>;
|
||||||
|
private polling?: Promise<void>;
|
||||||
|
|
||||||
|
constructor(private readonly options: RegulatoryAutoMappingOptions) {}
|
||||||
|
|
||||||
|
get isEnabled(): boolean { return this.enabled; }
|
||||||
|
|
||||||
|
async start(): Promise<void> {
|
||||||
|
const settings = await this.readSettings();
|
||||||
|
if (settings?.enabled) await this.activate();
|
||||||
|
}
|
||||||
|
|
||||||
|
async setEnabled(enabled: boolean): Promise<void> {
|
||||||
|
if (typeof enabled !== 'boolean') throw new TypeError('enabled must be a boolean.');
|
||||||
|
await this.writeSettings({ version: 1, enabled });
|
||||||
|
if (enabled && !this.enabled) await this.activate();
|
||||||
|
if (!enabled && this.enabled) await this.stop();
|
||||||
|
}
|
||||||
|
|
||||||
|
async stop(): Promise<void> {
|
||||||
|
this.enabled = false;
|
||||||
|
if (this.timer) clearInterval(this.timer);
|
||||||
|
this.timer = undefined;
|
||||||
|
await this.polling;
|
||||||
|
this.knownScanRuns = null;
|
||||||
|
this.knownGpoRows = null;
|
||||||
|
this.pending.clear();
|
||||||
|
}
|
||||||
|
|
||||||
|
/** One watcher pass; serialized so a slow pass never overlaps the next. */
|
||||||
|
poll(): Promise<void> {
|
||||||
|
this.polling ??= this.pass().catch(() => undefined).finally(() => { this.polling = undefined; });
|
||||||
|
return this.polling;
|
||||||
|
}
|
||||||
|
|
||||||
|
private async activate(): Promise<void> {
|
||||||
|
this.enabled = true;
|
||||||
|
await this.poll();
|
||||||
|
this.timer = setInterval(() => { void this.poll(); }, this.options.pollIntervalMs ?? DEFAULT_POLL_INTERVAL_MS);
|
||||||
|
this.timer.unref?.();
|
||||||
|
}
|
||||||
|
|
||||||
|
private async pass(): Promise<void> {
|
||||||
|
if (!this.enabled) return;
|
||||||
|
const scanRuns: Array<{ id: string; projectId: string; assessmentId: string }> = [];
|
||||||
|
const gpoRows = new Map<string, { projectId: string; assessmentId: string; ids: string[] }>();
|
||||||
|
for (const project of await this.options.read.listProjects()) {
|
||||||
|
for (const run of await this.options.read.listProjectScanRuns(project.id)) {
|
||||||
|
if (!run.supersededByScanRunId) scanRuns.push({ id: run.id, projectId: project.id, assessmentId: run.assessmentId });
|
||||||
|
}
|
||||||
|
for (const assessment of await this.options.read.listAssessments(project.id)) {
|
||||||
|
const ids = await this.gpoDeviationRowIds(project.id, assessment.id);
|
||||||
|
if (ids.length) gpoRows.set(assessment.id, { projectId: project.id, assessmentId: assessment.id, ids });
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if (!this.knownScanRuns || !this.knownGpoRows) {
|
||||||
|
this.knownScanRuns = new Set(scanRuns.map(run => run.id));
|
||||||
|
this.knownGpoRows = new Set([...gpoRows.values()].flatMap(entry => entry.ids));
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
for (const run of scanRuns) {
|
||||||
|
if (this.knownScanRuns.has(run.id)) continue;
|
||||||
|
this.knownScanRuns.add(run.id);
|
||||||
|
this.pending.set(`scan:${run.id}`, { projectId: run.projectId, request: {
|
||||||
|
// Stable per scan run, so a restart or a second pass never creates a duplicate batch.
|
||||||
|
idempotencyKey: safeKey(`auto:${run.id}`),
|
||||||
|
assessmentIds: [run.assessmentId], scanRunIds: [run.id], filters: { deviation: true }
|
||||||
|
} });
|
||||||
|
}
|
||||||
|
for (const { projectId, assessmentId, ids } of gpoRows.values()) {
|
||||||
|
const added = ids.filter(id => !this.knownGpoRows!.has(id));
|
||||||
|
if (!added.length) continue;
|
||||||
|
for (const id of added) this.knownGpoRows.add(id);
|
||||||
|
// Stable per set of new GPO rows; one batch maps every GPO deviation of the timeslot.
|
||||||
|
const fingerprint = createHash('sha256').update(added.sort().join('\n'), 'utf8').digest('hex').slice(0, 16);
|
||||||
|
this.pending.set(`gpo:${assessmentId}`, { projectId, request: {
|
||||||
|
idempotencyKey: safeKey(`auto-gpo:${fingerprint}:${assessmentId}`),
|
||||||
|
assessmentIds: [assessmentId], scanRunIds: [], filters: { sourceKind: ['policy-evaluation'], deviation: true }
|
||||||
|
} });
|
||||||
|
}
|
||||||
|
// New imports wait until the ISO corpus and model are available.
|
||||||
|
if (!this.options.mapping.generationEnabled) return;
|
||||||
|
for (const [key, { projectId, request }] of [...this.pending]) {
|
||||||
|
if (!this.enabled) return;
|
||||||
|
this.pending.delete(key);
|
||||||
|
try { await this.options.mapping.startBatch(projectId, request); }
|
||||||
|
catch { /* A removed or superseded timeslot is skipped; the manual batch stays available. */ }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
private async gpoDeviationRowIds(projectId: string, assessmentId: string): Promise<string[]> {
|
||||||
|
const ids: string[] = [];
|
||||||
|
for (let offset = 0; ; offset += PAGE_SIZE) {
|
||||||
|
const page = await this.options.read.queryUnifiedResults({ audience: 'result-db', projectId, assessmentIds: [assessmentId],
|
||||||
|
sourceKind: ['policy-evaluation'], deviation: true, offset, limit: PAGE_SIZE });
|
||||||
|
ids.push(...page.items.map(item => item.resultProjectionId));
|
||||||
|
if (offset + page.items.length >= page.total || page.items.length === 0) return ids;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
private async readSettings(): Promise<AutoMappingSettings | null> {
|
||||||
|
try {
|
||||||
|
const parsed = JSON.parse(await readFile(this.options.settingsPath, 'utf8')) as Partial<AutoMappingSettings>;
|
||||||
|
return { version: 1, enabled: parsed.enabled === true };
|
||||||
|
} catch (error) {
|
||||||
|
if ((error as NodeJS.ErrnoException).code === 'ENOENT') return null;
|
||||||
|
throw error;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
private async writeSettings(settings: AutoMappingSettings): Promise<void> {
|
||||||
|
await mkdir(dirname(this.options.settingsPath), { recursive: true, mode: 0o700 });
|
||||||
|
const temporary = `${this.options.settingsPath}.${randomUUID()}.tmp`;
|
||||||
|
try {
|
||||||
|
await writeFile(temporary, JSON.stringify(settings), { mode: 0o600, flag: 'wx' });
|
||||||
|
await rename(temporary, this.options.settingsPath);
|
||||||
|
} finally { await rm(temporary, { force: true }); }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
function safeKey(value: string): string {
|
||||||
|
return value.replace(/[^A-Za-z0-9._:-]/g, '-').slice(0, 128);
|
||||||
|
}
|
||||||
|
|
@ -7,6 +7,7 @@ import type { IsoPdfCorpusOptions } from './iso-pdf-corpus';
|
||||||
import { prepareIsoCorpusInWorker } from './iso-pdf-worker-client';
|
import { prepareIsoCorpusInWorker } from './iso-pdf-worker-client';
|
||||||
import type { RegulatoryIngestionService } from './regulatory-rag-service';
|
import type { RegulatoryIngestionService } from './regulatory-rag-service';
|
||||||
import type { RegulatoryMappingService } from './regulatory-mapping-service';
|
import type { RegulatoryMappingService } from './regulatory-mapping-service';
|
||||||
|
import type { RegulatoryAutoMapping } from './regulatory-auto-mapping';
|
||||||
|
|
||||||
const MAXIMUM_PDF_BYTES = 32 * 1024 * 1024;
|
const MAXIMUM_PDF_BYTES = 32 * 1024 * 1024;
|
||||||
const MAXIMUM_CORPUS_BYTES = 64 * 1024 * 1024;
|
const MAXIMUM_CORPUS_BYTES = 64 * 1024 * 1024;
|
||||||
|
|
@ -21,6 +22,7 @@ export interface RegulatoryCorpusUploadOptions {
|
||||||
root: string;
|
root: string;
|
||||||
ingestion?: RegulatoryIngestionService;
|
ingestion?: RegulatoryIngestionService;
|
||||||
mapping: RegulatoryMappingService;
|
mapping: RegulatoryMappingService;
|
||||||
|
autoMapping?: Pick<RegulatoryAutoMapping, 'isEnabled' | 'setEnabled'>;
|
||||||
embeddingModel?: string;
|
embeddingModel?: string;
|
||||||
disabledReason?: string;
|
disabledReason?: string;
|
||||||
countIndexed?: (corpusId: string, corpusVersion: string) => Promise<number>;
|
countIndexed?: (corpusId: string, corpusVersion: string) => Promise<number>;
|
||||||
|
|
@ -77,10 +79,18 @@ export class RegulatoryCorpusUploadService {
|
||||||
configured: Boolean(this.options.ingestion),
|
configured: Boolean(this.options.ingestion),
|
||||||
...(!this.options.ingestion ? { disabledReason: this.options.disabledReason ?? 'Embedding or Qdrant is not configured.' } : {}),
|
...(!this.options.ingestion ? { disabledReason: this.options.disabledReason ?? 'Embedding or Qdrant is not configured.' } : {}),
|
||||||
active: this.active,
|
active: this.active,
|
||||||
job: this.job
|
job: this.job,
|
||||||
|
...(this.options.autoMapping ? { autoMapping: { enabled: this.options.autoMapping.isEnabled } } : {})
|
||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|
||||||
|
async setAutoMapping(enabled: boolean): Promise<RegulatoryCorpusStatus> {
|
||||||
|
if (!this.options.autoMapping) throw new RegulatoryCorpusUploadError('regulatory_auto_mapping_unavailable', 'Automatic ISO mapping is not configured.', 503);
|
||||||
|
if (typeof enabled !== 'boolean') throw new RegulatoryCorpusUploadError('invalid_auto_mapping', 'enabled must be true or false.');
|
||||||
|
await this.options.autoMapping.setEnabled(enabled);
|
||||||
|
return this.liveStatus();
|
||||||
|
}
|
||||||
|
|
||||||
async liveStatus(): Promise<RegulatoryCorpusStatus> {
|
async liveStatus(): Promise<RegulatoryCorpusStatus> {
|
||||||
const status = this.status();
|
const status = this.status();
|
||||||
if (!this.active || !this.options.countIndexed) return status;
|
if (!this.active || !this.options.countIndexed) return status;
|
||||||
|
|
|
||||||
|
|
@ -484,6 +484,10 @@ export class AssessorApiClient {
|
||||||
return this.request('/reference-data/iso-corpus');
|
return this.request('/reference-data/iso-corpus');
|
||||||
}
|
}
|
||||||
|
|
||||||
|
setIsoAutoMapping(enabled: boolean): Promise<RegulatoryCorpusStatus> {
|
||||||
|
return this.request('/reference-data/iso-corpus/auto-mapping', { method: 'PUT', body: JSON.stringify({ enabled }) });
|
||||||
|
}
|
||||||
|
|
||||||
async submitIsoCorpus(iso27001: File, iso27002: File, token: string): Promise<RegulatoryCorpusStatus> {
|
async submitIsoCorpus(iso27001: File, iso27002: File, token: string): Promise<RegulatoryCorpusStatus> {
|
||||||
const [first, second] = await Promise.all([iso27001.arrayBuffer(), iso27002.arrayBuffer()]);
|
const [first, second] = await Promise.all([iso27001.arrayBuffer(), iso27002.arrayBuffer()]);
|
||||||
return this.request('/reference-data/iso-corpus', {
|
return this.request('/reference-data/iso-corpus', {
|
||||||
|
|
|
||||||
|
|
@ -45,6 +45,11 @@ function subscribe(listener: () => void) {
|
||||||
return () => { listeners.delete(listener); window.removeEventListener('storage', changed); };
|
return () => { listeners.delete(listener); window.removeEventListener('storage', changed); };
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/** Calls the listener on every engine choice, including one made in another tab; returns the unsubscribe. */
|
||||||
|
export function subscribeEnginePreferences(listener: () => void): () => void {
|
||||||
|
return subscribe(listener);
|
||||||
|
}
|
||||||
|
|
||||||
export function useEnginePreferences(): Preferences {
|
export function useEnginePreferences(): Preferences {
|
||||||
return useSyncExternalStore(subscribe, readEnginePreferences, () => defaults);
|
return useSyncExternalStore(subscribe, readEnginePreferences, () => defaults);
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -27,4 +27,6 @@ export interface RegulatoryCorpusStatus {
|
||||||
active: RegulatoryCorpusSummary | null;
|
active: RegulatoryCorpusSummary | null;
|
||||||
indexStatus?: 'ready' | 'incomplete' | 'unavailable';
|
indexStatus?: 'ready' | 'incomplete' | 'unavailable';
|
||||||
job: RegulatoryCorpusJob | null;
|
job: RegulatoryCorpusJob | null;
|
||||||
|
/** Data Hub switch: map the deviations of newly uploaded scans automatically. */
|
||||||
|
autoMapping?: { enabled: boolean };
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -5,9 +5,13 @@ import { useBackendWorkActivity } from './use-backend-work-activity';
|
||||||
import { statusPollApi } from './status-poll-client';
|
import { statusPollApi } from './status-poll-client';
|
||||||
import type { RegulatoryCorpusStatus } from '../regulatory-corpus-management';
|
import type { RegulatoryCorpusStatus } from '../regulatory-corpus-management';
|
||||||
import { TriangleIcon } from '../../components/TriangleIcon';
|
import { TriangleIcon } from '../../components/TriangleIcon';
|
||||||
|
import './compliance.css';
|
||||||
|
import { linkAutoMappingToEngineSwitch } from './auto-mapping-engine-link';
|
||||||
|
|
||||||
const api = new AssessorApiClient();
|
const api = new AssessorApiClient();
|
||||||
const MAXIMUM_PDF_BYTES = 32 * 1024 * 1024;
|
const MAXIMUM_PDF_BYTES = 32 * 1024 * 1024;
|
||||||
|
// Data Hub loads with the app, so the link is active in every view that offers the Engines popup.
|
||||||
|
if (typeof window !== 'undefined') linkAutoMappingToEngineSwitch(api);
|
||||||
|
|
||||||
/** Data Hub ISO corpus tile; a switched-off Compliance Engine renders nothing and requests nothing. */
|
/** Data Hub ISO corpus tile; a switched-off Compliance Engine renders nothing and requests nothing. */
|
||||||
export function IsoCorpusLibrary() {
|
export function IsoCorpusLibrary() {
|
||||||
|
|
@ -20,6 +24,7 @@ function IsoCorpusLibraryPanel() {
|
||||||
const [iso27002, setIso27002] = useState<File | null>(null);
|
const [iso27002, setIso27002] = useState<File | null>(null);
|
||||||
const [token, setToken] = useState('');
|
const [token, setToken] = useState('');
|
||||||
const [submitting, setSubmitting] = useState(false);
|
const [submitting, setSubmitting] = useState(false);
|
||||||
|
const [savingAutoMapping, setSavingAutoMapping] = useState(false);
|
||||||
const [error, setError] = useState<string | null>(null);
|
const [error, setError] = useState<string | null>(null);
|
||||||
const load = useCallback(async (poll = false) => {
|
const load = useCallback(async (poll = false) => {
|
||||||
try { setStatus(await (poll ? statusPollApi : api).getIsoCorpusStatus()); setError(null); }
|
try { setStatus(await (poll ? statusPollApi : api).getIsoCorpusStatus()); setError(null); }
|
||||||
|
|
@ -50,6 +55,13 @@ function IsoCorpusLibraryPanel() {
|
||||||
finally { setSubmitting(false); }
|
finally { setSubmitting(false); }
|
||||||
};
|
};
|
||||||
|
|
||||||
|
const toggleAutoMapping = async (enabled: boolean) => {
|
||||||
|
setSavingAutoMapping(true); setError(null);
|
||||||
|
try { setStatus(await api.setIsoAutoMapping(enabled)); }
|
||||||
|
catch (reason) { setError(reason instanceof Error ? reason.message : 'Automatic ISO mapping could not be changed.'); }
|
||||||
|
finally { setSavingAutoMapping(false); }
|
||||||
|
};
|
||||||
|
|
||||||
const running = status?.job?.status === 'running';
|
const running = status?.job?.status === 'running';
|
||||||
return <details className="tenant-scan-history baseline-catalog iso-corpus-catalog">
|
return <details className="tenant-scan-history baseline-catalog iso-corpus-catalog">
|
||||||
<summary className="tenant-section-bar">
|
<summary className="tenant-section-bar">
|
||||||
|
|
@ -71,6 +83,12 @@ function IsoCorpusLibraryPanel() {
|
||||||
{status?.job?.status === 'failed' && <div className="datahub-alert" role="alert">{status.job.error}</div>}
|
{status?.job?.status === 'failed' && <div className="datahub-alert" role="alert">{status.job.error}</div>}
|
||||||
{status?.job?.status === 'succeeded' && <div className="tenant-save-feedback success" role="status"><strong>ISO corpus indexed and active.</strong></div>}
|
{status?.job?.status === 'succeeded' && <div className="tenant-save-feedback success" role="status"><strong>ISO corpus indexed and active.</strong></div>}
|
||||||
{!status?.configured && status?.disabledReason && <div className="datahub-alert" role="alert">{status.disabledReason}</div>}
|
{!status?.configured && status?.disabledReason && <div className="datahub-alert" role="alert">{status.disabledReason}</div>}
|
||||||
|
{status?.autoMapping && <label className="iso-auto-mapping-toggle">
|
||||||
|
<input type="checkbox" checked={status.autoMapping.enabled} disabled={savingAutoMapping}
|
||||||
|
onChange={event => void toggleAutoMapping(event.target.checked)}/>
|
||||||
|
<span>Map new scans automatically</span>
|
||||||
|
</label>}
|
||||||
|
{status?.autoMapping && <p className="iso-corpus-note">After each upload, deviations of rules without an ISO mapping are mapped like <strong>Map all ISO controls</strong>. Rules mapped before are reused.</p>}
|
||||||
<form className="iso-corpus-form" onSubmit={event => void submit(event)}>
|
<form className="iso-corpus-form" onSubmit={event => void submit(event)}>
|
||||||
<label><span>DIN EN ISO/IEC 27001:2024-01 PDF</span><input className="input" type="file" accept=".pdf,application/pdf" required disabled={submitting || running} onChange={event => setIso27001(event.target.files?.[0] ?? null)}/></label>
|
<label><span>DIN EN ISO/IEC 27001:2024-01 PDF</span><input className="input" type="file" accept=".pdf,application/pdf" required disabled={submitting || running} onChange={event => setIso27001(event.target.files?.[0] ?? null)}/></label>
|
||||||
<label><span>DIN EN ISO/IEC 27002:2024-01 PDF</span><input className="input" type="file" accept=".pdf,application/pdf" required disabled={submitting || running} onChange={event => setIso27002(event.target.files?.[0] ?? null)}/></label>
|
<label><span>DIN EN ISO/IEC 27002:2024-01 PDF</span><input className="input" type="file" accept=".pdf,application/pdf" required disabled={submitting || running} onChange={event => setIso27002(event.target.files?.[0] ?? null)}/></label>
|
||||||
|
|
|
||||||
|
|
@ -0,0 +1,21 @@
|
||||||
|
import { AssessorApiClient } from '../../api/assessor-api-client';
|
||||||
|
import { readEnginePreferences, subscribeEnginePreferences } from '../../app/engine-preferences';
|
||||||
|
|
||||||
|
let unlink: (() => void) | undefined;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Switching the Compliance Engine off also switches off the server-side automatic ISO mapping, in whichever
|
||||||
|
* view the Engines popup is used. Switching it on again leaves automatic mapping off until the Data Hub
|
||||||
|
* switch is used. Batches that already started are not cancelled.
|
||||||
|
*/
|
||||||
|
export function linkAutoMappingToEngineSwitch(api: Pick<AssessorApiClient, 'setIsoAutoMapping'> = new AssessorApiClient()): () => void {
|
||||||
|
unlink?.();
|
||||||
|
let active = readEnginePreferences().compliance;
|
||||||
|
const unsubscribe = subscribeEnginePreferences(() => {
|
||||||
|
const next = readEnginePreferences().compliance;
|
||||||
|
if (active && !next) void Promise.resolve().then(() => api.setIsoAutoMapping(false)).catch(() => undefined);
|
||||||
|
active = next;
|
||||||
|
});
|
||||||
|
unlink = () => { unsubscribe(); unlink = undefined; };
|
||||||
|
return unlink;
|
||||||
|
}
|
||||||
|
|
@ -55,3 +55,6 @@ button.iso-controls__theme:hover .iso-controls__theme-name { text-decoration: un
|
||||||
.regulatory-earlier-controls { margin-top: 16px; }
|
.regulatory-earlier-controls { margin-top: 16px; }
|
||||||
.regulatory-earlier-controls > h4 { margin-bottom: 4px; }
|
.regulatory-earlier-controls > h4 { margin-bottom: 4px; }
|
||||||
.regulatory-earlier-controls > p { color: var(--text-muted); }
|
.regulatory-earlier-controls > p { color: var(--text-muted); }
|
||||||
|
|
||||||
|
/* Data Hub ISO tile: switch for automatic mapping of newly uploaded scans. */
|
||||||
|
.iso-auto-mapping-toggle { display: flex; align-items: center; gap: var(--space-2); color: var(--text-primary); font-size: var(--type-table-body-size); }
|
||||||
|
|
|
||||||
|
|
@ -9,7 +9,7 @@ const components = readFileSync(new URL('../../src/styles/components.css', impor
|
||||||
|
|
||||||
describe('ASSESSOR version in the brand header', () => {
|
describe('ASSESSOR version in the brand header', () => {
|
||||||
it('shows the centrally maintained product version and opens About from the brand', () => {
|
it('shows the centrally maintained product version and opens About from the brand', () => {
|
||||||
expect(ASSESSOR_PRODUCT_VERSION).toBe('2.19.1');
|
expect(ASSESSOR_PRODUCT_VERSION).toBe('2.20.0');
|
||||||
expect(app).toContain("'db-structure': 'Data Flow'");
|
expect(app).toContain("'db-structure': 'Data Flow'");
|
||||||
expect(app).toContain("['db-structure', 'Data Flow']");
|
expect(app).toContain("['db-structure', 'Data Flow']");
|
||||||
expect(app).not.toContain("'DB Structure'");
|
expect(app).not.toContain("'DB Structure'");
|
||||||
|
|
|
||||||
|
|
@ -0,0 +1,59 @@
|
||||||
|
// @vitest-environment jsdom
|
||||||
|
import { act } from 'react';
|
||||||
|
import { createRoot, type Root } from 'react-dom/client';
|
||||||
|
import { afterEach, beforeEach, expect, it, vi } from 'vitest';
|
||||||
|
import { AssessorApiClient } from '../../src/api/assessor-api-client';
|
||||||
|
import { ENGINE_PREFERENCES_KEY, resetEnginePreferencesForTests } from '../../src/app/engine-preferences';
|
||||||
|
import { EnginesControl } from '../../src/components/EnginesControl';
|
||||||
|
// Importing the Data Hub ISO tile installs the link, as it does when the app loads.
|
||||||
|
import '../../src/compliance/ui/IsoCorpusLibrary';
|
||||||
|
|
||||||
|
let host: HTMLDivElement;
|
||||||
|
let root: Root;
|
||||||
|
|
||||||
|
beforeEach(() => {
|
||||||
|
vi.stubGlobal('IS_REACT_ACT_ENVIRONMENT', true);
|
||||||
|
window.localStorage.removeItem(ENGINE_PREFERENCES_KEY);
|
||||||
|
resetEnginePreferencesForTests();
|
||||||
|
host = document.createElement('div');
|
||||||
|
document.body.append(host);
|
||||||
|
root = createRoot(host);
|
||||||
|
});
|
||||||
|
|
||||||
|
afterEach(async () => {
|
||||||
|
await act(async () => root.unmount());
|
||||||
|
host.remove();
|
||||||
|
window.localStorage.removeItem(ENGINE_PREFERENCES_KEY);
|
||||||
|
resetEnginePreferencesForTests();
|
||||||
|
vi.restoreAllMocks();
|
||||||
|
vi.unstubAllGlobals();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('switches automatic ISO mapping off when the Compliance Engine is unticked in the Engines popup, and not back on', async () => {
|
||||||
|
const setAutoMapping = vi.spyOn(AssessorApiClient.prototype, 'setIsoAutoMapping')
|
||||||
|
.mockResolvedValue({ version: 1, configured: true, active: null, job: null, autoMapping: { enabled: false } });
|
||||||
|
await act(async () => root.render(<EnginesControl/>));
|
||||||
|
await act(async () => host.querySelector<HTMLButtonElement>('.engines-button')!.click());
|
||||||
|
const compliance = () => [...document.querySelectorAll<HTMLLabelElement>('.engines-option')]
|
||||||
|
.find(option => option.textContent?.includes('Compliance Engine'))!.querySelector('input')!;
|
||||||
|
|
||||||
|
await act(async () => { compliance().click(); await Promise.resolve(); });
|
||||||
|
expect(compliance().checked).toBe(false);
|
||||||
|
expect(setAutoMapping).toHaveBeenCalledTimes(1);
|
||||||
|
expect(setAutoMapping).toHaveBeenCalledWith(false);
|
||||||
|
|
||||||
|
await act(async () => { compliance().click(); await Promise.resolve(); });
|
||||||
|
expect(compliance().checked).toBe(true);
|
||||||
|
expect(setAutoMapping).toHaveBeenCalledTimes(1);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('leaves automatic mapping alone when another engine is switched off', async () => {
|
||||||
|
const setAutoMapping = vi.spyOn(AssessorApiClient.prototype, 'setIsoAutoMapping')
|
||||||
|
.mockResolvedValue({ version: 1, configured: true, active: null, job: null });
|
||||||
|
await act(async () => root.render(<EnginesControl/>));
|
||||||
|
await act(async () => host.querySelector<HTMLButtonElement>('.engines-button')!.click());
|
||||||
|
const context = [...document.querySelectorAll<HTMLLabelElement>('.engines-option')]
|
||||||
|
.find(option => option.textContent?.includes('Context Engine'))!.querySelector('input')!;
|
||||||
|
await act(async () => { context.click(); await Promise.resolve(); });
|
||||||
|
expect(setAutoMapping).not.toHaveBeenCalled();
|
||||||
|
});
|
||||||
|
|
@ -70,3 +70,30 @@ it('clears a transient status polling error once a later poll succeeds', async (
|
||||||
vi.useRealTimers();
|
vi.useRealTimers();
|
||||||
}
|
}
|
||||||
});
|
});
|
||||||
|
|
||||||
|
it('switches automatic mapping of new scans on and off from the Data Hub ISO tile', async () => {
|
||||||
|
vi.stubGlobal('IS_REACT_ACT_ENVIRONMENT', true);
|
||||||
|
const off: RegulatoryCorpusStatus = { version: 1, configured: true, active: null, job: null, autoMapping: { enabled: false } };
|
||||||
|
vi.spyOn(AssessorApiClient.prototype, 'getIsoCorpusStatus').mockResolvedValue(off);
|
||||||
|
const toggle = vi.spyOn(AssessorApiClient.prototype, 'setIsoAutoMapping')
|
||||||
|
.mockImplementation(async enabled => ({ ...off, autoMapping: { enabled } }));
|
||||||
|
const container = document.createElement('div');
|
||||||
|
document.body.append(container);
|
||||||
|
const root = createRoot(container);
|
||||||
|
|
||||||
|
try {
|
||||||
|
await act(async () => root.render(<IsoCorpusLibrary/>));
|
||||||
|
const checkbox = () => container.querySelector<HTMLInputElement>('.iso-auto-mapping-toggle input[type="checkbox"]')!;
|
||||||
|
expect(container.querySelector('.iso-auto-mapping-toggle')?.textContent).toBe('Map new scans automatically');
|
||||||
|
expect(checkbox().checked).toBe(false);
|
||||||
|
await act(async () => checkbox().click());
|
||||||
|
expect(toggle).toHaveBeenLastCalledWith(true);
|
||||||
|
expect(checkbox().checked).toBe(true);
|
||||||
|
await act(async () => checkbox().click());
|
||||||
|
expect(toggle).toHaveBeenLastCalledWith(false);
|
||||||
|
expect(checkbox().checked).toBe(false);
|
||||||
|
} finally {
|
||||||
|
await act(async () => root.unmount());
|
||||||
|
container.remove();
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
|
||||||
|
|
@ -0,0 +1,116 @@
|
||||||
|
import { mkdtemp, readFile, rm } from 'node:fs/promises';
|
||||||
|
import { tmpdir } from 'node:os';
|
||||||
|
import { join } from 'node:path';
|
||||||
|
import { afterEach, beforeEach, expect, it, vi } from 'vitest';
|
||||||
|
import { RegulatoryAutoMapping } from '../../server/compliance/regulatory-auto-mapping';
|
||||||
|
import type { ScanRun } from '../../src/domain/canonical-model';
|
||||||
|
import type { UnifiedResultQuery } from '../../src/domain/unified-result';
|
||||||
|
|
||||||
|
let root: string;
|
||||||
|
beforeEach(async () => { root = await mkdtemp(join(tmpdir(), 'assessor-auto-mapping-')); });
|
||||||
|
afterEach(async () => { await rm(root, { recursive: true, force: true }); });
|
||||||
|
|
||||||
|
function run(id: string, extra: Partial<ScanRun> = {}): ScanRun {
|
||||||
|
return { id, assessmentId: 'synthetic-assessment', sequence: 1, label: id, tool: { id: 'synthetic', name: 'Synthetic' } as ScanRun['tool'],
|
||||||
|
importedAt: '2026-10-08T00:00:00Z', ...extra };
|
||||||
|
}
|
||||||
|
|
||||||
|
function harness(initialRuns: ScanRun[], generationEnabled = true) {
|
||||||
|
const runs = [...initialRuns];
|
||||||
|
const gpoRows: string[] = [];
|
||||||
|
const read = {
|
||||||
|
listProjects: vi.fn(async () => [{ id: 'synthetic-project' }] as never),
|
||||||
|
listAssessments: vi.fn(async () => [{ id: 'synthetic-assessment' }] as never),
|
||||||
|
listProjectScanRuns: vi.fn(async () => runs),
|
||||||
|
queryUnifiedResults: vi.fn(async (query: UnifiedResultQuery) => {
|
||||||
|
const items = gpoRows.slice(query.offset ?? 0, (query.offset ?? 0) + (query.limit ?? gpoRows.length))
|
||||||
|
.map(resultProjectionId => ({ resultProjectionId }));
|
||||||
|
return { total: gpoRows.length, items } as never;
|
||||||
|
})
|
||||||
|
};
|
||||||
|
const mapping = { generationEnabled, startBatch: vi.fn(async (_projectId: string, _request: unknown) => ({}) as never) };
|
||||||
|
const auto = new RegulatoryAutoMapping({ settingsPath: join(root, 'auto-mapping.json'), read, mapping, pollIntervalMs: 3_600_000 });
|
||||||
|
return { runs, gpoRows, read, mapping, auto };
|
||||||
|
}
|
||||||
|
|
||||||
|
it('stays off by default and reads nothing from the Core', async () => {
|
||||||
|
const { auto, read, mapping } = harness([run('existing')]);
|
||||||
|
await auto.start();
|
||||||
|
await auto.poll();
|
||||||
|
expect(auto.isEnabled).toBe(false);
|
||||||
|
expect(read.listProjects).not.toHaveBeenCalled();
|
||||||
|
expect(mapping.startBatch).not.toHaveBeenCalled();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('maps the deviations of a scan uploaded after switching on, but not the scans that existed before', async () => {
|
||||||
|
const { auto, runs, mapping } = harness([run('existing')]);
|
||||||
|
await auto.setEnabled(true);
|
||||||
|
expect(mapping.startBatch).not.toHaveBeenCalled();
|
||||||
|
|
||||||
|
runs.push(run('new-scan'));
|
||||||
|
await auto.poll();
|
||||||
|
expect(mapping.startBatch).toHaveBeenCalledTimes(1);
|
||||||
|
expect(mapping.startBatch).toHaveBeenCalledWith('synthetic-project', {
|
||||||
|
idempotencyKey: 'auto:new-scan', assessmentIds: ['synthetic-assessment'], scanRunIds: ['new-scan'], filters: { deviation: true }
|
||||||
|
});
|
||||||
|
|
||||||
|
await auto.poll();
|
||||||
|
expect(mapping.startBatch).toHaveBeenCalledTimes(1);
|
||||||
|
await auto.stop();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('maps the GPO deviations of a timeslot when a GPO import adds new rows, reading only Result DB pages', async () => {
|
||||||
|
const h = harness([]);
|
||||||
|
h.gpoRows.push('policy-evaluation-comparison:existing');
|
||||||
|
await h.auto.setEnabled(true);
|
||||||
|
expect(h.mapping.startBatch).not.toHaveBeenCalled();
|
||||||
|
expect(h.read.queryUnifiedResults.mock.calls.every(([query]) => query.audience === 'result-db')).toBe(true);
|
||||||
|
|
||||||
|
h.gpoRows.push('policy-evaluation-comparison:new-a', 'policy-evaluation-comparison:new-b');
|
||||||
|
await h.auto.poll();
|
||||||
|
expect(h.mapping.startBatch).toHaveBeenCalledTimes(1);
|
||||||
|
expect(h.mapping.startBatch.mock.calls[0][1]).toMatchObject({
|
||||||
|
assessmentIds: ['synthetic-assessment'], scanRunIds: [], filters: { sourceKind: ['policy-evaluation'], deviation: true }
|
||||||
|
});
|
||||||
|
|
||||||
|
await h.auto.poll();
|
||||||
|
expect(h.mapping.startBatch).toHaveBeenCalledTimes(1);
|
||||||
|
// A later GPO import in the same timeslot gets its own batch, not the earlier one by idempotency key.
|
||||||
|
h.gpoRows.push('policy-evaluation-comparison:new-c');
|
||||||
|
await h.auto.poll();
|
||||||
|
expect(h.mapping.startBatch).toHaveBeenCalledTimes(2);
|
||||||
|
const keys = h.mapping.startBatch.mock.calls.map(call => (call[1] as { idempotencyKey: string }).idempotencyKey);
|
||||||
|
expect(new Set(keys).size).toBe(2);
|
||||||
|
await h.auto.stop();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('ignores superseded runs and keeps a new scan pending until mapping is available', async () => {
|
||||||
|
const h = harness([], false);
|
||||||
|
await h.auto.setEnabled(true);
|
||||||
|
h.runs.push(run('replaced', { supersededByScanRunId: 'replacement' }), run('replacement'));
|
||||||
|
await h.auto.poll();
|
||||||
|
expect(h.mapping.startBatch).not.toHaveBeenCalled();
|
||||||
|
|
||||||
|
h.mapping.generationEnabled = true;
|
||||||
|
await h.auto.poll();
|
||||||
|
expect(h.mapping.startBatch).toHaveBeenCalledTimes(1);
|
||||||
|
expect(h.mapping.startBatch.mock.calls[0][1]).toMatchObject({ scanRunIds: ['replacement'] });
|
||||||
|
await h.auto.stop();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('persists the switch in its own settings file and stops watching when switched off', async () => {
|
||||||
|
const first = harness([]);
|
||||||
|
await first.auto.setEnabled(true);
|
||||||
|
expect(JSON.parse(await readFile(join(root, 'auto-mapping.json'), 'utf8'))).toEqual({ version: 1, enabled: true });
|
||||||
|
await first.auto.stop();
|
||||||
|
|
||||||
|
const restarted = harness([]);
|
||||||
|
await restarted.auto.start();
|
||||||
|
expect(restarted.auto.isEnabled).toBe(true);
|
||||||
|
await restarted.auto.setEnabled(false);
|
||||||
|
restarted.runs.push(run('after-off'));
|
||||||
|
restarted.gpoRows.push('policy-evaluation-comparison:after-off');
|
||||||
|
await restarted.auto.poll();
|
||||||
|
expect(restarted.mapping.startBatch).not.toHaveBeenCalled();
|
||||||
|
expect(JSON.parse(await readFile(join(root, 'auto-mapping.json'), 'utf8'))).toEqual({ version: 1, enabled: false });
|
||||||
|
});
|
||||||
|
|
@ -188,3 +188,33 @@ describe('operator ISO corpus upload from Data Hub', () => {
|
||||||
await expect(stat(temporary)).rejects.toMatchObject({ code: 'ENOENT' });
|
await expect(stat(temporary)).rejects.toMatchObject({ code: 'ENOENT' });
|
||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
|
describe('automatic ISO mapping switch', () => {
|
||||||
|
it('reports and changes the switch through the ISO corpus route without the upload token', async () => {
|
||||||
|
const directory = await mkdtemp(join(tmpdir(), 'assessor-iso-auto-'));
|
||||||
|
await writeFile(join(directory, 'index.html'), '<main>synthetic app</main>');
|
||||||
|
let enabled = false;
|
||||||
|
const autoMapping = { get isEnabled() { return enabled; }, setEnabled: vi.fn(async (value: boolean) => { enabled = value; }) };
|
||||||
|
const service = new RegulatoryCorpusUploadService({ root: join(directory, 'artifacts', 'regulatory-corpora'),
|
||||||
|
mapping: {} as RegulatoryMappingService, autoMapping });
|
||||||
|
await service.initialize();
|
||||||
|
const server = createAssessorHttpServer({ repository: new LocalAssessorRepository(), staticRoot: directory,
|
||||||
|
regulatoryCorpusUploadService: service, isoUploadToken: token });
|
||||||
|
await new Promise<void>((resolve, reject) => { server.once('error', reject); server.listen(0, '127.0.0.1', resolve); });
|
||||||
|
const baseUrl = `http://127.0.0.1:${(server.address() as AddressInfo).port}`;
|
||||||
|
const put = (body: unknown) => fetch(`${baseUrl}${path}/auto-mapping`, { method: 'PUT',
|
||||||
|
headers: { 'Content-Type': 'application/json', 'X-Assessor-Request': '1', Origin: baseUrl }, body: JSON.stringify(body) });
|
||||||
|
try {
|
||||||
|
expect((await (await fetch(`${baseUrl}${path}`)).json()).autoMapping).toEqual({ enabled: false });
|
||||||
|
const switched = await put({ enabled: true });
|
||||||
|
expect(switched.status).toBe(200);
|
||||||
|
expect((await switched.json()).autoMapping).toEqual({ enabled: true });
|
||||||
|
expect(autoMapping.setEnabled).toHaveBeenCalledWith(true);
|
||||||
|
expect((await put({ enabled: 'yes' })).status).toBe(400);
|
||||||
|
expect(autoMapping.setEnabled).toHaveBeenCalledTimes(1);
|
||||||
|
} finally {
|
||||||
|
await new Promise<void>((resolve, reject) => server.close(error => error ? reject(error) : resolve()));
|
||||||
|
await rm(directory, { recursive: true, force: true });
|
||||||
|
}
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
|
||||||
|
|
@ -59,7 +59,7 @@ describe('local ASSESSOR HTTP server', () => {
|
||||||
|
|
||||||
const health = await fetch(`${baseUrl}/api/v1/health`);
|
const health = await fetch(`${baseUrl}/api/v1/health`);
|
||||||
expect(await health.json()).toEqual({
|
expect(await health.json()).toEqual({
|
||||||
status: 'ok', application: 'ASSESSOR', productVersion: '2.19.1', apiVersion: 1, repositoryRevision: 0, persistence: 'memory'
|
status: 'ok', application: 'ASSESSOR', productVersion: '2.20.0', apiVersion: 1, repositoryRevision: 0, persistence: 'memory'
|
||||||
});
|
});
|
||||||
expect(health.headers.get('access-control-allow-origin')).toBeNull();
|
expect(health.headers.get('access-control-allow-origin')).toBeNull();
|
||||||
});
|
});
|
||||||
|
|
|
||||||
Loading…
Reference in New Issue