feat(comparison): register the Comparison Engine and relocate the Phase 8 scan-run comparison

Move compareScanRuns, comparison normalization and local comparison rules
unchanged to src/evaluation/scan-run-comparison so src/comparison can become
the Comparison Engine; only import paths change. Register the engine paths,
document it in AGENTS.md and onboarding, extend the boundary test and add the
permanent on/off invariance test. Record the decision and staging in the plan.
This commit is contained in:
Binh 2026-10-13 11:34:10 +02:00
parent f83edc83ac
commit c982f0a0a2
16 changed files with 144 additions and 23 deletions

View File

@ -17,6 +17,7 @@ export const ENGINE_PATHS = [
['Report Engine', ['src/reporting/', 'server/reporting/', 'tests/reporting/', 'tests/server/report-archive.test.ts', ['Report Engine', ['src/reporting/', 'server/reporting/', 'tests/reporting/', 'tests/server/report-archive.test.ts',
'tests/postgres/report-read-only-runtime.test.ts']], 'tests/postgres/report-read-only-runtime.test.ts']],
['Compliance Engine', ['src/compliance/', 'server/compliance/', 'tests/compliance/']], ['Compliance Engine', ['src/compliance/', 'server/compliance/', 'tests/compliance/']],
['Comparison Engine', ['src/comparison/', 'server/comparison/', 'tests/comparison/']],
]; ];
export const CORE_AREAS = [ export const CORE_AREAS = [
@ -30,7 +31,7 @@ export const CORE_AREAS = [
'server/import-executor.ts', 'server/import-worker-client.ts', 'server/import-worker.ts', 'server/import-executor.ts', 'server/import-worker-client.ts', 'server/import-worker.ts',
'server/node-import-runtime-adapters.ts', 'server/artifacts/']], 'server/node-import-runtime-adapters.ts', 'server/artifacts/']],
['Normalization', ['src/domain/']], ['Normalization', ['src/domain/']],
['Evaluation', ['src/evaluation/', 'src/comparison/', 'src/scoring/']], ['Evaluation', ['src/evaluation/', 'src/scoring/']],
['Store', ['src/persistence/', 'server/postgres/', 'server/repository-factory.ts', 'docker-compose.postgres.yml']], ['Store', ['src/persistence/', 'server/postgres/', 'server/repository-factory.ts', 'docker-compose.postgres.yml']],
['Reference Catalogs', ['src/references/', 'server/reference-data/']], ['Reference Catalogs', ['src/references/', 'server/reference-data/']],
['Shared Kernel', ['src/shared/', 'src/vite-env.d.ts']], ['Shared Kernel', ['src/shared/', 'src/vite-env.d.ts']],

File diff suppressed because one or more lines are too long

View File

@ -1,5 +1,21 @@
# ASSESSOR – Umsetzungsplan # ASSESSOR – Umsetzungsplan
## Comparison Engine: decision and staging (planned 2026-10-13, Phase 8 timeslot comparison as an engine)
Status: **steps 1 and 2 done and verified (2026-10-13), steps 3-4 open**. Explicit user decision 2026-10-13: the timeslot comparison of [TIMESLOT_COMPARISON_PLAN.md](TIMESLOT_COMPARISON_PLAN.md) (P1-P6) becomes a separately switchable **Comparison Engine** that owns its comparison logic. Its folders follow the other engines without an `-engine` suffix: `src/comparison/`, `server/comparison/`, `tests/comparison/`. It reads the Core only through a declared read-only port and never writes, overrides or re-derives a Core verdict, score, review, risk acceptance or Result DB row. All contracts, the acceptance matrix and the conservative rules of the timeslot plan stay binding unchanged (object/scope identity, no `absent = resolved` without a declared profile negative-evidence contract, a missing source keeps its open deviations as unverified, score delta only over the shared population, UI is a consumer, never its own matcher).
UI decision 2026-10-13 (mockups A and B in the local, git-ignored `artifacts/compare-engine/`; synthetic Timeslot A): entry is a Client Info tab `Compare` with A/B timeslot selection, change counts, Security Domain and source deltas, each count opening Work; Work receives a `Change` facet and change badges, and the Finding Reader a `Compare` section with A and B values side by side. A retained value change of a still deviating GPO setting is shown neutral (`Value changed`), neither better nor worse. Mockup C (better/worse bridge) is not selected.
Staging and required Core releases:
1. Move the Core Phase 8 scan-run comparison (`comparison-engine.ts`, `comparison-normalization.ts`, `local-mapping-rules.ts` and their tests) from `src/comparison/` to a Core Evaluation location (proposed `src/evaluation/scan-run-comparison/`), updating its Core importers in Store (`src/persistence/local-assessor-repository.ts`, `server/postgres/postgres-assessor-repository.ts`) and Presentation (`src/features/project-import/datahub-scan-analysis.ts`). Without this move, registering `src/comparison/` as an engine path makes Core import an engine and breaks the boundary test. Needs `Governance, Core Tests, Evaluation, Store, Presentation`.
2. Register the engine paths in `.claude/hooks/core-areas.mjs`, the engine section of `AGENTS.md` and `tests/architecture/core-engine-boundary.test.ts`; the unused timeslot code (`timeslot-comparison-engine.ts` and its test) stays in `src/comparison/` and thereby becomes engine code; add the permanent on/off invariance test. Same release as step 1.
3. Engine work P2/P4 (GPO pairing, A/B score basis) inside the engine paths, then the extension points (Engines popup entry, Client Info tab, Work facet, Finding Reader section; `Presentation`) and the backend route (`Platform`).
4. P3 carried-forward obligations need an engine-owned schema: separate approval, verified backup and migration.
Steps 1-2 (Core release Governance, Core Tests, Evaluation, Store, Presentation): the Phase 8 files moved unchanged to `src/evaluation/scan-run-comparison/` (test to `tests/evaluation/scan-run-comparison/`); only import paths changed in the three Core importers and two tests. `assessor_comparisons`, its routes and data are untouched (0 rows). `.claude/hooks/core-areas.mjs` registers `Comparison Engine` and drops `src/comparison/` from Evaluation; `AGENTS.md`, the boundary test and `docs/ONBOARDING.md` name the engine; new permanent invariance test `tests/comparison/comparison-engine-invariance.test.ts` (synthetic, real local repository: a comparison leaves export, revision and models unchanged; one verified fail-to-pass, one confirmed open). Verified: import gate 528/528 (twice); main suite 2,451 passed, 18 optional skipped, 1 failed: the pre-existing `db-structure-layout.test.ts` tile-radius check, unrelated (no CSS or DB Structure file changed). The first gate run hit the documented 5-second timeout of the combined Node import test once; it passed alone (757 ms) and in the rerun, limit unchanged. Both production builds pass (existing chunk-size warning only). No product version change: no visible behaviour or contract changed.
Change log 2026-10-13: recorded the Comparison Engine decision, naming and staging; relocated the Core Phase 8 scan-run comparison and registered the Comparison Engine with its invariance test.
## Security finding reading hierarchy (2026-10-11, Phase 7 Presentation) ## Security finding reading hierarchy (2026-10-11, Phase 7 Presentation)
Current Core release: Presentation, Core Tests, Platform. User correction: Security finding headings use the shared Bold 700 token; the scanner/rule subtitle is removed from these compact rows. The original Description appears immediately after Affected and before Impact, including when its text equals Result. Findings without an Affected field retain Description after Result, or first if Result is absent; no empty source fields are invented. Imported fields, tool identities, verdicts and counts remain unchanged. GPO baseline explanations retain their existing dedicated presentation. Permanent synthetic Light/Dark disclosure interaction verifies the exact order, heading weight, absent scanner subtitle and unchanged source data. All six Security UI checks, 17 UI/API version and Core-boundary checks, and design-language check pass. Patch decision: 2.24.2 for this backward-compatible correction, with shared package/lock and UI/API version contracts synchronized. All 504 mandatory import compatibility checks and both production builds pass (existing client chunk-size warning only); local-only check passes. Actual Dummy browser acceptance in Light/Dark confirms Bold 700 finding headings, no scanner subtitles, and Result → Affected → Description → Impact. Light restored and temporary test tab closed. No full-suite, phase-completion or backend-restart claim. Concurrent work preserved. Current Core release: Presentation, Core Tests, Platform. User correction: Security finding headings use the shared Bold 700 token; the scanner/rule subtitle is removed from these compact rows. The original Description appears immediately after Affected and before Impact, including when its text equals Result. Findings without an Affected field retain Description after Result, or first if Result is absent; no empty source fields are invented. Imported fields, tool identities, verdicts and counts remain unchanged. GPO baseline explanations retain their existing dedicated presentation. Permanent synthetic Light/Dark disclosure interaction verifies the exact order, heading weight, absent scanner subtitle and unchanged source data. All six Security UI checks, 17 UI/API version and Core-boundary checks, and design-language check pass. Patch decision: 2.24.2 for this backward-compatible correction, with shared package/lock and UI/API version contracts synchronized. All 504 mandatory import compatibility checks and both production builds pass (existing client chunk-size warning only); local-only check passes. Actual Dummy browser acceptance in Light/Dark confirms Bold 700 finding headings, no scanner subtitles, and Result → Affected → Description → Impact. Light restored and temporary test tab closed. No full-suite, phase-completion or backend-restart claim. Concurrent work preserved.

View File

@ -76,7 +76,7 @@ GPO backup.
| **Review / risk acceptance** | Client-owned work state. Stored separately and kept across reimports. | | **Review / risk acceptance** | Client-owned work state. Stored separately and kept across reimports. |
| **Result DB** | The stored, privacy-checked projection of all results. The only dataset that may be shared or fed to AI. | | **Result DB** | The stored, privacy-checked projection of all results. The only dataset that may be shared or fed to AI. |
| **Core / Core area** | Everything ASSESSOR does today. It is split into areas such as Ingest or Evaluation. Changing an area needs explicit consent (section 7). | | **Core / Core area** | Everything ASSESSOR does today. It is split into areas such as Ingest or Evaluation. Changing an area needs explicit consent (section 7). |
| **Engine** | An optional capability that reads the Core and never writes to it: Context, Report, Compliance. | | **Engine** | An optional capability that reads the Core and never writes to it: Context, Report, Compliance, Comparison. |
| **Extension point** | A place in a Core view where an engine shows its content, for example the Finding Reader `Context` section. | | **Extension point** | A place in a Core view where an engine shows its content, for example the Finding Reader `Context` section. |
--- ---
@ -235,7 +235,7 @@ switched off. It is divided into **Core areas**. The exact path map is in
|---|---|---| |---|---|---|
| Ingest | `src/importers`, `src/recognition`, `src/profiles`, `src/gpo`, `server/gpo`, import workers | getting data in | | Ingest | `src/importers`, `src/recognition`, `src/profiles`, `src/gpo`, `server/gpo`, import workers | getting data in |
| Normalization | `src/domain` | canonical model, finding identity, status/severity | | Normalization | `src/domain` | canonical model, finding identity, status/severity |
| Evaluation | `src/evaluation`, `src/comparison`, `src/scoring` | verdicts, comparison, scores, security domains | | Evaluation | `src/evaluation` (incl. `scan-run-comparison`), `src/scoring` | verdicts, comparison, scores, security domains |
| Store | `src/persistence`, `server/postgres` | repositories, PostgreSQL, migrations | | Store | `src/persistence`, `server/postgres` | repositories, PostgreSQL, migrations |
| Client Records | reviews, risk acceptances, review transfer on reimport | client-owned state | | Client Records | reviews, risk acceptances, review transfer on reimport | client-owned state |
| Disclosure | `src/domain/result-db-disclosure.ts`, `src/exports` | Result DB policy, exports | | Disclosure | `src/domain/result-db-disclosure.ts`, `src/exports` | Result DB policy, exports |
@ -259,6 +259,7 @@ Documentation under `docs/` is not gated.
| **Context Engine** | `src/assessment-context/`, `src/features/context-layer/` | Gives every deviation a context cluster, in a fixed order of tiers: *catalogued* (catalogue rule, CIS section, declared ATT&CK technique), *recognized* (distinctive terms, technical anchors) or *estimated* (local TF-IDF similarity, category fallback). Deterministic, no AI. Shown as badges, the `Context Cluster` facet, the Finding Reader section and the Client Info heatmap. A context statement says "favours", never "exploitable". | none (in memory) | | **Context Engine** | `src/assessment-context/`, `src/features/context-layer/` | Gives every deviation a context cluster, in a fixed order of tiers: *catalogued* (catalogue rule, CIS section, declared ATT&CK technique), *recognized* (distinctive terms, technical anchors) or *estimated* (local TF-IDF similarity, category fallback). Deterministic, no AI. Shown as badges, the `Context Cluster` facet, the Finding Reader section and the Client Info heatmap. A context statement says "favours", never "exploitable". | none (in memory) |
| **Report Engine** | `src/reporting/`, `server/reporting/` | Builds the Management Report (German, offline, no LLM, requires published Context results) and the Technical Report from one shared projection. | own file archive, separate read-only DB pool | | **Report Engine** | `src/reporting/`, `server/reporting/` | Builds the Management Report (German, offline, no LLM, requires published Context results) and the Technical Report from one shared projection. | own file archive, separate read-only DB pool |
| **Compliance Engine** | `src/compliance/`, `server/compliance/` | Maps findings to regulatory controls; ISO/IEC 27001/27002 is the first framework. Owns **all AI integration**: ISO mapping and the AI overview. | own tables in PostgreSQL, Qdrant index | | **Compliance Engine** | `src/compliance/`, `server/compliance/` | Maps findings to regulatory controls; ISO/IEC 27001/27002 is the first framework. Owns **all AI integration**: ISO mapping and the AI overview. | own tables in PostgreSQL, Qdrant index |
| **Comparison Engine** | `src/comparison/`, `server/comparison/` | Compares two timeslots A and B of one client: verified resolved, still open, unverified, new, regressed and neutral value changes (see `docs/TIMESLOT_COMPARISON_PLAN.md`). In development; no UI yet. | none yet (own schema planned) |
**Engine rules**: **Engine rules**:

View File

@ -4,6 +4,8 @@ Stand: 2026-09-11. Zugehörigkeit: Phase 8, mit UI aus Phase 7, Persistenz/API a
Status: **in Arbeit, nicht fachlich abgenommen**. P0 ist abgeschlossen. P1 besitzt den additiven Domänenvertrag und verpflichtende profilgebundene Scope-/Zeit-/Evidenzdeklarationen; Persistenz/API und Übergangsvertrag bleiben offen. P2 ist begonnen: `compareProjectTimeslotEvidence` paart echte Scannerläufe je Werkzeug und bewertet Processing, Scope, Reportzeit, Diagnosen sowie Fail-zu-Pass konservativ. Die bisherige Engine und Oberfläche verwenden diesen Pfad noch nicht. P3-P6 sind offen. Der aktuelle Implementierungsschnitt führt keine Migration, Reimporte, Datenmutation oder Aktivierung einer Vergleichsoberfläche aus. Der historische Phase-8-Abschluss vom 2026-07-17 bleibt dokumentiert, ist aber keine Abnahme dieses Arbeitspakets. Status: **in Arbeit, nicht fachlich abgenommen**. P0 ist abgeschlossen. P1 besitzt den additiven Domänenvertrag und verpflichtende profilgebundene Scope-/Zeit-/Evidenzdeklarationen; Persistenz/API und Übergangsvertrag bleiben offen. P2 ist begonnen: `compareProjectTimeslotEvidence` paart echte Scannerläufe je Werkzeug und bewertet Processing, Scope, Reportzeit, Diagnosen sowie Fail-zu-Pass konservativ. Die bisherige Engine und Oberfläche verwenden diesen Pfad noch nicht. P3-P6 sind offen. Der aktuelle Implementierungsschnitt führt keine Migration, Reimporte, Datenmutation oder Aktivierung einer Vergleichsoberfläche aus. Der historische Phase-8-Abschluss vom 2026-07-17 bleibt dokumentiert, ist aber keine Abnahme dieses Arbeitspakets.
Ausdrückliche Nutzerentscheidung 2026-10-13: P1-P6 werden als eigenständig abschaltbare **Comparison Engine** umgesetzt (`src/comparison/`, `server/comparison/`, `tests/comparison/`); der Core-Phase-8-Scanlaufvergleich zieht dafür in einen Core-Evaluation-Ort um. Alle Verträge und die Abnahmematrix dieses Plans bleiben unverändert verbindlich. Stufen und Freigaben: [IMPLEMENTATION_PLAN.md](IMPLEMENTATION_PLAN.md), Abschnitt „Comparison Engine: decision and staging“.
Dieser Plan präzisiert das Arbeitspaket „faire Vergleiche bei fehlenden Folgequellen“ in [IMPLEMENTATION_PLAN.md](IMPLEMENTATION_PLAN.md). Für die künftige Umsetzung ersetzt er ausdrücklich die ältere Schlussfolgerung „abwesend = behoben“ und die pauschale Ablösung offener Verpflichtungen durch einen neuen Lauf derselben Quelle. Historische Ergebnisse und Originaldaten werden dadurch nicht rückwirkend geändert. Dieser Plan präzisiert das Arbeitspaket „faire Vergleiche bei fehlenden Folgequellen“ in [IMPLEMENTATION_PLAN.md](IMPLEMENTATION_PLAN.md). Für die künftige Umsetzung ersetzt er ausdrücklich die ältere Schlussfolgerung „abwesend = behoben“ und die pauschale Ablösung offener Verpflichtungen durch einen neuen Lauf derselben Quelle. Historische Ergebnisse und Originaldaten werden dadurch nicht rückwirkend geändert.
## 1. Ziel und Grenzen ## 1. Ziel und Grenzen
@ -22,7 +24,7 @@ Codeprüfung vom 2026-09-11, keine neue Laufzeitabnahme:
- `compareProjectTimeslots` in [datahub-scan-analysis.ts](../src/features/project-import/datahub-scan-analysis.ts) besitzt derzeit keinen Produktaufrufer. Mehrfachauswahl/Öffnen ist kein expliziter A/B-Bericht. - `compareProjectTimeslots` in [datahub-scan-analysis.ts](../src/features/project-import/datahub-scan-analysis.ts) besitzt derzeit keinen Produktaufrufer. Mehrfachauswahl/Öffnen ist kein expliziter A/B-Bericht.
- Die Timeslot-Projektion ersetzt beide Laufwerkzeuge durch `multiple`, lässt `importFiles` aber an Original-Lauf-IDs. Dadurch kann `sameTool` fälschlich gelten und die dateigebundene Kompatibilitätsprüfung ausfallen. - Die Timeslot-Projektion ersetzt beide Laufwerkzeuge durch `multiple`, lässt `importFiles` aber an Original-Lauf-IDs. Dadurch kann `sameTool` fälschlich gelten und die dateigebundene Kompatibilitätsprüfung ausfallen.
- Die automatischen Matchingstufen der [comparison-engine.ts](../src/comparison/comparison-engine.ts) erzwingen keine objektgenaue Identität. Eine gemeinsame Regeldefinition ist kein Nachweis desselben betroffenen Objekts. Die Zwei-Lauf-Projektion entfernt außerdem frühere Beobachtungen, die zur Verifikation eines Wiederauftretens gebraucht werden. - Die automatischen Matchingstufen der [comparison-engine.ts](../src/evaluation/scan-run-comparison/comparison-engine.ts) erzwingen keine objektgenaue Identität. Eine gemeinsame Regeldefinition ist kein Nachweis desselben betroffenen Objekts. Die Zwei-Lauf-Projektion entfernt außerdem frühere Beobachtungen, die zur Verifikation eines Wiederauftretens gebraucht werden.
- Abwesende Vorkommen können `resolved` werden, während zugeordnete Fail-zu-Pass-Ergebnisse `persistent` mit Statusänderung bleiben. Die lokale `isOpen`-Prüfung verwechselt fachliche Abweichungen mit technischen beziehungsweise ungeprüften Zuständen. - Abwesende Vorkommen können `resolved` werden, während zugeordnete Fail-zu-Pass-Ergebnisse `persistent` mit Statusänderung bleiben. Die lokale `isOpen`-Prüfung verwechselt fachliche Abweichungen mit technischen beziehungsweise ungeprüften Zuständen.
- Die Score-Paarbildung prüft viele Kompatibilitätsbedingungen. Ihr `evaluationScopeKey` enthält jedoch keine Asset-Identität; eine unveränderte Regelanzahl kann einen Objektwechsel verdecken. - Die Score-Paarbildung prüft viele Kompatibilitätsbedingungen. Ihr `evaluationScopeKey` enthält jedoch keine Asset-Identität; eine unveränderte Regelanzahl kann einen Objektwechsel verdecken.
- `awaitingRevalidation` verwendet schwächere Bedingungen als die Score-Paarbildung. Zusätzlich überschreibt `lastKnown` im [Workspace-Lader](../src/features/project-import/load-canonical-project-workspace.ts) ältere Quellstände, ohne jede offene Verpflichtung zu erhalten. Das kann Verpflichtungen aus der Anzeige verdrängen, löscht aber keine gespeicherten Originalfindings. - `awaitingRevalidation` verwendet schwächere Bedingungen als die Score-Paarbildung. Zusätzlich überschreibt `lastKnown` im [Workspace-Lader](../src/features/project-import/load-canonical-project-workspace.ts) ältere Quellstände, ohne jede offene Verpflichtung zu erhalten. Das kann Verpflichtungen aus der Anzeige verdrängen, löscht aber keine gespeicherten Originalfindings.

View File

@ -79,7 +79,7 @@ import { compareReimportSourceHashes } from '../../src/domain/reimport-source';
import { planReimportReviewTransfer, type ReimportReviewTransferPlan, type ReimportReviewTransferSummary } from '../../src/domain/reimport-review-transfer'; import { planReimportReviewTransfer, type ReimportReviewTransferPlan, type ReimportReviewTransferSummary } from '../../src/domain/reimport-review-transfer';
import { canonicalJson } from '../../src/shared/canonical-json'; import { canonicalJson } from '../../src/shared/canonical-json';
import { RevisionReadCache } from './revision-read-cache'; import { RevisionReadCache } from './revision-read-cache';
import { validateLocalComparisonRules } from '../../src/comparison/local-mapping-rules'; import { validateLocalComparisonRules } from '../../src/evaluation/scan-run-comparison/local-mapping-rules';
import type { import type {
AssessorRepository, AssessorRepository,
AssessmentDeleteConfirmation, AssessmentDeleteConfirmation,

View File

@ -6,7 +6,7 @@ import type {
FindingOccurrence, FindingOccurrence,
JsonValue, JsonValue,
ScanRun ScanRun
} from '../domain/canonical-model'; } from '../../domain/canonical-model';
import type { import type {
ComparisonMatch, ComparisonMatch,
ComparisonMatchMethod, ComparisonMatchMethod,
@ -16,11 +16,11 @@ import type {
FindingComparison, FindingComparison,
LocalComparisonRule, LocalComparisonRule,
ScanRunComparison ScanRunComparison
} from '../domain/comparison'; } from '../../domain/comparison';
import { jaccardSimilarity, normalizeComparisonText, similarityTokens, stableJson } from './comparison-normalization'; import { jaccardSimilarity, normalizeComparisonText, similarityTokens, stableJson } from './comparison-normalization';
import { localRuleConnects, validateLocalComparisonRules } from './local-mapping-rules'; import { localRuleConnects, validateLocalComparisonRules } from './local-mapping-rules';
import { definitionHasDurableIdentity } from '../domain/finding-identity'; import { definitionHasDurableIdentity } from '../../domain/finding-identity';
import { resolvedCompatibilityKey } from '../domain/processing-compatibility'; import { resolvedCompatibilityKey } from '../../domain/processing-compatibility';
interface ComparisonContext { interface ComparisonContext {
definitions: Map<string, FindingDefinition>; definitions: Map<string, FindingDefinition>;

View File

@ -1,4 +1,4 @@
import type { JsonValue } from '../domain/canonical-model'; import type { JsonValue } from '../../domain/canonical-model';
export function normalizeComparisonText(value: unknown): string { export function normalizeComparisonText(value: unknown): string {
return String(value ?? '') return String(value ?? '')

View File

@ -1,7 +1,7 @@
import type { FindingDefinition, FindingOccurrence } from '../domain/canonical-model'; import type { FindingDefinition, FindingOccurrence } from '../../domain/canonical-model';
import type { ComparisonRuleEndpoint, LocalComparisonRule } from '../domain/comparison'; import type { ComparisonRuleEndpoint, LocalComparisonRule } from '../../domain/comparison';
import { normalizeComparisonText } from './comparison-normalization'; import { normalizeComparisonText } from './comparison-normalization';
import { definitionHasDurableIdentity } from '../domain/finding-identity'; import { definitionHasDurableIdentity } from '../../domain/finding-identity';
function ruleId(occurrence: FindingOccurrence): string { function ruleId(occurrence: FindingOccurrence): string {
return normalizeComparisonText(occurrence.sourceIdentity.canonicalRuleId ?? occurrence.sourceIdentity.originalRuleId); return normalizeComparisonText(occurrence.sourceIdentity.canonicalRuleId ?? occurrence.sourceIdentity.originalRuleId);

View File

@ -1,4 +1,4 @@
import { compareScanRuns } from '../../comparison/comparison-engine'; import { compareScanRuns } from '../../evaluation/scan-run-comparison/comparison-engine';
import type { import type {
CanonicalAssessmentModel, CanonicalAssessmentModel,
FindingOccurrence, FindingOccurrence,

View File

@ -48,8 +48,8 @@ import { planReimportReviewTransfer, type ReimportReviewTransferSummary } from '
import { canonicalJson } from '../shared/canonical-json'; import { canonicalJson } from '../shared/canonical-json';
import { latestReferenceRuns, type ReferenceCatalogSnapshot, type ReferenceRecognitionRun } from '../domain/reference-recognition'; import { latestReferenceRuns, type ReferenceCatalogSnapshot, type ReferenceRecognitionRun } from '../domain/reference-recognition';
import { validateReferenceRun } from './reference-run-validation'; import { validateReferenceRun } from './reference-run-validation';
import { normalizeComparisonText } from '../comparison/comparison-normalization'; import { normalizeComparisonText } from '../evaluation/scan-run-comparison/comparison-normalization';
import { validateLocalComparisonRules } from '../comparison/local-mapping-rules'; import { validateLocalComparisonRules } from '../evaluation/scan-run-comparison/local-mapping-rules';
import { import {
createEnvironmentUnifiedResult, createEnvironmentUnifiedResult,
createCurrentPolicyEvaluationUnifiedResults, createCurrentPolicyEvaluationUnifiedResults,

View File

@ -87,7 +87,7 @@ const ENGINE_FREE_AREAS = ['Ingest', 'Normalization', 'Evaluation', 'Store', 'Di
describe('Core Engine and engines boundary', () => { describe('Core Engine and engines boundary', () => {
it('knows the registered engines and finds source files for each of them', () => { it('knows the registered engines and finds source files for each of them', () => {
const engines = areas.ENGINE_PATHS.map(([engine]) => engine); const engines = areas.ENGINE_PATHS.map(([engine]) => engine);
expect(engines).toEqual(expect.arrayContaining(['Context Engine', 'Report Engine', 'Remediation Engine', 'Compliance Engine'])); expect(engines).toEqual(expect.arrayContaining(['Context Engine', 'Report Engine', 'Remediation Engine', 'Compliance Engine', 'Comparison Engine']));
for (const engine of engines) { for (const engine of engines) {
expect(allSources().some(file => engineOf(posix(file)) === engine), engine).toBe(true); expect(allSources().some(file => engineOf(posix(file)) === engine), engine).toBe(true);
} }

View File

@ -0,0 +1,100 @@
import { describe, expect, it } from 'vitest';
import type {
CanonicalFindingStatus,
FindingOccurrence,
ImportFileRecord,
ScanRun
} from '../../src/domain/canonical-model';
import type { EnvironmentObservation } from '../../src/domain/environment-context';
import { LocalAssessorRepository } from '../../src/persistence/local-assessor-repository';
import { compareProjectTimeslotEvidence } from '../../src/comparison/timeslot-comparison-engine';
// Permanent on/off invariance of the Comparison Engine (AGENTS.md, "Core Engine and engines"): the Core
// works unchanged with the engine switched off, and running a comparison of two stored timeslots leaves
// every stored ASSESSOR value, the repository revision and the compared models exactly as they were.
const timestamp = '2026-01-01T08:00:00.000Z';
const provenance = {
decisionSource: 'source-native' as const,
confidence: { level: 'high' as const, score: 1, reasons: ['synthetic'] },
mappingVersion: '1'
};
const tool = { id: 'cis-cat', name: 'CIS-CAT Pro' };
function run(id: string, assessmentId: string, importedAt: string): ScanRun {
return { id, assessmentId, sequence: 1, label: id, tool, importedAt };
}
function importFile(runId: string, importedAt: string): ImportFileRecord {
return {
id: `file-${runId}`, scanRunId: runId, fileName: `${runId}.html`, format: 'HTML', sizeBytes: 10, importedAt,
detectedTool: tool, detection: { level: 'high', score: 1, reasons: ['synthetic'] },
processedWith: {
parserId: 'html', parserVersion: '1', profileId: 'cis-cat', profileVersion: '1.4.0', mappingVersion: '1.3.0',
normalizationVersion: '1.3.0', presentationVersion: '1.3.0', compatibilityKey: 'cis-cat-canonical-v1'
},
sourceArtifact: { kind: 'not-retained', reason: 'synthetic' }
};
}
function observation(runId: string, property: string, value: string): EnvironmentObservation {
return {
version: 1, id: `${runId}-${property}`, scanRunId: runId, importFileId: `file-${runId}`, tool,
subject: { type: 'host' }, property, label: property, originalValue: value, normalizedValue: value,
valueType: property === 'report.generated_at' ? 'datetime' : 'string', state: 'observed',
source: {}, mappingVersion: '1', attributes: {}
};
}
function occurrence(runId: string, ruleId: string, status: CanonicalFindingStatus): FindingOccurrence {
return {
id: `${runId}-${ruleId}`, definitionId: `definition-${ruleId}`, scanRunId: runId, importFileId: `file-${runId}`,
sourceIdentity: { tool, canonicalRuleId: ruleId }, sourcePosition: { rowIndex: 1 },
originalFields: [{ key: 'rule', label: 'Rule', columnIndex: 0, value: ruleId }], fieldMappings: [],
severity: { originalValue: 'high', normalizedValue: 'high', provenance },
status: { originalValue: status, normalizedValue: status, provenance },
outcome: { originalValue: status === 'passed' ? 'pass' : 'fail', normalizedValue: status === 'passed' ? 'pass' : 'fail', provenance }
};
}
async function commitTimeslot(repository: LocalAssessorRepository, name: string, observedAt: string, findings: Array<[string, CanonicalFindingStatus]>) {
const assessmentId = `assessment-${name}`;
await repository.saveAssessment({ id: assessmentId, projectId: 'dummy-project', name, createdAt: observedAt, updatedAt: observedAt });
const scanRun = run(`run-${name}`, assessmentId, observedAt);
const session = await repository.beginImport({ sessionId: `session-${name}`, projectId: 'dummy-project', assessmentId, scanRun });
await session.stage({
importFiles: [importFile(scanRun.id, observedAt)],
environmentObservations: [
observation(scanRun.id, 'host.name', 'server-a'),
observation(scanRun.id, 'benchmark.name', 'benchmark-a'),
observation(scanRun.id, 'report.generated_at', observedAt)
],
findingDefinitions: findings.map(([ruleId]) => ({
id: `definition-${ruleId}`, canonicalKey: `cis-cat:${ruleId}`, title: ruleId, frameworkReferences: [],
createdAt: timestamp, updatedAt: timestamp
})),
findingOccurrences: findings.map(([ruleId, status]) => occurrence(scanRun.id, ruleId, status))
});
await session.commit();
return { assessmentId, runId: scanRun.id };
}
describe('Comparison Engine on/off invariance', () => {
it('leaves every stored ASSESSOR value, the repository revision and the compared models unchanged', async () => {
const repository = new LocalAssessorRepository();
await repository.saveProject({ id: 'dummy-project', name: 'Dummy Customer', createdAt: timestamp, updatedAt: timestamp, attributes: {} });
const a = await commitTimeslot(repository, 'a', '2026-01-01T10:00:00.000Z', [['1.1.1', 'open'], ['1.1.2', 'open']]);
const b = await commitTimeslot(repository, 'b', '2026-02-01T10:00:00.000Z', [['1.1.1', 'passed'], ['1.1.2', 'open']]);
const off = { revision: repository.revision, data: JSON.stringify(await repository.exportProject('dummy-project')) };
const models = [(await repository.loadAssessmentModel(a.assessmentId))!, (await repository.loadAssessmentModel(b.assessmentId))!];
const modelsBefore = JSON.stringify(models);
const comparison = compareProjectTimeslotEvidence(models, [a.runId], [b.runId]);
// The engine really ran: the stored fail-to-pass of the same identity is a verified remediation.
expect(comparison.summary.verifiedRemediated).toBe(1);
expect(comparison.summary.confirmedOpen).toBe(1);
expect(JSON.stringify(models)).toBe(modelsBefore);
expect(repository.revision).toBe(off.revision);
expect(JSON.stringify(await repository.exportProject('dummy-project'))).toBe(off.data);
});
});

View File

@ -9,9 +9,9 @@ import type {
ImportFileRecord, ImportFileRecord,
ScanRun, ScanRun,
ToolIdentity ToolIdentity
} from '../../src/domain/canonical-model'; } from '../../../src/domain/canonical-model';
import { compareScanRuns } from '../../src/comparison/comparison-engine'; import { compareScanRuns } from '../../../src/evaluation/scan-run-comparison/comparison-engine';
import { createLocalComparisonRule } from '../../src/comparison/local-mapping-rules'; import { createLocalComparisonRule } from '../../../src/evaluation/scan-run-comparison/local-mapping-rules';
const timestamp = '2026-07-17T08:00:00.000Z'; const timestamp = '2026-07-17T08:00:00.000Z';
const tool = (id: string): ToolIdentity => ({ id, name: id }); const tool = (id: string): ToolIdentity => ({ id, name: id });

View File

@ -9,7 +9,7 @@ import type {
SourceTableRecord SourceTableRecord
} from '../../src/domain/canonical-model'; } from '../../src/domain/canonical-model';
import type { ImportStagingData } from '../../src/persistence/assessor-repository'; import type { ImportStagingData } from '../../src/persistence/assessor-repository';
import { compareScanRuns } from '../../src/comparison/comparison-engine'; import { compareScanRuns } from '../../src/evaluation/scan-run-comparison/comparison-engine';
import { LocalAssessorRepository } from '../../src/persistence/local-assessor-repository'; import { LocalAssessorRepository } from '../../src/persistence/local-assessor-repository';
import { RepositoryConflictError, RepositoryValidationError } from '../../src/persistence/repository-errors'; import { RepositoryConflictError, RepositoryValidationError } from '../../src/persistence/repository-errors';
import { riskCommand } from '../domain/risk-acceptance-fixture'; import { riskCommand } from '../domain/risk-acceptance-fixture';

View File

@ -12,7 +12,7 @@ import type {
import type { StoredGpoBackupInstance } from '../../src/domain/gpo-analysis-persistence'; import type { StoredGpoBackupInstance } from '../../src/domain/gpo-analysis-persistence';
import type { ImportStagingData } from '../../src/persistence/assessor-repository'; import type { ImportStagingData } from '../../src/persistence/assessor-repository';
import type { ImportArtifact, ImportProcessingRun, IngestionJob } from '../../src/domain/ingestion-persistence'; import type { ImportArtifact, ImportProcessingRun, IngestionJob } from '../../src/domain/ingestion-persistence';
import { compareScanRuns } from '../../src/comparison/comparison-engine'; import { compareScanRuns } from '../../src/evaluation/scan-run-comparison/comparison-engine';
import { RepositoryConflictError } from '../../src/persistence/repository-errors'; import { RepositoryConflictError } from '../../src/persistence/repository-errors';
import { runDatabaseMigrations } from '../../server/postgres/migration-runner'; import { runDatabaseMigrations } from '../../server/postgres/migration-runner';
import { PostgresAssessorRepository } from '../../server/postgres/postgres-assessor-repository'; import { PostgresAssessorRepository } from '../../server/postgres/postgres-assessor-repository';