fix(comparison): compare real timeslots (2.26.1)

Split runs imported from several scanners into one source per file scanner
and pair several sources of one scanner by verified scope. On user decisions:
the timeslot order is the time basis (a stated report or backup time only
blocks when B is not newer), a PingCastle rule absent from a complete report
of the same domain and engine version counts as remediated (profile contract
1.1.0), and one report per scanner and timeslot without a stated scope uses
the client as scope. Detail-link and sanitizing diagnostics no longer block;
the scanner score delta uses the engine's verified source pairs.
This commit is contained in:
Binh 2026-10-13 15:00:31 +02:00
parent c26216a4f7
commit dbbbf5a834
14 changed files with 340 additions and 123 deletions

View File

@ -20,6 +20,8 @@ Step 3b, first part (Core release Presentation, Platform): engine read port `src
Step 3b, second part (Core release Governance, Presentation, Core Tests, Platform; the Data Flow diagram part finished under a renewed Presentation, Core Tests, Platform release after an earlier release had expired mid-turn): the engine UI path `src/features/comparison/` is registered (area map, AGENTS.md, onboarding) and holds the formerly parked `ComparisonTab`, `useTimeslotComparison`, `comparison-work-facets` and `FindingComparison`. Extension points (pinned in the boundary test): Engines popup entry `Comparison Engine` (default on); Client Info tab `Compare` after `Compliance`; Work facet `Change` before `Source: Scan` in the shared filter pipeline (filter-records, work-facets, use-filtered-records, SearchFilterBar, ActionlistControls, SidebarSecurityDomains), present only after the Compare tab loaded a comparison whose B is the open timeslot, so opening Work never starts a comparison; a change count opens Work with that facet value (`Resolved` with status All, all others Deviation) and is clickable only when timeslot B holds a record for it; Finding Reader section `Compare` after Context, collapsed, with both states and, for GPO settings, both retained configured values as Work displays them (the engine now carries them). The Data Flow engine diagram shows the Comparison Engine and its two relationships. Product version 2.26.0 (backward-compatible capability). Synthetic test client `compare test` (2026-06-10 and 2026-09-15) created through the regular import API from generated synthetic CIS-CAT reports (resolved, unchanged, regressed, new and absent rules) plus the synthetic Maester fixture in B only; the generated files are in git-ignored `artifacts/compare-engine/test-client/`. Verified: import gate 528/528; main suite 2,474 passed, 18 optional skipped, 1 failed (the same pre-existing, unrelated `db-structure-layout.test.ts` check); local-only and design-language checks pass; both production builds pass. Not yet verified in the running app: it was started outside this session and still runs 2.25.0. Step 3b, second part (Core release Governance, Presentation, Core Tests, Platform; the Data Flow diagram part finished under a renewed Presentation, Core Tests, Platform release after an earlier release had expired mid-turn): the engine UI path `src/features/comparison/` is registered (area map, AGENTS.md, onboarding) and holds the formerly parked `ComparisonTab`, `useTimeslotComparison`, `comparison-work-facets` and `FindingComparison`. Extension points (pinned in the boundary test): Engines popup entry `Comparison Engine` (default on); Client Info tab `Compare` after `Compliance`; Work facet `Change` before `Source: Scan` in the shared filter pipeline (filter-records, work-facets, use-filtered-records, SearchFilterBar, ActionlistControls, SidebarSecurityDomains), present only after the Compare tab loaded a comparison whose B is the open timeslot, so opening Work never starts a comparison; a change count opens Work with that facet value (`Resolved` with status All, all others Deviation) and is clickable only when timeslot B holds a record for it; Finding Reader section `Compare` after Context, collapsed, with both states and, for GPO settings, both retained configured values as Work displays them (the engine now carries them). The Data Flow engine diagram shows the Comparison Engine and its two relationships. Product version 2.26.0 (backward-compatible capability). Synthetic test client `compare test` (2026-06-10 and 2026-09-15) created through the regular import API from generated synthetic CIS-CAT reports (resolved, unchanged, regressed, new and absent rules) plus the synthetic Maester fixture in B only; the generated files are in git-ignored `artifacts/compare-engine/test-client/`. Verified: import gate 528/528; main suite 2,474 passed, 18 optional skipped, 1 failed (the same pre-existing, unrelated `db-structure-layout.test.ts` check); local-only and design-language checks pass; both production builds pass. Not yet verified in the running app: it was started outside this session and still runs 2.25.0.
Corrections after the first real-data test (2026-10-13; engine paths plus Core release Ingest, Core Tests): (1) a scan run imported from several scanners (tool `multiple`) is split into one source per file scanner, and several sources of one scanner pair only by their verified scope (for example two PingCastle domains); before, both made every source not comparable. (2) Explicit user decision 2026-10-13 (supersedes the requirement of a verified newer observation time in TIMESLOT_COMPARISON_PLAN.md §3.2/§3.4 and its matrix row on missing scan times): the explicit timeslot order is the time basis; a source's own report or backup time blocks only when both timeslots state one and B's is not newer (same or older report: `target-observation-not-newer`, not comparable). Import times are still never used. Reason: real GPO backups state their backup time without a zone, which the Core parser drops, and CIS-CAT/Purple Knight times only restate the timeslot order. (3) Diagnostic policy `fail-closed-by-impact`: only codes known to leave identity, results and coverage untouched are exempt (`html-active-content-removed`, `unicode-null-escaped`, `cis-cat-summary-verified`, the five `pingcastle-linked-section-*` detail-link codes); every other or unknown code still blocks. (4) Explicit user decision 2026-10-13: PingCastle declares `negativeEvidence: complete-explicit-negative` (comparison contract 1.1.0, Ingest); a rule absent from B counts as remediated only with the same domain, the same PingCastle engine version (`report.engine_version`), compatible processing and no blocking diagnostic, otherwise it stays unverified. Comparison-contract change only, no extraction change, therefore no reimport (REIMPORT_POLICY.md). (5) The scanner score delta uses the engine's verified source pairs (one scan per side, same method, policy, weight and evaluated rule population); the Core labels `Scan identity or time not verified`, `Scanner version unavailable` and `Incomplete assessment` no longer block it, every other issue does. Test client `iso compare test`: the Dummy lab scans (same formats as `iso test`: Purple Knight and Maester Excel, PingCastle and CIS-CAT HTML, GPO folder) in 2026-07-22 and a changed copy in 2026-10-01 (later report times; Fail/Pass swaps inside one section or category so report totals stay consistent; four GPO password/lockout values; generated in git-ignored `artifacts/compare-engine/iso-copy/B`). (6) Explicit user decision 2026-10-13 (supersedes TIMESLOT_COMPARISON_PLAN.md §3.1 "missing identity is not the empty identity of a supposedly equal object" for this case only): both timeslots belong to one client, so when each timeslot holds exactly one report of a scanner and neither states its scope (the Maester Excel export has no tenant column), the client is the scope. A scope stated on one side only, or several reports of one scanner per side, never pairs this way. Result after all corrections: 9 resolved, 2 value changed, 7 regressed, 467 unchanged, nothing unverified or not comparable; overall score 36.1 -> 36.9 (GPO pass rate 50 -> 54 over 50 settings decided in both). Patch decision: 2.26.1 for these backward-compatible comparison corrections (Core release Platform, Core Tests), package/lock and UI/API version contracts synchronized.
Change log 2026-10-13: recorded the Comparison Engine decision, naming and staging; relocated the Core Phase 8 scan-run comparison and registered the Comparison Engine with its invariance test; step 3a engine comparison of scanners, GPO settings and score basis; step 3b read port, timeslot route, API client, Compare tab, Work Change facet, Finding Reader section and engine diagram; version 2.26.0. Change log 2026-10-13: recorded the Comparison Engine decision, naming and staging; relocated the Core Phase 8 scan-run comparison and registered the Comparison Engine with its invariance test; step 3a engine comparison of scanners, GPO settings and score basis; step 3b read port, timeslot route, API client, Compare tab, Work Change facet, Finding Reader section and engine diagram; version 2.26.0.
## Security finding reading hierarchy (2026-10-11, Phase 7 Presentation) ## Security finding reading hierarchy (2026-10-11, Phase 7 Presentation)

View File

@ -6,6 +6,8 @@ Status: **in Arbeit, nicht fachlich abgenommen**. P0 ist abgeschlossen. P1 besit
Ausdrückliche Nutzerentscheidung 2026-10-13: P1-P6 werden als eigenständig abschaltbare **Comparison Engine** umgesetzt (`src/comparison/`, `server/comparison/`, `tests/comparison/`); der Core-Phase-8-Scanlaufvergleich zieht dafür in einen Core-Evaluation-Ort um. Alle Verträge und die Abnahmematrix dieses Plans bleiben unverändert verbindlich. Stufen und Freigaben: [IMPLEMENTATION_PLAN.md](IMPLEMENTATION_PLAN.md), Abschnitt „Comparison Engine: decision and staging“. Ausdrückliche Nutzerentscheidung 2026-10-13: P1-P6 werden als eigenständig abschaltbare **Comparison Engine** umgesetzt (`src/comparison/`, `server/comparison/`, `tests/comparison/`); der Core-Phase-8-Scanlaufvergleich zieht dafür in einen Core-Evaluation-Ort um. Alle Verträge und die Abnahmematrix dieses Plans bleiben unverändert verbindlich. Stufen und Freigaben: [IMPLEMENTATION_PLAN.md](IMPLEMENTATION_PLAN.md), Abschnitt „Comparison Engine: decision and staging“.
Ausdrückliche Nutzerentscheidungen 2026-10-13 (ersetzen die genannten Stellen): (a) Zeitbasis ist die ausdrückliche Timeslot-Reihenfolge A vor B; die Forderung nach einer verifizierten neueren Beobachtungszeit in §3.2/§3.4 und in der Matrixzeile „Neuerer Scanzeitpunkt fehlt …“ entfällt. Eine eigene Bericht- oder Backupzeit sperrt nur, wenn beide Timeslots eine nennen und B nicht neuer ist (gleicher oder älterer Bericht: nicht vergleichbar). Importzeit wird weiterhin nie verwendet. (b) PingCastle erfüllt den profilgebundenen Negativnachweisvertrag: eine in B fehlende Regel gilt als behoben bei gleicher Domain, gleicher PingCastle-Engineversion, kompatibler Verarbeitung und ohne sperrende Diagnose; sonst bleibt sie ungeklärt. (c) Beide Timeslots gehören zu einem Kunden: Enthält jeder Timeslot genau einen Bericht eines Scanners und nennt keiner seinen Scope (Maester-Excel ohne Tenant), ist der Kunde der Scope. Ein nur einseitig genannter Scope oder mehrere Berichte eines Scanners je Timeslot werden so nie gepaart; ersetzt §3.1 nur für diesen Fall. Details: [IMPLEMENTATION_PLAN.md](IMPLEMENTATION_PLAN.md), Abschnitt „Comparison Engine“.
Dieser Plan präzisiert das Arbeitspaket „faire Vergleiche bei fehlenden Folgequellen“ in [IMPLEMENTATION_PLAN.md](IMPLEMENTATION_PLAN.md). Für die künftige Umsetzung ersetzt er ausdrücklich die ältere Schlussfolgerung „abwesend = behoben“ und die pauschale Ablösung offener Verpflichtungen durch einen neuen Lauf derselben Quelle. Historische Ergebnisse und Originaldaten werden dadurch nicht rückwirkend geändert. Dieser Plan präzisiert das Arbeitspaket „faire Vergleiche bei fehlenden Folgequellen“ in [IMPLEMENTATION_PLAN.md](IMPLEMENTATION_PLAN.md). Für die künftige Umsetzung ersetzt er ausdrücklich die ältere Schlussfolgerung „abwesend = behoben“ und die pauschale Ablösung offener Verpflichtungen durch einen neuen Lauf derselben Quelle. Historische Ergebnisse und Originaldaten werden dadurch nicht rückwirkend geändert.
## 1. Ziel und Grenzen ## 1. Ziel und Grenzen

4
package-lock.json generated
View File

@ -1,12 +1,12 @@
{ {
"name": "assessor", "name": "assessor",
"version": "2.26.0", "version": "2.26.1",
"lockfileVersion": 3, "lockfileVersion": 3,
"requires": true, "requires": true,
"packages": { "packages": {
"": { "": {
"name": "assessor", "name": "assessor",
"version": "2.26.0", "version": "2.26.1",
"dependencies": { "dependencies": {
"@fontsource-variable/inter": "^5.3.0", "@fontsource-variable/inter": "^5.3.0",
"jsdom": "^29.1.1", "jsdom": "^29.1.1",

View File

@ -1,6 +1,6 @@
{ {
"name": "assessor", "name": "assessor",
"version": "2.26.0", "version": "2.26.1",
"private": true, "private": true,
"type": "module", "type": "module",
"scripts": { "scripts": {

View File

@ -98,13 +98,14 @@ function rollUp(results: PolicyWorkResult[]): Map<string, TimeslotSetting> {
return settings; return settings;
} }
const newer = (baseline?: string, target?: string) => Boolean(baseline && target /** The timeslot order is the time basis (user decision 2026-10-13); stated backup times only block when B's is not newer. */
&& Number.isFinite(Date.parse(baseline)) && Number.isFinite(Date.parse(target)) && Date.parse(target) > Date.parse(baseline)); const notNewer = (baseline?: string, target?: string) => Boolean(baseline && target
&& Number.isFinite(Date.parse(baseline)) && Number.isFinite(Date.parse(target)) && Date.parse(target) <= Date.parse(baseline));
/** /**
* Conservative A/B comparison of rolled-up GPO settings. The authoritative verdicts are read as * Conservative A/B comparison of rolled-up GPO settings. The authoritative verdicts are read as
* they are and never re-derived. Remediation needs the same setting with an explicit Pass against the * they are and never re-derived. Remediation needs the same setting with an explicit Pass against the
* same baselines in a newer backup set; an absent or undecided setting stays unverified, because a * same baselines in timeslot B (whose stated backup time, if any, must be newer); an absent or undecided setting stays unverified, because a
* GPO backup set is no proof of a removed configuration. * GPO backup set is no proof of a removed configuration.
*/ */
export function compareGpoTimeslots(baseline: GpoTimeslotInput, target: GpoTimeslotInput): GpoTimeslotComparison { export function compareGpoTimeslots(baseline: GpoTimeslotInput, target: GpoTimeslotInput): GpoTimeslotComparison {
@ -129,11 +130,11 @@ export function compareGpoTimeslots(baseline: GpoTimeslotInput, target: GpoTimes
decisions: baselineFails.map(item => decision(item, 'not-comparable', sourceReasons)), decisions: baselineFails.map(item => decision(item, 'not-comparable', sourceReasons)),
newDeviations: [], regressedDeviations: [] newDeviations: [], regressedDeviations: []
}; };
const timeVerified = newer(baseline.observedAt, target.observedAt); const timeVerified = !notNewer(baseline.observedAt, target.observedAt);
const decisions = baselineFails.map(item => { const decisions = baselineFails.map(item => {
if (item.ambiguous) return decision(item, 'not-comparable', ['gpo-setting-ambiguous']); if (item.ambiguous) return decision(item, 'not-comparable', ['gpo-setting-ambiguous']);
const other = targetSettings.get(item.key); const other = targetSettings.get(item.key);
if (!timeVerified) return decision(item, 'unverified-absent', ['newer-observation-not-verified'], other); if (!timeVerified) return decision(item, 'not-comparable', ['target-observation-not-newer'], other);
if (!other) return decision(item, 'unverified-absent', ['target-setting-absent']); if (!other) return decision(item, 'unverified-absent', ['target-setting-absent']);
if (other.ambiguous) return decision(item, 'unverified-absent', ['gpo-setting-ambiguous'], other); if (other.ambiguous) return decision(item, 'unverified-absent', ['gpo-setting-ambiguous'], other);
if (other.baselineIds !== item.baselineIds) return decision(item, 'not-comparable', ['baseline-set-changed'], other); if (other.baselineIds !== item.baselineIds) return decision(item, 'not-comparable', ['baseline-set-changed'], other);
@ -165,7 +166,7 @@ export function compareGpoTimeslots(baseline: GpoTimeslotInput, target: GpoTimes
}); });
const rate = (values: string[]) => values.filter(value => value === 'Pass').length / values.length * 100; const rate = (values: string[]) => values.filter(value => value === 'Pass').length / values.length * 100;
return { return {
version, state: 'compared', reasons: timeVerified ? [] : ['newer-observation-not-verified'], version, state: timeVerified ? 'compared' : 'not-comparable', reasons: timeVerified ? [] : ['target-observation-not-newer'],
decisions, newDeviations, regressedDeviations, decisions, newDeviations, regressedDeviations,
...(timeVerified && shared.length ? { sharedPassRate: { ...(timeVerified && shared.length ? { sharedPassRate: {
settings: shared.length, baseline: rate(shared.map(item => item.before)), target: rate(shared.map(item => item.after)) settings: shared.length, baseline: rate(shared.map(item => item.before)), target: rate(shared.map(item => item.after))

View File

@ -24,6 +24,9 @@ export interface TimeslotSourceComparison {
toolId: string; toolId: string;
baselineRunId?: string; baselineRunId?: string;
targetRunId?: string; targetRunId?: string;
/** The import files of this source in each timeslot; a run from several scanners holds several sources. */
baselineFileIds: string[];
targetFileIds: string[];
state: 'compared' | 'not-comparable'; state: 'compared' | 'not-comparable';
reasons: string[]; reasons: string[];
decisions: TimeslotOccurrenceDecision[]; decisions: TimeslotOccurrenceDecision[];
@ -34,7 +37,7 @@ export interface TimeslotSourceComparison {
} }
export interface ProjectTimeslotEvidenceComparison { export interface ProjectTimeslotEvidenceComparison {
version: '1.1.0'; version: '1.2.0';
projectId: string; projectId: string;
baselineRunIds: string[]; baselineRunIds: string[];
targetRunIds: string[]; targetRunIds: string[];
@ -53,6 +56,40 @@ export interface ProjectTimeslotEvidenceComparison {
const normalizedTool = (value: string): string => value.trim().toLocaleLowerCase('en-US'); const normalizedTool = (value: string): string => value.trim().toLocaleLowerCase('en-US');
/**
* Diagnostic policy `fail-closed-by-impact`: a diagnostic blocks remediation claims unless it is known to leave
* identity, results and coverage untouched. These codes only sanitize or confirm the source; every other code,
* including unknown future ones, still blocks.
*/
const NON_IMPACTING_DIAGNOSTICS = new Set(['html-active-content-removed', 'unicode-null-escaped', 'cis-cat-summary-verified',
// PingCastle detail links affect only attached evidence tables, never a rule, its scope or its presence.
'pingcastle-linked-section-unresolved', 'pingcastle-linked-section-ambiguous', 'pingcastle-linked-section-depth-limit',
'pingcastle-linked-section-target-limit', 'pingcastle-linked-section-table-limit']);
/**
* User decision 2026-10-13: the explicit timeslot order is the time basis; timeslot A lies before B. These markers
* carry that order into the shared evidence contract. A source's own report time only blocks when both timeslots
* state one and B's is not newer (an older report or the same report again).
*/
const TIMESLOT_A = '2000-01-01T00:00:00.000Z';
const TIMESLOT_B = '2000-01-02T00:00:00.000Z';
function oneValue(observations: SourceUnit['observations'], property: string): string | undefined {
const values = [...new Set(observations.filter(item => item.state === 'observed' && item.property === property)
.map(item => String(item.normalizedValue ?? item.originalValue ?? '').trim()).filter(Boolean))];
return values.length === 1 ? values[0] : undefined;
}
type AvailableDefinition = Exclude<ToolTimeslotComparisonDefinition, { kind: 'unavailable' }>;
interface SourceUnit {
toolId: string;
run: ScanRun;
files: ImportFileRecord[];
fileIds: Set<string>;
occurrences: FindingOccurrence[];
observations: NonNullable<CanonicalAssessmentModel['environmentObservations']>;
}
function occurrenceOutcome(occurrence: FindingOccurrence) { function occurrenceOutcome(occurrence: FindingOccurrence) {
return occurrence.outcome?.normalizedValue return occurrence.outcome?.normalizedValue
?? outcomeFromLegacyStatus(occurrence.status.normalizedValue ?? 'unknown'); ?? outcomeFromLegacyStatus(occurrence.status.normalizedValue ?? 'unknown');
@ -120,78 +157,121 @@ export function compareProjectTimeslotEvidence(
const runById = new Map(runs.map(run => [run.id, run])); const runById = new Map(runs.map(run => [run.id, run]));
const missing = [...baselineRunIds, ...targetRunIds].filter(id => !runById.has(id)); const missing = [...baselineRunIds, ...targetRunIds].filter(id => !runById.has(id));
if (missing.length) throw new Error('At least one selected scanner run is unavailable.'); if (missing.length) throw new Error('At least one selected scanner run is unavailable.');
const selected = (ids: string[]) => ids.map(id => runById.get(id)!);
const byTool = (items: ScanRun[]) => {
const map = new Map<string, ScanRun[]>();
items.forEach(run => {
const toolId = normalizedTool(run.tool.id);
map.set(toolId, [...(map.get(toolId) ?? []), run]);
});
return map;
};
const baselineByTool = byTool(selected(baselineRunIds));
const targetByTool = byTool(selected(targetRunIds));
const toolIds = [...new Set([...baselineByTool.keys(), ...targetByTool.keys()])].sort();
const occurrences = models.flatMap(model => model.findingOccurrences); const occurrences = models.flatMap(model => model.findingOccurrences);
const files = models.flatMap(model => model.importFiles); const files = models.flatMap(model => model.importFiles);
const observations = models.flatMap(model => model.environmentObservations ?? []); const observations = models.flatMap(model => model.environmentObservations ?? []);
const diagnostics = models.flatMap(model => model.diagnostics); const diagnostics = models.flatMap(model => model.diagnostics);
const sources = toolIds.map((toolId): TimeslotSourceComparison => { // A source is the files of one run that one scanner produced. A run imported from several scanners
const baselineRuns = baselineByTool.get(toolId) ?? []; // (tool `multiple`) is split by each file's own scanner; sources are never merged into one contract.
const targetRuns = targetByTool.get(toolId) ?? []; const sourcesOf = (ids: string[]): SourceUnit[] => ids.flatMap(runId => {
const baselineRun = baselineRuns.length === 1 ? baselineRuns[0] : undefined; const run = runById.get(runId)!;
const targetRun = targetRuns.length === 1 ? targetRuns[0] : undefined; const byTool = new Map<string, ImportFileRecord[]>();
const baselineOccurrences = baselineRun ? occurrences.filter(item => item.scanRunId === baselineRun.id) : []; for (const file of files.filter(item => item.scanRunId === runId)) {
const profile = toolProfileDefinitions.find(item => item.id === toolId); const toolId = normalizedTool(file.detectedTool.id && file.detectedTool.id !== 'unknown' ? file.detectedTool.id : run.tool.id);
const pairingReasons = [ byTool.set(toolId, [...(byTool.get(toolId) ?? []), file]);
...(baselineRuns.length !== 1 ? [baselineRuns.length ? 'baseline-source-ambiguous' : 'baseline-source-missing'] : []), }
...(targetRuns.length !== 1 ? [targetRuns.length ? 'target-source-ambiguous' : 'target-source-missing'] : []), if (!byTool.size) byTool.set(normalizedTool(run.tool.id), []);
...(!profile ? ['comparison-profile-missing'] : []), return [...byTool].map(([toolId, unitFiles]) => {
...(profile?.timeslotComparison.kind === 'unavailable' ? ['comparison-profile-unavailable'] : []) const fileIds = new Set(unitFiles.map(item => item.id));
]; return {
if (!baselineRun || !targetRun || !profile || profile.timeslotComparison.kind === 'unavailable') return { toolId, run, files: unitFiles, fileIds,
toolId, baselineRunId: baselineRun?.id, targetRunId: targetRun?.id, occurrences: occurrences.filter(item => item.scanRunId === runId && (!fileIds.size || fileIds.has(item.importFileId))),
state: 'not-comparable', reasons: pairingReasons, observations: observations.filter(item => item.scanRunId === runId && fileIds.has(item.importFileId) && normalizedTool(item.tool.id) === toolId)
decisions: unmatchedDecisions(baselineOccurrences, pairingReasons), newDeviationOccurrenceIds: [], };
regressedDeviationOccurrenceIds: [] });
}; });
const baselineSources = sourcesOf(baselineRunIds);
const targetSources = sourcesOf(targetRunIds);
const toolIds = [...new Set([...baselineSources, ...targetSources].map(item => item.toolId))].sort();
const notComparable = (toolId: string, baseline: SourceUnit | undefined, target: SourceUnit | undefined, reasons: string[]): TimeslotSourceComparison => ({
toolId, baselineRunId: baseline?.run.id, targetRunId: target?.run.id,
baselineFileIds: baseline ? [...baseline.fileIds] : [], targetFileIds: target ? [...target.fileIds] : [],
state: 'not-comparable', reasons, decisions: unmatchedDecisions(baseline?.occurrences ?? [], reasons),
newDeviationOccurrenceIds: [], regressedDeviationOccurrenceIds: []
});
const sources = toolIds.flatMap((toolId): TimeslotSourceComparison[] => {
const baselineUnits = baselineSources.filter(item => item.toolId === toolId);
const targetUnits = targetSources.filter(item => item.toolId === toolId);
const profile = toolProfileDefinitions.find(item => item.id === toolId);
if (!profile || profile.timeslotComparison.kind === 'unavailable') {
const reasons = [profile ? 'comparison-profile-unavailable' : 'comparison-profile-missing'];
return [...baselineUnits.map(unit => notComparable(toolId, unit, undefined, [...reasons, ...(targetUnits.length ? [] : ['target-source-missing'])])),
...targetUnits.map(unit => notComparable(toolId, undefined, unit, [...reasons, ...(baselineUnits.length ? [] : ['baseline-source-missing'])]))];
}
const definition = profile.timeslotComparison; const definition = profile.timeslotComparison;
const baselineFiles = files.filter(item => item.scanRunId === baselineRun.id); // Several sources of one scanner (for example two domains) pair only by their verified scope.
const targetFiles = files.filter(item => item.scanRunId === targetRun.id); const scopeOf = (unit: SourceUnit) => comparisonScopeIdentity(definition, unit.observations);
const compatibility = filesCompatibility(baselineFiles, targetFiles); const keyOf = (unit: SourceUnit) => { const scope = scopeOf(unit); return scope.state === 'verified' ? scope.scope.key : undefined; };
const baselineObservations = observations.filter(item => item.scanRunId === baselineRun.id && normalizedTool(item.tool.id) === toolId); const paired = new Set<SourceUnit>();
const targetObservations = observations.filter(item => item.scanRunId === targetRun.id && normalizedTool(item.tool.id) === toolId); const results = baselineUnits.map((baseline): TimeslotSourceComparison => {
const baselineScope = comparisonScopeIdentity(definition, baselineObservations); if (baselineUnits.length === 1 && targetUnits.length === 1) {
const targetScope = comparisonScopeIdentity(definition, targetObservations); paired.add(targetUnits[0]);
const baselineTime = comparisonObservationTime(definition, baselineObservations); return compareSources(toolId, definition, baseline, targetUnits[0]);
const targetTime = comparisonObservationTime(definition, targetObservations); }
const scope = scopeOf(baseline);
if (scope.state !== 'verified') return notComparable(toolId, baseline, undefined, [...scope.reasons, 'baseline-source-ambiguous']);
const sameBaseline = baselineUnits.filter(unit => keyOf(unit) === scope.scope.key);
const matches = targetUnits.filter(unit => keyOf(unit) === scope.scope.key);
if (sameBaseline.length > 1 || matches.length > 1) {
matches.forEach(unit => paired.add(unit));
return notComparable(toolId, baseline, undefined, [sameBaseline.length > 1 ? 'baseline-source-ambiguous' : 'target-source-ambiguous']);
}
if (!matches.length) return notComparable(toolId, baseline, undefined, [targetUnits.length ? 'comparison-scope-changed' : 'target-source-missing']);
paired.add(matches[0]);
return compareSources(toolId, definition, baseline, matches[0]);
});
// A source only in B is never new or worse; its deviations are reported as not comparable.
for (const target of targetUnits.filter(unit => !paired.has(unit))) {
results.push(notComparable(toolId, undefined, target, [baselineUnits.length ? 'comparison-scope-changed' : 'baseline-source-missing']));
}
return results;
});
function compareSources(toolId: string, definition: AvailableDefinition, baseline: SourceUnit, target: SourceUnit): TimeslotSourceComparison {
const baselineOccurrences = baseline.occurrences;
const compatibility = filesCompatibility(baseline.files, target.files);
let baselineScope = comparisonScopeIdentity(definition, baseline.observations);
let targetScope = comparisonScopeIdentity(definition, target.observations);
// User decision 2026-10-13: both timeslots belong to one client. When each holds exactly one report of this
// scanner and neither states its scope (a Maester Excel export has no tenant), the client is the scope.
// A scope stated on one side only, or several reports per side, never pairs this way.
const single = baselineSources.filter(unit => unit.toolId === toolId).length === 1
&& targetSources.filter(unit => unit.toolId === toolId).length === 1;
const unstated = (scope: typeof baselineScope) => scope.state === 'unverified'
&& scope.reasons.every(reason => reason === 'scope-property-missing');
if (single && unstated(baselineScope) && unstated(targetScope)) {
const clientScope = { state: 'verified' as const, scope: { kind: 'client', key: JSON.stringify(['client', projectId]) } };
baselineScope = clientScope;
targetScope = clientScope;
}
const baselineTime = comparisonObservationTime(definition, baseline.observations);
const targetTime = comparisonObservationTime(definition, target.observations);
const sourceReasons = [ const sourceReasons = [
...compatibility.reasons, ...compatibility.reasons,
...(baselineScope.state === 'unverified' ? baselineScope.reasons : []), ...(baselineScope.state === 'unverified' ? baselineScope.reasons : []),
...(targetScope.state === 'unverified' ? targetScope.reasons : []), ...(targetScope.state === 'unverified' ? targetScope.reasons : []),
...(baselineTime.state === 'unverified' ? [baselineTime.reason] : []), ...(baselineTime.state === 'verified' && targetTime.state === 'verified'
...(targetTime.state === 'unverified' ? [targetTime.reason] : []) && Date.parse(targetTime.observedAt) <= Date.parse(baselineTime.observedAt) ? ['target-observation-not-newer'] : [])
]; ];
if (baselineScope.state === 'verified' && targetScope.state === 'verified' if (baselineScope.state === 'verified' && targetScope.state === 'verified'
&& baselineScope.scope.key !== targetScope.scope.key) sourceReasons.push('comparison-scope-changed'); && baselineScope.scope.key !== targetScope.scope.key) sourceReasons.push('comparison-scope-changed');
const baselineFileIds = new Set(baselineFiles.map(item => item.id));
const targetFileIds = new Set(targetFiles.map(item => item.id));
const blockingDiagnostics = diagnostics const blockingDiagnostics = diagnostics
.filter(item => item.severity !== 'info' && (baselineFileIds.has(item.importFileId) || targetFileIds.has(item.importFileId))) .filter(item => item.severity !== 'info' && !NON_IMPACTING_DIAGNOSTICS.has(item.code)
&& (baseline.fileIds.has(item.importFileId) || target.fileIds.has(item.importFileId)))
.map(item => `diagnostic:${item.code}`); .map(item => `diagnostic:${item.code}`);
const sourceComparable = sourceReasons.length === 0; if (sourceReasons.length || baselineScope.state !== 'verified' || targetScope.state !== 'verified') {
if (!sourceComparable || baselineScope.state !== 'verified' || targetScope.state !== 'verified' return notComparable(toolId, baseline, target, [...new Set(sourceReasons)]);
|| baselineTime.state !== 'verified' || targetTime.state !== 'verified') return { }
toolId, baselineRunId: baselineRun.id, targetRunId: targetRun.id, // A findings-only report proves absence only under its declared negative-evidence contract: same scope
state: 'not-comparable', reasons: [...new Set(sourceReasons)], // (paired above), compatible processing, no blocking diagnostic and the same scanner engine version.
decisions: unmatchedDecisions(baselineOccurrences, [...new Set(sourceReasons)]), newDeviationOccurrenceIds: [], const baselineEngine = oneValue(baseline.observations, 'report.engine_version');
regressedDeviationOccurrenceIds: [] const absenceProven = definition.negativeEvidence === 'complete-explicit-negative' && compatibility.compatible
}; && !blockingDiagnostics.length && Boolean(baselineEngine) && baselineEngine === oneValue(target.observations, 'report.engine_version');
const targetOccurrences = occurrences.filter(item => item.scanRunId === targetRun.id); const targetIdentities = target.occurrences.map(item => ({ item, identity: identityFor(item, definition, targetScope.scope) }));
const targetIdentities = targetOccurrences.map(item => ({ item, identity: identityFor(item, definition, targetScope.scope) }));
const targetByIdentity = new Map<string, typeof targetIdentities>(); const targetByIdentity = new Map<string, typeof targetIdentities>();
targetIdentities.forEach(entry => { targetIdentities.forEach(entry => {
if (entry.identity.state !== 'verified') return; if (entry.identity.state !== 'verified') return;
@ -213,21 +293,22 @@ export function compareProjectTimeslotEvidence(
baselineOccurrenceId: item.id, identityKey: identity.identity.key, baselineOccurrenceId: item.id, identityKey: identity.identity.key,
state: 'not-comparable', reasons: ['comparison-identity-ambiguous'] state: 'not-comparable', reasons: ['comparison-identity-ambiguous']
}; };
const target = matches.length === 1 ? matches[0] : undefined; const match = matches.length === 1 ? matches[0] : undefined;
const decision = evaluateRemediationEvidence({ const decision = evaluateRemediationEvidence({
baselineIdentity: identity, baselineIdentity: identity,
targetIdentity: target?.identity, targetIdentity: match?.identity,
baselineObservedAt: baselineTime.observedAt, baselineObservedAt: TIMESLOT_A,
targetObservedAt: targetTime.observedAt, targetObservedAt: TIMESLOT_B,
processingCompatible: compatibility.compatible, processingCompatible: compatibility.compatible,
targetOutcome: target ? occurrenceOutcome(target.item) : undefined, targetOutcome: match ? occurrenceOutcome(match.item) : undefined,
targetIsIoe: target?.item.isIoe, targetIsIoe: match?.item.isIoe,
coverage: definition.resultEvidence === 'explicit-results' ? 'complete' : 'unknown', coverage: definition.resultEvidence === 'explicit-results' || absenceProven ? 'complete' : 'unknown',
coveredIdentityKeys: absenceProven ? [identity.identity.key] : [],
absenceSemantics: definition.negativeEvidence === 'complete-explicit-negative' ? 'explicit-negative' : 'not-reported', absenceSemantics: definition.negativeEvidence === 'complete-explicit-negative' ? 'explicit-negative' : 'not-reported',
blockingDiagnostics blockingDiagnostics
}); });
return { return {
baselineOccurrenceId: item.id, ...(target ? { targetOccurrenceId: target.item.id } : {}), baselineOccurrenceId: item.id, ...(match ? { targetOccurrenceId: match.item.id } : {}),
identityKey: decision.identityKey, state: decision.state, reasons: decision.reasons identityKey: decision.identityKey, state: decision.state, reasons: decision.reasons
}; };
}); });
@ -242,20 +323,21 @@ export function compareProjectTimeslotEvidence(
&& isDeviation(Boolean(entry.item.isIoe), occurrenceOutcome(entry.item)) && isDeviation(Boolean(entry.item.isIoe), occurrenceOutcome(entry.item))
&& (targetByIdentity.get(entry.identity.identity.key) ?? []).length === 1 && (targetByIdentity.get(entry.identity.identity.key) ?? []).length === 1
&& !baselineKeys.has(entry.identity.identity.key)); && !baselineKeys.has(entry.identity.identity.key));
const keyOf = (entry: typeof targetIdentities[number]) => entry.identity.state === 'verified' ? entry.identity.identity.key : ''; const keyOfEntry = (entry: typeof targetIdentities[number]) => entry.identity.state === 'verified' ? entry.identity.identity.key : '';
const regressed = currentDeviations.filter(entry => baselinePassCounts.get(keyOf(entry)) === 1); const regressed = currentDeviations.filter(entry => baselinePassCounts.get(keyOfEntry(entry)) === 1);
// Several passes of one identity in A are ambiguous history: neither new nor regressed. // Several passes of one identity in A are ambiguous history: neither new nor regressed.
const newDeviationOccurrenceIds = currentDeviations.filter(entry => !baselinePassCounts.has(keyOf(entry))).map(entry => entry.item.id); const newDeviationOccurrenceIds = currentDeviations.filter(entry => !baselinePassCounts.has(keyOfEntry(entry))).map(entry => entry.item.id);
return { return {
toolId, baselineRunId: baselineRun.id, targetRunId: targetRun.id, toolId, baselineRunId: baseline.run.id, targetRunId: target.run.id,
baselineFileIds: [...baseline.fileIds], targetFileIds: [...target.fileIds],
state: 'compared', reasons: [...new Set(blockingDiagnostics)], decisions, newDeviationOccurrenceIds, state: 'compared', reasons: [...new Set(blockingDiagnostics)], decisions, newDeviationOccurrenceIds,
regressedDeviationOccurrenceIds: regressed.map(entry => entry.item.id) regressedDeviationOccurrenceIds: regressed.map(entry => entry.item.id)
}; };
}); }
const decisions = sources.flatMap(source => source.decisions); const decisions = sources.flatMap(source => source.decisions);
return { return {
version: '1.1.0', projectId, baselineRunIds: [...baselineRunIds], targetRunIds: [...targetRunIds], sources, version: '1.2.0', projectId, baselineRunIds: [...baselineRunIds], targetRunIds: [...targetRunIds], sources,
summary: { summary: {
baselineDeviations: decisions.length, baselineDeviations: decisions.length,
verifiedRemediated: decisions.filter(item => item.state === 'verified-remediated').length, verifiedRemediated: decisions.filter(item => item.state === 'verified-remediated').length,

View File

@ -3,7 +3,7 @@ import type { RemediationEvidenceState } from '../domain/timeslot-comparison';
import { isDeviation } from '../domain/deviation-predicate'; import { isDeviation } from '../domain/deviation-predicate';
import { outcomeFromLegacyStatus } from '../domain/unified-result'; import { outcomeFromLegacyStatus } from '../domain/unified-result';
import type { SecurityScanScore } from '../domain/security-scoring'; import type { SecurityScanScore } from '../domain/security-scoring';
import { compareSecurityScores, defaultSecurityScoringPolicy, weightedSecurityScore } from '../domain/security-scoring'; import { defaultSecurityScoringPolicy, weightedSecurityScore } from '../domain/security-scoring';
import { canonicalSecurityScores } from '../scoring/canonical-security-score'; import { canonicalSecurityScores } from '../scoring/canonical-security-score';
import { compareProjectTimeslotEvidence, type ProjectTimeslotEvidenceComparison } from './timeslot-comparison-engine'; import { compareProjectTimeslotEvidence, type ProjectTimeslotEvidenceComparison } from './timeslot-comparison-engine';
import { compareGpoTimeslots, type GpoTimeslotComparison, type GpoTimeslotInput } from './gpo-timeslot-comparison'; import { compareGpoTimeslots, type GpoTimeslotComparison, type GpoTimeslotInput } from './gpo-timeslot-comparison';
@ -91,7 +91,8 @@ function scannerChanges(models: CanonicalAssessmentModel[], scanner: ProjectTime
})), })),
// Deviations of a source that cannot be compared are neither new nor worse. // Deviations of a source that cannot be compared are neither new nor worse.
...(source.state === 'not-comparable' && source.targetRunId ? occurrences ...(source.state === 'not-comparable' && source.targetRunId ? occurrences
.filter(item => item.scanRunId === source.targetRunId && isDeviation(Boolean(item.isIoe), outcomeOf(item))) .filter(item => item.scanRunId === source.targetRunId && (!source.targetFileIds.length || source.targetFileIds.includes(item.importFileId))
&& isDeviation(Boolean(item.isIoe), outcomeOf(item)))
.map((item): TimeslotChangeEntry => ({ .map((item): TimeslotChangeEntry => ({
sourceId: source.toolId, change: 'not-comparable', side: 'target', recordIds: [item.id], counterpartIds: [], sourceId: source.toolId, change: 'not-comparable', side: 'target', recordIds: [item.id], counterpartIds: [],
reasons: source.reasons reasons: source.reasons
@ -127,37 +128,51 @@ function gpoChanges(gpo: GpoTimeslotComparison, target?: GpoTimeslotInput): Time
return changes; return changes;
} }
function scannerScores(models: CanonicalAssessmentModel[], baselineRunIds: string[], targetRunIds: string[]) { /**
* Score labels the engine verifies itself through its source pairing (scope, newer report time, compatible
* processing), or that describe both sides of an identical evaluated population. Every other issue blocks the delta.
*/
const ENGINE_VERIFIED_SCORE_ISSUES = new Set(['Scan identity or time not verified', 'Scanner version unavailable', 'Incomplete assessment']);
/**
* Score delta per scanner over the engine's verified source pairs: exactly one scan per side, the same method,
* scoring policy and weight, and the same evaluated rule population. A scanner counts only when every source
* it has in either timeslot is such a pair; native scores are never recomputed for a subset.
*/
function scannerScores(models: CanonicalAssessmentModel[], scanner: ProjectTimeslotEvidenceComparison): TimeslotSourceScore[] {
const scans = canonicalSecurityScores(models); const scans = canonicalSecurityScores(models);
const pick = (ids: string[]) => scans.filter(scan => scan.runIds.some(id => ids.includes(id))); const scansOf = (fileIds: string[]) => scans.filter(scan => scan.importFileIds.some(id => fileIds.includes(id)));
const baseline = pick(baselineRunIds); const toolIds = [...new Set(scanner.sources.map(source => source.toolId))].sort();
const target = pick(targetRunIds); return toolIds.map((toolId): TimeslotSourceScore => {
const byTool = (items: SecurityScanScore[]) => { const sources = scanner.sources.filter(source => source.toolId === toolId);
const map = new Map<string, SecurityScanScore[]>(); const anyScan = scansOf(sources.flatMap(source => [...source.baselineFileIds, ...source.targetFileIds]))[0];
items.forEach(item => map.set(item.toolId, [...(map.get(item.toolId) ?? []), item])); const policy = anyScan?.policy ?? models[0]?.assessment.scoringPolicy ?? defaultSecurityScoringPolicy();
return map;
};
const baselineByTool = byTool(baseline);
const targetByTool = byTool(target);
return [...new Set([...baselineByTool.keys(), ...targetByTool.keys()])].sort().map((toolId): TimeslotSourceScore => {
const left = baselineByTool.get(toolId) ?? [];
const right = targetByTool.get(toolId) ?? [];
const policy = (left[0] ?? right[0]).policy;
const weight = policy.weights[toolId] ?? policy.defaultWeight; const weight = policy.weights[toolId] ?? policy.defaultWeight;
if (!left.length || !right.length) return { const result = (reasons: string[], scores?: { baseline: number; target: number }): TimeslotSourceScore => ({
sourceId: toolId, weight, method: 'paired-scans', reasons: [left.length ? 'target-source-missing' : 'baseline-source-missing'] sourceId: toolId, weight, method: 'paired-scans', reasons: [...new Set(reasons)],
}; ...(scores ? { baselineScore: scores.baseline, targetScore: scores.target, change: scores.target - scores.baseline } : {})
const compared = compareSecurityScores(left, right); });
const reasons = [ const notCompared = sources.filter(source => source.state !== 'compared');
...(compared.comparable === 0 ? ['no-comparable-scan-pair'] : []), if (notCompared.length) return result(notCompared.flatMap(source => source.reasons));
...(compared.scopeChanged ? ['scan-scope-changed'] : []) const pairs: Array<[SecurityScanScore, SecurityScanScore]> = [];
]; for (const source of sources) {
return { const left = scansOf(source.baselineFileIds);
sourceId: toolId, weight, method: 'paired-scans', reasons, const right = scansOf(source.targetFileIds);
...(compared.change !== undefined && !compared.scopeChanged ? { if (left.length !== 1 || right.length !== 1) return result(['scan-pair-ambiguous']);
baselineScore: compared.baselineScore, targetScore: compared.currentScore, change: compared.change const [a, b] = [left[0], right[0]];
} : {}) const blocking = [...a.issues, ...b.issues].filter(issue => !ENGINE_VERIFIED_SCORE_ISSUES.has(issue));
}; if (blocking.length) return result(blocking);
if (a.method !== b.method) return result(['score-method-changed']);
if (a.policy.version !== b.policy.version || (a.policy.weights[toolId] ?? a.policy.defaultWeight) !== (b.policy.weights[toolId] ?? b.policy.defaultWeight)) {
return result(['scoring-policy-changed']);
}
if (!a.evaluationScopeKey || a.evaluationScopeKey !== b.evaluationScopeKey) return result(['scan-scope-changed']);
if (a.score === undefined || b.score === undefined) return result(['score-unavailable']);
pairs.push([a, b]);
}
const baseline = weightedSecurityScore(pairs.map(([a]) => a));
const target = weightedSecurityScore(pairs.map(([, b]) => b));
return baseline === undefined || target === undefined ? result(['score-unavailable']) : result([], { baseline, target });
}); });
} }
@ -176,7 +191,7 @@ export function compareTimeslots(input: {
if (input.models.some(model => model.project.id !== input.projectId)) throw new Error('Timeslots must belong to the same project.'); if (input.models.some(model => model.project.id !== input.projectId)) throw new Error('Timeslots must belong to the same project.');
const scanner: ProjectTimeslotEvidenceComparison = input.baselineRunIds.length || input.targetRunIds.length const scanner: ProjectTimeslotEvidenceComparison = input.baselineRunIds.length || input.targetRunIds.length
? compareProjectTimeslotEvidence(input.models, input.baselineRunIds, input.targetRunIds) ? compareProjectTimeslotEvidence(input.models, input.baselineRunIds, input.targetRunIds)
: { version: '1.1.0', projectId: input.projectId, baselineRunIds: [], targetRunIds: [], sources: [], summary: { : { version: '1.2.0', projectId: input.projectId, baselineRunIds: [], targetRunIds: [], sources: [], summary: {
baselineDeviations: 0, verifiedRemediated: 0, confirmedOpen: 0, unverifiedAbsent: 0, notComparable: 0, baselineDeviations: 0, verifiedRemediated: 0, confirmedOpen: 0, unverifiedAbsent: 0, notComparable: 0,
notApplicable: 0, newDeviations: 0, regressedDeviations: 0 notApplicable: 0, newDeviations: 0, regressedDeviations: 0
} }; } };
@ -184,7 +199,7 @@ export function compareTimeslots(input: {
? compareGpoTimeslots(input.gpo.baseline, input.gpo.target) : undefined; ? compareGpoTimeslots(input.gpo.baseline, input.gpo.target) : undefined;
const changes = [...scannerChanges(input.models, scanner), ...(gpo ? gpoChanges(gpo, input.gpo?.target) : [])]; const changes = [...scannerChanges(input.models, scanner), ...(gpo ? gpoChanges(gpo, input.gpo?.target) : [])];
const sourceScores = scannerScores(input.models, input.baselineRunIds, input.targetRunIds); const sourceScores = scannerScores(input.models, scanner);
if (gpo) { if (gpo) {
const policy = input.models[0]?.assessment.scoringPolicy ?? defaultSecurityScoringPolicy(); const policy = input.models[0]?.assessment.scoringPolicy ?? defaultSecurityScoringPolicy();
const rate = gpo.sharedPassRate; const rate = gpo.sharedPassRate;

View File

@ -35,6 +35,7 @@ const REASONS: Record<string, string> = {
'processing-contract-incompatible': 'Processed with different import versions', 'processing-contract-incompatible': 'Processed with different import versions',
'processing-contract-unknown': 'Import version unknown', 'processing-contract-unknown': 'Import version unknown',
'newer-observation-not-verified': 'No newer scan or backup time', 'newer-observation-not-verified': 'No newer scan or backup time',
'target-observation-not-newer': 'Timeslot B holds the same or an older report',
'no-shared-decided-settings': 'No setting decided in both timeslots', 'no-shared-decided-settings': 'No setting decided in both timeslots',
'comparison-profile-unavailable': 'Scanner not yet comparable', 'comparison-profile-unavailable': 'Scanner not yet comparable',
'comparison-profile-missing': 'Scanner not yet comparable' 'comparison-profile-missing': 'Scanner not yet comparable'
@ -62,7 +63,14 @@ function Delta({ before, after }: { before?: number; after?: number }) {
} }
function sourceRows(report: TimeslotComparisonReport) { function sourceRows(report: TimeslotComparisonReport) {
return report.sources.map(source => { // A scanner may hold several sources (for example two domains); the table shows one row per scanner.
const byScanner = new Map<string, TimeslotComparisonReport['sources'][number]>();
for (const item of report.sources) {
const known = byScanner.get(item.sourceId);
byScanner.set(item.sourceId, known ? { sourceId: item.sourceId, state: known.state === 'compared' || item.state === 'compared' ? 'compared' : 'not-comparable',
reasons: [...new Set([...known.reasons, ...item.reasons])] } : item);
}
return [...byScanner.values()].map(source => {
const changes = report.changes.filter(item => item.sourceId === source.sourceId); const changes = report.changes.filter(item => item.sourceId === source.sourceId);
const count = (change: TimeslotChange) => changes.filter(item => item.change === change).length; const count = (change: TimeslotChange) => changes.filter(item => item.change === change).length;
const score = report.score.sources.find(item => item.sourceId === source.sourceId); const score = report.score.sources.find(item => item.sourceId === source.sourceId);

View File

@ -39,11 +39,13 @@ export const pingCastleDefinition: ToolProfileDefinition = {
{ property: 'risk.domain', reason: 'Category or domain risk total, not a per-finding rank.' } { property: 'risk.domain', reason: 'Category or domain risk total, not a per-finding rank.' }
] ]
}, },
// 1.1.0 (user decision 2026-10-13): a PingCastle report lists every triggered rule of its version, so a rule
// absent from a complete report of the same domain, PingCastle version and processing counts as remediated.
timeslotComparison: { timeslotComparison: {
version: '1.0.0', kind: 'available', granularity: 'scope', version: '1.1.0', kind: 'available', granularity: 'scope',
scope: { kind: 'ad-domain', properties: ['domain.fqdn'] }, scope: { kind: 'ad-domain', properties: ['domain.fqdn'] },
observationTimeProperties: ['report.generated_at'], observationTimeProperties: ['report.generated_at'],
resultEvidence: 'findings-only', negativeEvidence: 'unsupported', resultEvidence: 'findings-only', negativeEvidence: 'complete-explicit-negative',
diagnosticPolicy: 'fail-closed-by-impact' diagnosticPolicy: 'fail-closed-by-impact'
}, },
name: 'PingCastle', name: 'PingCastle',

View File

@ -9,7 +9,7 @@ const components = readFileSync(new URL('../../src/styles/components.css', impor
describe('ASSESSOR version in the brand header', () => { describe('ASSESSOR version in the brand header', () => {
it('shows the centrally maintained product version and opens About from the brand', () => { it('shows the centrally maintained product version and opens About from the brand', () => {
expect(ASSESSOR_PRODUCT_VERSION).toBe('2.26.0'); expect(ASSESSOR_PRODUCT_VERSION).toBe('2.26.1');
expect(app).toContain("'db-structure': 'Data Flow'"); expect(app).toContain("'db-structure': 'Data Flow'");
expect(app).toContain("['db-structure', 'Data Flow']"); expect(app).toContain("['db-structure', 'Data Flow']");
expect(app).not.toContain("'DB Structure'"); expect(app).not.toContain("'DB Structure'");

View File

@ -86,8 +86,11 @@ describe('GPO timeslot comparison', () => {
it('requires a newer backup set, the same baselines and a GPO source in both timeslots', () => { it('requires a newer backup set, the same baselines and a GPO source in both timeslots', () => {
const fail = { observedAt: observedA, results: [setting('a-1', 'NoLMHash', 'Fail')] }; const fail = { observedAt: observedA, results: [setting('a-1', 'NoLMHash', 'Fail')] };
const sameTime = compareGpoTimeslots(fail, { observedAt: observedA, results: [setting('b-1', 'NoLMHash', 'Pass')] }); const sameTime = compareGpoTimeslots(fail, { observedAt: observedA, results: [setting('b-1', 'NoLMHash', 'Pass')] });
expect(sameTime.decisions[0]).toMatchObject({ state: 'unverified-absent', reasons: ['newer-observation-not-verified'] }); expect(sameTime.decisions[0]).toMatchObject({ state: 'not-comparable', reasons: ['target-observation-not-newer'] });
expect(sameTime.sharedPassRate).toBeUndefined(); expect(sameTime.sharedPassRate).toBeUndefined();
// Without stated backup times (real backups state no time zone) the timeslot order decides.
const untimed = compareGpoTimeslots({ results: fail.results }, { results: [setting('b-1', 'NoLMHash', 'Pass')] });
expect(untimed.decisions[0]).toMatchObject({ state: 'verified-remediated' });
const changedBaseline = compareGpoTimeslots(fail, { observedAt: observedB, results: [setting('b-1', 'NoLMHash', 'Pass', { baselineId: 'baseline-new' })] }); const changedBaseline = compareGpoTimeslots(fail, { observedAt: observedB, results: [setting('b-1', 'NoLMHash', 'Pass', { baselineId: 'baseline-new' })] });
expect(changedBaseline.decisions[0]).toMatchObject({ state: 'not-comparable', reasons: ['baseline-set-changed'] }); expect(changedBaseline.decisions[0]).toMatchObject({ state: 'not-comparable', reasons: ['baseline-set-changed'] });

View File

@ -107,12 +107,103 @@ describe('conservative project timeslot comparison', () => {
occurrence('b-3', 'run-b', 'RULE-3', 'failed') occurrence('b-3', 'run-b', 'RULE-3', 'failed')
]); ]);
const result = compareProjectTimeslotEvidence([baseline, target], ['run-a'], ['run-b']); const result = compareProjectTimeslotEvidence([baseline, target], ['run-a'], ['run-b']);
expect(result.version).toBe('1.1.0'); expect(result.version).toBe('1.2.0');
expect(result.sources[0].regressedDeviationOccurrenceIds).toEqual(['b-1']); expect(result.sources[0].regressedDeviationOccurrenceIds).toEqual(['b-1']);
expect(result.sources[0].newDeviationOccurrenceIds).toEqual(['b-3']); expect(result.sources[0].newDeviationOccurrenceIds).toEqual(['b-3']);
expect(result.summary).toMatchObject({ newDeviations: 1, regressedDeviations: 1, baselineDeviations: 0 }); expect(result.summary).toMatchObject({ newDeviations: 1, regressedDeviations: 1, baselineDeviations: 0 });
}); });
it('splits a run imported from several scanners by the scanner of each file', () => {
const baseline = model('run-a', observedA, [occurrence('a-1', 'run-a', 'RULE-1', 'failed')]);
const target = model('run-b', observedB, [occurrence('b-1', 'run-b', 'RULE-1', 'passed')]);
// One batch with CIS-CAT and Maester: the run is labelled 'multiple', each file keeps its scanner.
target.scanRuns[0].tool = { id: 'multiple', name: 'Multiple Tools' };
const maesterFile = { ...importFile('run-b'), id: 'file-run-b-maester', fileName: 'maester.csv',
detectedTool: { id: 'maester', name: 'Maester' }, processedWith: { ...importFile('run-b').processedWith, profileId: 'maester' } };
target.importFiles.push(maesterFile);
target.findingOccurrences.push({ ...occurrence('m-1', 'run-b', 'MT.1001', 'failed'), importFileId: maesterFile.id,
sourceIdentity: { tool: { id: 'maester', name: 'Maester' }, canonicalRuleId: 'MT.1001' } });
const result = compareProjectTimeslotEvidence([baseline, target], ['run-a'], ['run-b']);
expect(result.sources.map(source => [source.toolId, source.state])).toEqual([['cis-cat', 'compared'], ['maester', 'not-comparable']]);
expect(result.summary).toMatchObject({ verifiedRemediated: 1, newDeviations: 0 });
expect(result.sources[1]).toMatchObject({ reasons: ['baseline-source-missing'], targetFileIds: ['file-run-b-maester'] });
});
it('pairs several sources of one scanner by their verified scope', () => {
const withHost = (built: CanonicalAssessmentModel, host: string) => {
built.environmentObservations = built.environmentObservations?.map(item => item.property === 'host.name' ? { ...item, normalizedValue: host, originalValue: host } : item);
return built;
};
const merge = (left: CanonicalAssessmentModel, right: CanonicalAssessmentModel): CanonicalAssessmentModel => ({
...left, scanRuns: [...left.scanRuns, { ...right.scanRuns[0], assessmentId: left.assessment.id, sequence: 2 }],
importFiles: [...left.importFiles, ...right.importFiles], environmentObservations: [...left.environmentObservations!, ...right.environmentObservations!],
findingDefinitions: [...left.findingDefinitions, ...right.findingDefinitions], findingOccurrences: [...left.findingOccurrences, ...right.findingOccurrences]
});
const baseline = merge(withHost(model('run-a1', observedA, [occurrence('a-1', 'run-a1', 'RULE-1', 'failed')]), 'server-a'),
withHost(model('run-a2', observedA, [occurrence('a-2', 'run-a2', 'RULE-1', 'failed')]), 'server-b'));
const target = merge(withHost(model('run-b2', observedB, [occurrence('b-2', 'run-b2', 'RULE-1', 'failed')]), 'server-b'),
withHost(model('run-b1', observedB, [occurrence('b-1', 'run-b1', 'RULE-1', 'passed')]), 'server-a'));
const result = compareProjectTimeslotEvidence([baseline, target], ['run-a1', 'run-a2'], ['run-b1', 'run-b2']);
const decision = (id: string) => result.sources.flatMap(source => source.decisions).find(item => item.baselineOccurrenceId === id);
// The same rule is resolved on server A and still open on server B; never crossed between hosts.
expect(decision('a-1')).toMatchObject({ state: 'verified-remediated', targetOccurrenceId: 'b-1' });
expect(decision('a-2')).toMatchObject({ state: 'confirmed-open', targetOccurrenceId: 'b-2' });
});
it('blocks only diagnostics whose impact can reach the results', () => {
const diagnostic = (runId: string, code: string) => ({ id: `${runId}-${code}`, importFileId: `file-${runId}`, severity: 'warning' as const, code, message: 'synthetic' });
const baseline = model('run-a', observedA, [occurrence('a-1', 'run-a', 'RULE-1', 'failed')]);
const target = model('run-b', observedB, [occurrence('b-1', 'run-b', 'RULE-1', 'passed')]);
target.diagnostics = [diagnostic('run-b', 'html-active-content-removed')] as never;
expect(compareProjectTimeslotEvidence([baseline, target], ['run-a'], ['run-b']).summary.verifiedRemediated).toBe(1);
target.diagnostics = [diagnostic('run-b', 'cis-cat-summary-mismatch')] as never;
expect(compareProjectTimeslotEvidence([baseline, target], ['run-a'], ['run-b']).summary).toMatchObject({ verifiedRemediated: 0, unverifiedAbsent: 1 });
});
it('uses the timeslot order when a report states no time of its own', () => {
const withoutTime = (built: CanonicalAssessmentModel) => ({ ...built,
environmentObservations: built.environmentObservations?.filter(item => item.property !== 'report.generated_at') });
const baseline = withoutTime(model('run-a', observedA, [occurrence('a-1', 'run-a', 'RULE-1', 'failed')]));
const target = withoutTime(model('run-b', observedB, [occurrence('b-1', 'run-b', 'RULE-1', 'passed')]));
expect(compareProjectTimeslotEvidence([baseline, target], ['run-a'], ['run-b']).summary.verifiedRemediated).toBe(1);
// An older report in B stays not comparable.
const older = compareProjectTimeslotEvidence([model('run-c', observedB, [occurrence('c-1', 'run-c', 'RULE-1', 'failed')]),
model('run-d', observedA, [occurrence('d-1', 'run-d', 'RULE-1', 'passed')])], ['run-c'], ['run-d']);
expect(older.summary).toMatchObject({ verifiedRemediated: 0, notComparable: 1 });
});
it('accepts a PingCastle rule missing from a complete report of the same domain and engine version as remediated', () => {
const pingCastle = (runId: string, observedAt: string, engine: string, rules: string[]): CanonicalAssessmentModel => {
const built = model(runId, observedAt, rules.map(rule => ({ ...occurrence(`${runId}-${rule}`, runId, rule, 'failed'),
sourceIdentity: { tool: { id: 'pingcastle', name: 'PingCastle' }, canonicalRuleId: rule } })));
built.scanRuns[0].tool = { id: 'pingcastle', name: 'PingCastle' };
built.importFiles[0] = { ...built.importFiles[0], detectedTool: { id: 'pingcastle', name: 'PingCastle' },
processedWith: { ...built.importFiles[0].processedWith, profileId: 'pingcastle', compatibilityKey: 'pingcastle-canonical-v1' } };
built.environmentObservations = [
{ ...observation(runId, 'domain.fqdn', 'synthetic.example'), tool: { id: 'pingcastle', name: 'PingCastle' } },
{ ...observation(runId, 'report.engine_version', engine), tool: { id: 'pingcastle', name: 'PingCastle' } }
];
return built;
};
const baseline = pingCastle('run-a', observedA, '3.3.0', ['A-Krbtgt', 'P-AdminLogin']);
const same = compareProjectTimeslotEvidence([baseline, pingCastle('run-b', observedB, '3.3.0', ['P-AdminLogin'])], ['run-a'], ['run-b']);
expect(same.summary).toMatchObject({ verifiedRemediated: 1, confirmedOpen: 1, unverifiedAbsent: 0 });
// Another PingCastle version may have renamed or dropped the rule: absence stays unverified.
const other = compareProjectTimeslotEvidence([baseline, pingCastle('run-c', observedB, '3.4.0', ['P-AdminLogin'])], ['run-a'], ['run-c']);
expect(other.summary).toMatchObject({ verifiedRemediated: 0, confirmedOpen: 1, unverifiedAbsent: 1 });
});
it('uses the client as scope when each timeslot holds one report that states no scope', () => {
const withoutScope = (built: CanonicalAssessmentModel) => ({ ...built,
environmentObservations: built.environmentObservations?.filter(item => item.property !== 'host.name') });
const baseline = withoutScope(model('run-a', observedA, [occurrence('a-1', 'run-a', 'RULE-1', 'failed')]));
const target = withoutScope(model('run-b', observedB, [occurrence('b-1', 'run-b', 'RULE-1', 'passed')]));
expect(compareProjectTimeslotEvidence([baseline, target], ['run-a'], ['run-b']).summary.verifiedRemediated).toBe(1);
// A scope stated on one side only never pairs.
const stated = model('run-c', observedB, [occurrence('c-1', 'run-c', 'RULE-1', 'passed')]);
expect(compareProjectTimeslotEvidence([baseline, stated], ['run-a'], ['run-c']).summary).toMatchObject({ verifiedRemediated: 0, notComparable: 1 });
});
it('reports a source of only one timeslot as not comparable instead of failing', () => { it('reports a source of only one timeslot as not comparable instead of failing', () => {
const target = model('run-b', observedB, [occurrence('b-1', 'run-b', 'RULE-1', 'failed')]); const target = model('run-b', observedB, [occurrence('b-1', 'run-b', 'RULE-1', 'failed')]);
const result = compareProjectTimeslotEvidence([target], [], ['run-b']); const result = compareProjectTimeslotEvidence([target], [], ['run-b']);
@ -132,7 +223,9 @@ describe('conservative project timeslot comparison', () => {
const target = model('run-b', observedA, [occurrence('b-1', 'run-b', 'RULE-1', 'passed')]); const target = model('run-b', observedA, [occurrence('b-1', 'run-b', 'RULE-1', 'passed')]);
const sameTime = compareProjectTimeslotEvidence([baseline, target], ['run-a'], ['run-b']); const sameTime = compareProjectTimeslotEvidence([baseline, target], ['run-a'], ['run-b']);
expect(sameTime.summary).toMatchObject({ verifiedRemediated: 0, unverifiedAbsent: 1 }); // The same report again (unchanged report time) is never progress (user decision 2026-10-13: not comparable).
expect(sameTime.summary).toMatchObject({ verifiedRemediated: 0, notComparable: 1 });
expect(sameTime.sources[0].reasons).toEqual(['target-observation-not-newer']);
}); });
it('blocks scope changes and incompatible processing before lifecycle claims', () => { it('blocks scope changes and incompatible processing before lifecycle claims', () => {

View File

@ -83,7 +83,7 @@ function setting(id: string, identity: string, status: PolicyWorkStatus, actual?
}; };
} }
function report(reversed = false) { function reportInput(reversed = false) {
const order = <T,>(items: T[]) => reversed ? [...items].reverse() : items; const order = <T,>(items: T[]) => reversed ? [...items].reverse() : items;
const a = model('a', [{ runId: 'cis-a', toolId: 'cis-cat', findings: [ const a = model('a', [{ runId: 'cis-a', toolId: 'cis-cat', findings: [
occurrence('a-1', 'cis-a', 'cis-cat', '1.1.1', 'failed'), occurrence('a-1', 'cis-a', 'cis-cat', '1.1.1', 'failed'),
@ -98,14 +98,15 @@ function report(reversed = false) {
] }, ] },
{ runId: 'maester-b', toolId: 'maester', findings: [occurrence('m-1', 'maester-b', 'maester', 'MT.1001', 'failed')] } { runId: 'maester-b', toolId: 'maester', findings: [occurrence('m-1', 'maester-b', 'maester', 'MT.1001', 'failed')] }
]); ]);
return compareTimeslots({ return {
projectId: 'dummy-project', models: order([a, b]), baselineRunIds: ['cis-a'], targetRunIds: order(['cis-b', 'maester-b']), projectId: 'dummy-project', models: order([a, b]), baselineRunIds: ['cis-a'], targetRunIds: order(['cis-b', 'maester-b']),
gpo: { gpo: {
baseline: { observedAt: observedA, results: order([setting('ga-1', 'MinimumPasswordLength', 'Fail', 7), setting('ga-2', 'NoLMHash', 'Fail', 0)]) }, baseline: { observedAt: observedA, results: order([setting('ga-1', 'MinimumPasswordLength', 'Fail', 7), setting('ga-2', 'NoLMHash', 'Fail', 0)]) },
target: { observedAt: observedB, results: order([setting('gb-1', 'MinimumPasswordLength', 'Fail', 10), setting('gb-2', 'NoLMHash', 'Pass', 1)]) } target: { observedAt: observedB, results: order([setting('gb-1', 'MinimumPasswordLength', 'Fail', 10), setting('gb-2', 'NoLMHash', 'Pass', 1)]) }
} }
}); };
} }
const report = (reversed = false) => compareTimeslots(reportInput(reversed));
describe('Timeslot comparison report', () => { describe('Timeslot comparison report', () => {
it('assigns every change category from scanner evidence and rolled-up GPO verdicts', () => { it('assigns every change category from scanner evidence and rolled-up GPO verdicts', () => {
@ -135,6 +136,14 @@ describe('Timeslot comparison report', () => {
expect(result.score.targetScore).toBeCloseTo(scored.reduce((sum, item) => sum + item.targetScore! * item.weight, 0) / weight); expect(result.score.targetScore).toBeCloseTo(scored.reduce((sum, item) => sum + item.targetScore! * item.weight, 0) / weight);
}); });
it('keeps the score delta for report times without an ISO clock time, as CIS-CAT writes them', () => {
const result = compareTimeslots({ ...reportInput(), models: reportInput().models.map(model => ({ ...model,
environmentObservations: model.environmentObservations?.map(item => item.property === 'report.generated_at'
? { ...item, normalizedValue: model.assessment.id.endsWith('a') ? 'Thursday, January 1 2026 10:00:00' : 'Sunday, February 1 2026 10:00:00' } : item) })) });
expect(result.score.sources.find(item => item.sourceId === 'cis-cat')).toMatchObject({ change: 0 });
expect(result.summary.resolved).toBe(2);
});
it('is deterministic and independent of the input order', () => { it('is deterministic and independent of the input order', () => {
const first = report(); const first = report();
const reversed = report(true); const reversed = report(true);

View File

@ -60,7 +60,7 @@ describe('local ASSESSOR HTTP server', () => {
const health = await fetch(`${baseUrl}/api/v1/health`); const health = await fetch(`${baseUrl}/api/v1/health`);
expect(await health.json()).toEqual({ expect(await health.json()).toEqual({
status: 'ok', application: 'ASSESSOR', productVersion: '2.26.0', apiVersion: 1, repositoryRevision: 0, persistence: 'memory' status: 'ok', application: 'ASSESSOR', productVersion: '2.26.1', apiVersion: 1, repositoryRevision: 0, persistence: 'memory'
}); });
expect(health.headers.get('access-control-allow-origin')).toBeNull(); expect(health.headers.get('access-control-allow-origin')).toBeNull();
}); });