KPIs and Metrics - Monitoring and Operations
Scope and purpose
The indicators of this domain assess the operational effectiveness of the security monitoring programme: the coverage of centralised logging, the quality of the alerts, the speed of detection and response, and the capacity for correlation between sources. This domain operates on data produced by all the other chapters - the logs generated by the code (Ch. 06), by the pipeline (Ch. 07), by the containers (Ch. 09) and by the infrastructure (Ch. 08) converge here and are assessed for their operational usefulness.
This file is the canonical catalogue of
OPS-Kxx. The file07-metricas-indicadores.mdis a lighter operational synthesis for day-to-day use.
The central distinction of this domain: the LOG- requirements (Ch. 02) define what the software must record; the OPS-K indicators measure whether those records are centralised, analysed and acted upon effectively. A system with perfect logging but no correlation, alerts or response has zero visibility from an operational point of view.
The OPS indicators feed the cross-cutting dimension T-03 (Resolution speed) - specifically operational MTTD and MTTR.
Denominator and portfolio foundation
The indicators of this domain use as their denominator F-02 - applications with a formal risk classification (Ch. 01, CLA-K01). The percentages are interpretable only in relation to the set of applications classified at the relevant risk level - not in relation to the total portfolio or to ad-hoc subsets.
See kpis-governanca.md - section "Portfolio foundation" - for the SbD-ToE adoptability funnel and the relation between denominators.
Conventions
| Symbol | Meaning |
|---|---|
| ✔ | Mandatory threshold for this level |
| - | Not applicable or not mandatory at this level |
| ↓ | Inverse indicator - a lower value indicates better performance |
Thresholds are cumulative: L3 includes all the obligations of L1 and L2.
Indicator types:
| Code | Meaning |
|---|---|
| Q% | Quantitative, percentage |
| Q# | Quantitative, count |
| Qt | Quantitative, temporal (hours/minutes) |
Indicator catalogue
| ID | Indicator | Type | L1 | L2 | L3 | Dim. T | Period |
|---|---|---|---|---|---|---|---|
| OPS-K01 | % of production applications with security event logging centralised in a SIEM or equivalent | Q% | ≥ 60% | ≥ 90% | 100% | T-03 | Monthly |
| OPS-K02 | % of security alerts with a defined, tested and documented threshold (no default or uncalibrated alerts) | Q% | ≥ 50% | ≥ 80% | ≥ 95% | T-03 | Quarterly |
| OPS-K03 | MTTD - mean time from the occurrence of a security event to the generation of an alert | Qt | ≤ 24h | ≤ 4h | ≤ 30min | T-03 | Monthly |
| OPS-K04 | MTTR - mean time from the generation of an alert to the start of a mitigation action | Qt | ≤ 8h | ≤ 4h | ≤ 1h | T-03 | Monthly |
| OPS-K05 | % of triggered alerts that result in a false positive (noise rate) | Q% ↓ | ≤ 50% | ≤ 30% | ≤ 15% | T-03 | Monthly |
| OPS-K06 | % of critical security events with correlation coverage across at least two distinct sources | Q% | - | ≥ 70% | ≥ 95% | T-03 | Quarterly |
| OPS-K07 | % of security incidents with a formal response process (IRP) activated and traceable | Q% | ≥ 80% | 100% | 100% | T-03 | Per incident |
| OPS-K08 | % of L2/L3 system logs retained within the minimum period mandated by policy and applicable regulation | Q% | ≥ 80% | 100% | 100% | T-03 | Half-yearly |
Complementary definitions
OPS-K01 - Centralised logging: logging is considered centralised when security events are sent to a central collection system (SIEM, log aggregation platform) with retention, indexing and query capability. Local logging without forwarding does not satisfy this criterion.
OPS-K02 - Tested alert: an alert is tested when there is evidence that its threshold was validated in a real or simulated environment and that it produces the expected behaviour (a true-positive alert with sufficient context for triage). Alerts generated by tool defaults without review and calibration do not satisfy this criterion.
OPS-K03 - MTTD (Mean Time to Detect): measured as timestamp_alerta − timestamp_evento. For events that do not generate an immediate alert (detection by correlation or periodic analysis), the detection timestamp is the moment at which an analyst becomes aware of the event. Events without a recorded occurrence timestamp cannot be included in the calculation - their absence is in itself an indicator of a logging gap.
OPS-K04 - MTTR (Mean Time to Respond): measured as timestamp_primeira_acção − timestamp_alerta. The first action includes: isolation, blocking, formal escalation, or runbook activation. Analysis time before the first action counts towards MTTR. When the incident follows the regulatory track, escalation to GRC/Compliance and the DPO takes place within ≤ 1 h of confirmation, at any level (Policy 32 §4.1).
OPS-K05 - Noise rate: a false positive is an alert that, after formal triage by an analyst, is classified as irrelevant to the security context. A high noise rate has two effects: (a) it consumes response capacity; (b) it creates desensitisation to alerts, increasing the risk of ignoring true positives. Alert calibration is the control mechanism.
OPS-K06 - Correlation between sources: critical events (anomalous authentication, data exfiltration, privilege escalation) gain context and reduce false positives when correlated across, for example: authentication logs + network logs + application logs. The minimum coverage for L3 must include correlation between endpoint, network and application.
OPS-K07 - IRP activated: the formal incident response process (Incident Response Plan) is activated when there is a ticket or record with: the classification of the incident, a response owner, a timeline of actions, and a resolution state. Incidents resolved informally without a record do not satisfy this criterion.
OPS-K08 - Retention in line with regulation: the applicable minimum period depends on sectoral regulation. DORA: the entity defines the retention period based on the ICT risk assessment (Delegated Regulation (EU) 2024/1774, Art. 12); NIS2: for entities covered by Implementing Regulation (EU) 2024/2690, a «predefined period» is required (Annex, point 3.2.5), with no numerical value; the national transposing legislation should be checked. Systems without a regulatory classification apply the internal policy — as the Manual's choice, a minimum of 90 days for L1, 1 year for L2 and 2 years for L3.
MTTD/MTTR thresholds by level - summary
| Metric | L1 | L2 | L3 |
|---|---|---|---|
| MTTD (detection) | ≤ 24h | ≤ 4h | ≤ 30min |
| MTTR (response) | ≤ 8h | ≤ 4h | ≤ 1h |
These thresholds apply to security events of high/critical severity. For events of medium severity, the thresholds may be 3× higher. Regulated sectors (financial, health, critical infrastructure) may have more demanding requirements under sectoral rules.
Collection and instrumentation
| Indicator | Primary source | Reference tools | Automation |
|---|---|---|---|
| OPS-K01 | Application inventory + SIEM configuration | SIEM (Splunk, Elastic, Sentinel, QRadar) | Partial |
| OPS-K02 | Alert register with calibration metadata | SIEM + alert documentation | No |
| OPS-K03 | Event log + alert log with timestamps | SIEM (automatic MTTD calculation) | Yes |
| OPS-K04 | Alert log + response action log | SIEM + IR system (PagerDuty, JIRA) | Yes |
| OPS-K05 | IR platform with triage classification | SIEM + triage documented by an analyst | Partial |
| OPS-K06 | Correlation rule configuration in the SIEM | SIEM (count of rules with multiple sources) | Partial |
| OPS-K07 | Incident database | IR system (ServiceNow, Jira, PagerDuty) | Yes |
| OPS-K08 | Retention configuration + system inventory | Retention policy audit per source | Partial |
Cross-references
| Document | Relation |
|---|---|
addon/00-catalogo-requisitos.md | OPS-001..015 catalogue that frames the operational indicators |
addon/07-metricas-indicadores.md | Operational synthesis of MTTD/MTTR for the teams' day-to-day use |
addon/10-excecoes-operacoes.md | Exceptions to alerts and retention (OPS-K02, OPS-K08) |
Ch. 14 kpis-governanca.md | Cross-cutting dimension T-03 (resolution speed) |