📡

Section 19.4

AI-Assisted Operations and Transformation

Anomaly detection, correlation and incident summarisation as AI use-cases, and why observability has to be designed into a transformation rather than added after go-live.

What AI can do with the signals

AI in operations as a consumer of the signals, with a human or a closed loop at the end — and its own outputs fed back in as telemetry. The feedback arrow is the part that gets left out.

AI in operations is only as good as the telemetry beneath it. Every capability below reads metrics, logs, traces and inventory; none of them works on a green/red dashboard. The AI use-cases in the Academy — AI-driven service assurance, closed-loop self-healing — sit on top of this list.

Capability	What it does	What it needs	Where it fails
Anomaly detection	Flags a metric behaving unlike its own history — order latency up 40% for one product	Long, clean metric history; seasonality (bill runs, launches)	Alert fatigue: every change is an anomaly
Event correlation	Groups alarms and errors into one probable incident	Topology and inventory; timestamps that agree	Correlates by time alone and joins unrelated faults
Root-cause suggestion	Ranks likely origins from the trace and change history	Traces that join across systems; a deployment log	Blames the last deploy, every time
Incident summarisation	Writes the timeline and impact statement from the evidence	Logs with order and customer ids; the ticket	Confidently summarises the wrong incident
Predictive detection	Warns before an SLO burns — disk, queue depth, fallout rate trends	SLOs defined; the metrics behind them	Predicts from a metric nobody acts on
Remediation recommendation	Proposes the runbook step; in a closed loop, executes it	Idempotent, safe actions; an approval gate	An automated fix applied to a mis-diagnosed cause

Observe the AI, too

An AI that triages incidents is itself a system in the chain. Its recommendations, its confidence, what it changed and whether the change helped are signals to collect. Without that, the first wrong automated remediation is also the last time anyone trusts it.

From detection to action

Everything above is about seeing. This is about what happens next: how one alert leaves the observability platform and becomes an incident that a named team is working. The mechanics are simple — a webhook and four processing stages — and every stage depends on the CMDB knowing which service the failing thing belongs to.

One alert, from the platform to a person with a ticket. Step four is opened up because that is where telemetry stops being telemetry: the ITSM platform’s event management, correlation, rules and incident stages.

A SOM slowdown, end to end

Monitor

Observability platform

The platform — a Grafana stack here; the flow is the same for any of the four — collects metrics from SOM, traces across COM → SOM → ROM, and logs from the activation gateway.

Detect

Alert rules

An alert rule fires: p95 latency on the TMF641 service-order API above 5 seconds for 10 minutes. The threshold came from the order-completion SLO, not from a CPU graph.

Send

Integration

A webhook posts JSON to the ITSM platform: alert name, the configuration item, severity, the trace id, a link back to the dashboard. Authenticated by API key or OAuth; nothing else crosses.

Event management

ITSM — event management

The event is received and validated. Duplicates from the same rule are collapsed. It is now an event with a source, a CI and a time — not yet an incident.

Correlation and enrichment

ITSM — CMDB

The CI is matched in the CMDB: this SOM node serves the consumer broadband fulfilment service, owned by the fulfilment platform team. Two open network events on the same service are linked.

Rules: impact, priority, deduplication

ITSM — rules

Impact is high — a customer-facing service — and urgency is high because orders are in flight. Priority 2. An open incident on the same service already exists: this event attaches to it rather than creating a second.

Incident and action

Resolver group

The resolver group is paged with the trace attached. They find the activation gateway certificate renewal in progress, roll it back, and close the incident against the evidence — no CSV export, no four-team call.

Where the value actually is

Not in the webhook. Steps 5 and 6 — matching the CI to the service and the owner, and deciding that this is the same incident, not a new one — are what turn technical data into operational insight. Both read the CMDB. A CMDB that does not know which service a SOM node serves produces a Priority 4 ticket for a customer-facing outage.

This is the front half of Trouble-to-Resolve

Detection to incident is the automated version of the first steps in T2R: problem detection, isolation and ticketing. The service assurance module covers the same flow from a network alarm; this section covers it from an application signal. Both should land in the same incident queue.

The operating model: a centre of excellence with federated teams

One platform and one CMDB do not, on their own, end the fragmentation described in 19.1. The departments that chose their own tools did so because they own different estates — the network, the IT platforms, the digital channels, corporate systems — and that ownership is legitimate. The model that works keeps it, and adds a small central function that owns what has to be common.

The centre of excellence owns	The departments keep
Standards: what every system must emit, the identifiers it must carry, the alert-rule conventions	Instrumenting their own systems to those standards
Architecture: the platform, the integration to ITSM, the CMDB model	Their dashboards, their alert rules, their runbooks
Service mapping: the CMDB’s technology → service → owner model and its reconciliation to inventory	Keeping their CIs and services current in it
Automation: the correlation and enrichment rules, the closed-loop actions that are allowed	Deciding which of their incidents may be auto-remediated
Governance and reporting: SLO definitions, the service reports the business sees	Their operational reporting beneath those
Continuous improvement: the retrospectives that change the standards	Attending them with evidence

The ownership question nobody wants to answer

When a fault crosses the line between the network operations centre and IT operations — a SOM timeout caused by an activation gateway certificate, say — whose incident is it? In the fragmented state each side has evidence that the other is at fault and no shared timeline. The CoE does not resolve this by owning the incident. It resolves it by owning the CMDB entry that says which service the gateway belongs to, so the rule in step 6 assigns the incident before the argument starts.

Architecture decision

Centre-led, federated, or tool-first?

CoE with federated teams
Choose when several departments own real estates and will not give them up; the CoE is small, owns standards and the service model, and has authority over what must be common.
Fully centralised
Choose when the operator is small enough that one platform team can instrument everything — rare beyond a few hundred systems.
Tool-first
Choose when never as a strategy. Buying the platform before deciding who owns the service model is how an operator arrives at a very expensive version of the fragmented state.

Breaks if the CoE becomes a tooling team: it runs the platform, nobody owns the CMDB model, and every department reverts to its own console within a year.

What “better” has to mean — outcomes as measurables

Claimed outcome	The measure that proves it
Proactive detection	Share of incidents detected from signals before a customer or user reported them
Reduced downtime	Mean time to detect and mean time to resolve, per service, trended
Improved service reliability	SLO attainment per customer-facing service, with the error budget spent shown
Operational efficiency	Incidents created automatically versus by hand; events per incident after correlation
Clear ownership	Incidents reassigned after creation — the count that should fall

Transformation implications

Transformation reality	Why	What to do about it
Observability cannot be added after go-live	Trace propagation and shared ids are interface design; retrofitting them means changing every integration	Put trace and order-id propagation in the integration standards before the first interface is built
Coexistence doubles the surface	Two stacks, one order journey — the trace has to cross the legacy boundary too	Correlate on the order id across the anti-corruption layer; accept timing gaps
Vendor packages emit what they emit	Not every product supports OpenTelemetry	Make it a procurement requirement; where absent, log the order id and settle for less
Migration is the first real load test	Bulk loads run through the same order APIs at ten times the rate	SLOs and dashboards in place before migration, not after
The hypercare period is short	The programme team leaves; operations inherits the signals	Runbooks written against the traces and SLOs, not against the programme’s tribal knowledge

Scenario: every system green

Six weeks after go-live

Care calls rise on Monday

Care

Customers who ordered on Friday have no service. Every system dashboard is green. The programme team has rolled off.

Four teams, four consoles

Operations

CRM shows orders submitted. COM shows them in progress. SOM’s console shows 300 tasks “awaiting”. Nobody can join the three views by customer.

The cause is found by hand

Engineering

A support engineer greps SOM logs for one customer’s order id and finds a timeout to ROM — a certificate on the activation gateway expired on Saturday.

Impact is estimated, not known

Operations

Without a service-order-to-customer join, the list of affected customers is built from a spreadsheet export.

The retrospective

Governance

Trace propagation across COM → SOM → ROM, a fallout state that pages, and one SLO on order completion. All of it was in the original design and cut in month nine to make the date.

Architecture decision

When the programme is late, is observability the thing to descope?

Descope it
Choose when never, as a matter of principle — but honestly: when the scope is a dashboard product, not the identifiers and the fallout state. Tools can follow; design cannot.
Keep the design, defer the tooling
Choose when the date is immovable. Trace ids, order ids in every log, a fallout state and one SLO cost days; the platform can arrive in hypercare.

Breaks if the identifiers are descoped with the tooling: the operator goes live blind, and the first incident is diagnosed by grep.

Key Takeaways

AI in operations reads the same signals; without correlated telemetry it has nothing to reason over.
Identifiers and fallout states are design; dashboards are tooling. Descope the second if you must, never the first.
Migration and hypercare are when observability earns its cost — which is why it must exist before them. See the readiness assessment.
An alert becomes an incident through event management, CMDB enrichment and rules — the value is in the CMDB, not the webhook. A centre of excellence owns the standards and the service model; the departments keep their estates.