Insights

An observable platform is not an observable application

An APM platform sees infrastructure, dependencies and most calls. It cannot see what your business means. Finding root cause fast requires software written to be observable.

There is a conversation that repeats itself in many technology departments. The organization invested in a top tier APM platform, agents are deployed, dashboards show service maps and anomaly detection works. And yet, when an incident affects customers, diagnosis still takes hours, ends up in a room with five teams and gets resolved when someone who knows the code reads logs by hand.

The platform is not failing. It is doing exactly what it can do. The problem is a deeper confusion: believing that an observable platform makes the application running on top of it observable. It does not, and understanding why is the difference between buying visibility and having it.

What an APM platform sees on its own

Today's APM platforms capture a remarkable amount of information without the development team writing a single line. Dynatrace, with its OneAgent, and Datadog, with its agent and tracing libraries, automatically instrument the most common runtimes. In general terms, without changing the code you get:

  • Infrastructure: CPU, memory, disk, network, containers, processes and orchestrator events.
  • Auto-instrumented spans: incoming and outgoing HTTP requests, database calls, queues and well known frameworks, with their duration and result code.
  • Dependencies: which service calls which, which database each one uses, how the whole system connects.
  • Technical health signals: HTTP error rate, unhandled exceptions, latency per endpoint, resource saturation.

This is a lot, and it is enormously valuable. Before these tools, building the dependency map of a distributed application was a project in itself. Today the platform discovers it. Detecting that a service is saturated, that a SQL query became slow or that a deployment triggered a wave of 500 errors is, to a large extent, a solved problem.

We call this an observable platform: the infrastructure and the technical mechanics of the application are visible.

What no platform can guess

The limit shows up when the question stops being technical and becomes a business question. An agent sees that a function took 300 milliseconds and returned a 200 code. It cannot know that the 200 response carried a credit rejection caused by a misconfigured rule, that the transfer was left pending instead of confirmed, or that the affected customer is a merchant whose daily close depends on that operation.

Four kinds of information exist only inside the application:

  • Business context: which operation was running (an account opening, a payment, a policy issuance), for which product, through which channel, at which step of the flow.
  • Domain errors: failures that are not exceptions. Insufficient funds, a regulatory validation that does not pass, a provider that answers "under review." To the platform, all of these are successful responses.
  • Meaningful events: the milestones the business understands. "Payment authorized," "payment reconciled," "payment reversed." Without them, there are only HTTP calls.
  • Context continuity: when an operation crosses a queue, a batch process or a legacy system without automatic instrumentation, the trace breaks unless the code propagates context explicitly.

No anomaly detection, however good, can infer meaning that was never emitted. The platform's intelligence works on the data it receives. If the data describes only mechanics, the diagnosis will be mechanical.

The platform tells you which component is suffering. Only the application can tell you what happened to the customer.

One incident seen from both sides

An illustrative example helps show the difference. Suppose that on a Monday morning complaints start growing because transfers to other banks "are not arriving." No dashboard is red.

With an observable platform and an opaque application, the platform shows that the transfer service responds quickly and without errors. The queue to the clearing system has normal volume. Operations escalates to development. Development asks for logs, which are free text with no transaction identifier. Someone correlates by hand using time and amount. Hours later, the team finds that a change to a parameter table leaves one type of transfer in a "held for review" state without notifying anyone. Technically, everything worked.

With an observable platform and an observable application, the service emits a metric of transfers by final state. The dashboard shows that "held" transfers went from almost zero to a significant share starting at a specific time. Spans carry attributes such as transfer type and applied rule, and structured logs carry the trace_id. Within minutes it is clear that every held transfer shares the same rule, and the timeline matches the parameter change. The platform is the same. What changed is what the application tells it.

What it means to write software to be observable

An observable application is not an application that writes a lot of logs. It is an application that deliberately and consistently emits the information needed to explain its behavior. In practice, this comes down to five engineering habits:

  1. Structured logs: in JSON or equivalent, with stable fields (service, operation, outcome, domain error code), not sentences that change with every release.
  2. Correlation identifiers: every log carries the active trace_id and span_id and, where appropriate, a non sensitive business identifier (for example, an internal operation ID).
  3. Context propagation: trace context travels in HTTP headers, in queue message metadata and in asynchronous processes, using the W3C Trace Context standard.
  4. Business metrics: counters and histograms of what matters to the business (operations by state, aggregated amounts processed per channel, time per flow step), with low cardinality labels.
  5. Meaningful events and domain errors: custom spans for the key steps of the flow, with attributes that describe the decision made, and an error status when the business outcome is a failure even if the technical response succeeded.

OpenTelemetry is the reference open standard for all of this, and the main APM platforms, Dynatrace and Datadog included, accept OpenTelemetry data. That means the effort of instrumenting the code is not tied to one tool, and the platform you already have can combine what it captures on its own with what your application contributes.

Who provides what

Separating responsibilities avoids two symmetrical mistakes: asking the platform for what it cannot give, and asking development to re-instrument what the platform already handles.

NeedCovered by the APM platformMust come from the application
Health of hosts, containers and processesYes, automaticallyNo
Dependency map across servicesYes, for supported protocols and frameworksOnly for unsupported segments (custom queues, batch processes)
Technical latency and errors per endpointYesNo
Which business operation ran and with what outcomeNoYes: span attributes and events
Domain errors behind a successful technical responseNoYes: error status and domain code
Real time business indicatorsNoYes: custom metrics
Correlation between logs and tracesPartial, depending on language and configurationYes: structured logs with trace_id

How to start without rewriting the application

Making an existing application observable does not require a rewrite. It requires focus. We recommend this sequence:

  1. Pick the critical flows: two or three business operations whose failure has the greatest impact on customers, the SLA or compliance.
  2. Define the questions: for each flow, write down what you would need to know during an incident. "How many operations ended in each state?", "at which step did they stop?", "which rule rejected them?"
  3. Instrument the minimum to answer them: one metric per final state, spans on the key steps, non sensitive business attributes and structured logs with trace_id.
  4. Close the context gaps: identify where the trace breaks (queues, batch processes, legacy integrations) and propagate context there.
  5. Make it part of the development standard: observability belongs in the definition of done and in code review, not in a later operations task.

Measuring progress is also simple: mean time to identify the cause in real incidents, and the share of incidents in which the cause was found without reading logs by hand.

The APM platform remains indispensable. What changes is the expectation. The platform makes the ground observable; the development team makes the business running on it observable. When both do their part, root cause stops being an investigation and becomes a query.

Sources

Want to apply this to your operation?

Request an assessment