Logs, metrics and traces: what each signal answers

Each telemetry signal answers a different question at a different cost. Understanding what each one provides and how they correlate is what turns scattered data into diagnosis.

When an organization decides to "get observability," it usually starts with the tool: buy a platform, install agents, turn on dashboards. It rarely starts with the most useful question: what do we need to know when something breaks? Logs, metrics and traces are not three ways of storing the same thing. They are three instruments that answer different questions, with different costs and limits. Using the wrong instrument for a question produces full dashboards and slow diagnosis.

This article offers a practical way to think about the three signals: which question each one answers, what it costs to produce and keep, when it is the right tool, and why the real value appears when all three are correlated.

Three signals, three questions

A simple way to frame the discussion is to map each signal to the question it answers best during an incident:

Metrics: is something wrong, how much, and since when?
Traces: where, along the path of a request, is time lost or does the error appear?
Logs: what exactly happened at that point, with what data, and why?

The order matters. In a typical incident, a metric fires the alert (payment service latency crosses its threshold), a trace pinpoints the responsible segment (the call to the fraud service takes five times longer than usual) and a log explains the cause (the fraud service keeps retrying against a database that refuses connections). If any of these steps is missing, the team falls back to manual searching, which is exactly what observability should prevent.

Metrics: the aggregate pulse

A metric is a numeric value measured over time and aggregated: requests per second, error percentage, latency percentiles, CPU usage, queue depth. Its strength is compression. A request counter takes the same space whether the service handled ten requests or ten million in that interval. That is why metrics are cheap to keep for months, fast to query, and the natural basis for alerts, service level objectives (SLOs) and capacity planning.

Its weakness is the same compression. A metric tells you, for example, that 2% of requests fail, but not which ones or why. You can break it down with labels (service, region, method, response code), and this is where the limit worth understanding appears: cardinality. Every distinct combination of label values creates a new time series. Adding a label with a customer or transaction ID multiplies the series until the metric becomes expensive and unstable. Metrics are for bounded dimensions known in advance.

A proven starting point is the four golden signals described in Google's SRE book (latency, traffic, errors and saturation), or its service oriented variant (rate, errors and duration). They do not cover everything, but they force you to measure first what the user perceives.

Traces: the path of a request

A distributed trace reconstructs the path of a request through the services it touches. It is made of spans: each span represents one operation (an HTTP call, a database query, the processing of a message) with its start, duration, outcome and attributes. Spans share a trace ID and are linked through parent and child relationships.

The trace answers the question that neither metrics nor logs handle well in distributed architectures: which segment holds the problem? When a business operation crosses an API, three microservices, a queue and two databases, total latency says nothing about where it builds up. The trace does.

The cost of traces lies in volume: each request can produce dozens of spans. That is why almost every implementation uses sampling. Head sampling (deciding at the start whether to keep the trace) is simple and cheap, but it can drop precisely the trace that failed. Tail sampling (deciding at the end, with the full trace) lets you keep every trace with errors or high latency, at the price of more infrastructure to hold traces while the decision is made. Choosing a sampling strategy is a diagnostic decision, not just a budget one.

Logs: the detail of the event

A log is the record of a discrete event with context: what happened, when, in which component and with what data. It is the oldest and most flexible signal, because it can carry any information the developer chooses to write. It is also the most expensive in volume and the easiest to misuse.

The log answers the "why" that the other signals only hint at: the error message from the external provider, the business rule that rejected the operation, the unexpected value that arrived in a field. To be useful at scale, it should meet three conditions:

Structured: in key and value format (usually JSON), so it can be filtered and aggregated without fragile regular expressions.
Leveled and purposeful: levels (error, warning, info, debug) must mean something and be used consistently across teams.
Correlated: every line should carry the active trace and span IDs, so you can jump from the log to the trace and back.

A common mistake is using logs as a substitute for metrics: counting log lines to compute error rates. It works, but it is expensive and fragile, because any change in the message text breaks the indicator. If a number is checked every day or triggers alerts, it should be a metric.

Cardinality, volume and cost

The three signals have very different cost profiles, and understanding them avoids surprises both in the bill and in platform performance. The table summarizes the main trade-offs.

Signal	Question it answers	Tolerated cardinality	Main cost driver	Typical retention
Metrics	Is there a problem, how big, since when?	Low: bounded, known labels	Number of time series	Long (months), with aggregation
Traces	Which segment holds the problem?	High: per request attributes	Number of stored spans	Short or medium, with sampling
Logs	What exactly happened and why?	Very high: free text and fields	Volume ingested and indexed	Variable, by use and regulatory requirement

A few practical rules follow from the table. High cardinality data (customer, transaction or session IDs) belongs in span attributes and log fields, not in metric labels. Metrics are designed around recurring questions; traces and logs, around the questions you do not yet know you will ask. And retention is defined per signal: there is no point keeping every debug log as long as the records an audit requires.

In regulated industries, log retention is not only a cost decision. Some records carry their own retention and integrity requirements, and it pays to separate them by design from operational logs, whose value drops quickly after a few days.

Correlation turns signals into diagnosis

Having all three signals guarantees nothing if they live in silos. The most time consuming scenario in an incident is a team that sees the alert in one dashboard, searches another tool for logs from the same minute, filters by server and tries to guess which ones belong to the affected requests. Correlation removes the guesswork.

Correlating means the signals share identifiers and context:

Logs carry trace_id and span_id, so from a slow trace you reach the exact logs of that request.
Metrics and traces share the same resource attributes (service name, version, environment, host), so from a spike in a dashboard you reach example traces from the same service and moment.
Trace context propagates across services (for example, with the W3C Trace Context standard in HTTP headers), so the trace does not break at every hop.

OpenTelemetry, the open project of the Cloud Native Computing Foundation, was designed around this idea: a common model of context and resource attributes for all three signals, with semantic conventions that define how to name common attributes. That is why it makes sense to adopt it as the basis for instrumentation even when the analysis platform is a different one.

An isolated signal describes a symptom. Three correlated signals describe an incident.

How to decide which signal to use

For each visibility need, we recommend walking through these questions in order:

Does the question recur and need an alert or a trend? Use a metric, with low cardinality labels.
Is the question about where time goes or where an error starts in a flow that crosses components? Use traces, with attributes that identify the business operation.
Do you need the detail of a specific event, with its data and cause? Use a structured log, correlated with the active trace.
Do you need to jump from one signal to another? Check that they share identifiers and resource attributes. If not, fix that before adding more data.
Is the data sensitive or high cardinality? Decide where it lives, how long it is kept and who can see it before you emit it.

This sequence avoids the two usual extremes: instrumenting everything "just in case," with costs that grow unchecked, or instrumenting only what the platform captures on its own, leaving gaps exactly where the business rules live. The goal is not to accumulate telemetry but to shorten the time between "something is wrong" and "we know why."