Insights

How to choose an APM platform without vendor lock-in

Vendor neutral criteria to evaluate an APM platform: coverage, OpenTelemetry support, cost, root cause, access control, data residency and ITSM integration, without getting locked into one tool.

Choosing an APM platform is usually a multi-year decision. Once agents are deployed, dashboards built, alerts defined and teams trained, switching tools is expensive. That is why the evaluation deserves more rigor than a well rehearsed demo and a feature comparison in a spreadsheet.

This article proposes neutral criteria to evaluate any platform, whether Dynatrace, Datadog or another, and a strategy so that today's decision does not limit tomorrow's options. There is no right platform in the abstract: there is a platform that fits your stack, your operating model and your regulatory obligations.

Start with the questions, not the demo

Demos show the best of each product with prepared data. Before watching them, write down what your organization needs to solve. We recommend starting from three inputs:

  • The stack inventory: languages and runtimes, databases, middleware, messaging, clouds, containers, mainframe or legacy systems if any, and third party applications you cannot modify.
  • Recent incidents: which kinds of incidents took longest to diagnose and what information was missing. That list is the best script for a proof of concept.
  • Constraints: regulatory data requirements, security policies, budget, team skills and the tools it must integrate with.

With these inputs, the evaluation stops being "what does each tool have" and becomes "which one best solves our problems within our constraints."

Six criteria for a neutral evaluation

1. Stack coverage

Check, technology by technology, what is instrumented automatically, what requires configuration and what is not supported. Pay attention to the edges: legacy systems, packaged applications, network components and managed cloud services. A platform that is very complete for microservices may fall short on the transactional core that matters most.

2. OpenTelemetry support

Dynatrace and Datadog, like other commercial platforms, accept OpenTelemetry data today. The useful question is not whether they support it but how: whether they receive all three signals (traces, metrics and logs) over OTLP, whether OpenTelemetry data gets the same treatment as data from their own agent in analysis, dashboards and alerts, and whether they respect semantic conventions without translations that lose information.

3. Data model, retention and cost

Licensing models vary: per host or compute unit, per volume ingested, per volume retained or queried, per user, or combinations of these. Without going into prices, which change and depend on each negotiation, what matters is understanding which variable in your operation drives the bill. Ask for an estimate based on your real volume of logs, traces and custom metrics, and project it with expected growth. Also review retention per signal and the cost of keeping data longer for audit requirements.

4. Root cause analysis

Every platform promises assisted root cause analysis. Evaluate it with your incidents, not the demo's. Ask how it correlates events, how it uses topology, how it explains its conclusions and what it does when the cause sits in a component it does not monitor. An automatic diagnosis you cannot verify does not reduce resolution time; it just moves it.

5. Access control and data residency

In banking, insurance and healthcare, this criterion is often a deal breaker. Review the granularity of access control (per team, application or data type), integration with your identity provider, masking of sensitive data before storage, access auditing and the regions where data is hosted. Both vendors mentioned offer their SaaS service in several regions; confirm in the current documentation which ones are available and which deployment options exist for your case. Check everything against the outsourcing and cloud regulations that apply in each country where you operate, together with your Compliance and Risk teams.

6. ITSM integration and automation

An alert that does not become a well categorized, assigned incident with context creates manual work. Evaluate bidirectional integration with your ITSM tool (for example, ServiceNow), alert deduplication, the ability to trigger remediation automations and the availability of APIs so your own workflows can consume the data.

Evaluation table

The table works as a basis for an evaluation matrix. We recommend assigning a weight to each criterion according to your priorities before watching the demos, so the result does not depend on the latest presentation.

CriterionWhat to askRed flag
Stack coverageWhich technologies in our inventory are instrumented without code changes?Generic answers with no version compatibility matrix
OpenTelemetryAre traces, metrics and logs ingested over OTLP with the same level of analysis as the proprietary agent?Key features available only with the proprietary agent
Cost and retentionWhich variable drives the bill and what would it cost with our real volume?No way to estimate cost before ingesting data
Root causeHow does it explain its conclusions and how does it perform on our past incidents?Conclusions without traceable evidence
Access and residencyWhere is data hosted, who can access it and how is sensitive data masked?Region or masking not configurable
ITSM and automationIs the integration bidirectional and does it keep the problem context?Integration only via email or stateless webhook
ExitHow do we export data, dashboards and configuration if we switch platforms?No documented export or open formats

OpenTelemetry as a hedge against lock-in

The most effective way to avoid vendor lock-in is to separate instrumentation from analysis. If your code is instrumented with the OpenTelemetry API and telemetry flows through an OpenTelemetry Collector before reaching the platform, changing destinations is largely a Collector configuration change, not a re-instrumentation project.

The Collector also gives you control over your data: you can filter, mask sensitive attributes, sample and send the same telemetry to two destinations in parallel during a migration or a proof of concept.

It pays to be realistic about what OpenTelemetry does not solve. Dashboards, alert definitions, queries written in the platform's own language (for example, DQL in Dynatrace) and agent exclusive features will remain tied to the platform. To reduce that exit cost:

  • Version dashboards and alerts as code, where the platform allows it.
  • Keep your own catalog of SLOs, indicators and thresholds, independent of the tool.
  • Use attributes and names that follow OpenTelemetry semantic conventions, not those of a specific platform.
  • Document which exclusive features you use and why, so you know what you would lose by switching.

In regulated industries, this strategy also answers a growing requirement: the ability to exit a critical provider in an orderly way. In the European Union, the DORA Regulation (Regulation (EU) 2022/2554) requires financial entities to manage ICT third party risk, including exit strategies. Even if you operate outside Europe, the principle is good practice.

How to run a useful proof of concept

A well designed proof of concept replaces opinions with evidence. We recommend these steps:

  1. Define the scope: two or three representative applications, including at least one with legacy or third party technology.
  2. Instrument with OpenTelemetry wherever possible: send the same data to every platform under evaluation through a Collector, so you compare analysis on identical data.
  3. Reproduce real incidents: use the recent incidents from the initial inventory and measure how long each platform takes to lead the team to the cause.
  4. Test the deal breakers: access control, masking, data region and ITSM integration, with your Security and Compliance teams in the room.
  5. Measure cost with your volume: record the real volume ingested during the test and ask for the annual projection.
  6. Score with the weighted matrix: apply the weights defined before starting and document the decision.

Remember the limit of any platform: it will make your infrastructure observable, but the root cause of business incidents also depends on your software being written to be observable. The best tool choice delivers little if the application does not emit business context.

Sources

Want to apply this to your operation?

Request an assessment