Real complexity: the factors that sink an automation

Why complexity gets underestimated

When someone proposes an automation, they describe the happy path: the request arrives, it is validated, executed, closed. That path is usually easy to automate. What consumes the project is everything else: the request that arrives incomplete, the server that does not respond, the vendor that changed the format, the approval that depends on who is on shift.

That is why in our Value × Complexity framework complexity is not estimated from a single impression. It is the average of five factors, each scored from 1 to 5:

• Number of systems and integrations.
• Process variability (how many exceptions).
• Data quality and available access.
• Risk of the change in production.
• Third-party dependency.

None of these factors concerns the automation tool. That is deliberate: the tool is almost never what sinks a project.

A rubric for the five factors

The table describes the extremes and the midpoint of each factor; 2 and 4 are intermediate values. Like any rubric, it is worth calibrating to your own context after the first rounds of scoring.

Factor	1	3	5
Systems and integrations	A single system, with direct access.	Two or three systems with known interfaces.	Five or more systems, or integrations with no documented interface.
Process variability	A single path, almost no exceptions.	A few known variants that can be documented.	Many exceptions; decisions that depend on one person's judgment.
Data quality and access	Documented APIs, service accounts available, reliable data.	Access possible but with paperwork; data needs cleaning.	Only a graphical interface or manual files; no service accounts; inconsistent data.
Risk of the change in production	Read only, or easily reversible actions.	Changes configuration on non-critical systems, with rollback possible.	Changes critical systems with actions that are hard to reverse.
Third-party dependency	Everything is under the team's control.	Requires coordination with another internal department.	Depends on an external vendor with no commitment on the interface.

The per-factor score is also useful after classification. In a strategic bet, the highest factors show where to split the work: if complexity comes from the number of systems, the first delivery can cover just one; if it comes from change risk, it can start in read-only mode, producing reports before taking actions. Each delivery then has lower complexity than the whole and delivers value of its own.

Variability deserves special attention. It is the factor most often underestimated on the intake sheet and the one most often discovered during the build, when it is already expensive. The next two sections explain how to find it earlier.

Uncovering variability: the process walk-through

Process documentation describes how things should work. To know how they actually work, you have to watch. We recommend a process walk-through with the person who performs the task, not with the person who supervises it:

1. Watch a real run. Ask whoever does the task to perform it while explaining each step out loud. Not in a meeting room with slides: on their screen, with real data.
2. Ask about every decision. Each time the person chooses between two paths, note the criterion. "It depends" is a valuable answer: there is an exception there.
3. Ask what goes wrong. "When was the last time this did not work as usual?" often reveals more than any diagram.
4. Surface tacit knowledge. Shortcuts, personal spreadsheets, names of contacts someone writes to "when X happens". All of that is part of the real process.
5. Repeat with someone else. If two people run the same process differently, variability is higher than the intake sheet says.

Uncovering exceptions: ticket sampling

The walk-through shows how the task is done on an ordinary day. Tickets show what happened over months. If the process is logged in a service management tool, sampling is the cheapest way to measure variability with evidence:

1. Take a recent sample. For example, the last 40 tickets for the process, or those from the last three months if there are few.
2. Classify each ticket. Did it follow the main path, or did it need an extra step, a reopening, an escalation or a query to another department?
3. Group the exceptions. They tend to cluster around a few causes: incomplete input data, a specific system, a type of request.
4. Calculate the share. For example, if 12 out of 40 tickets left the main path, almost a third of runs need special handling. That is not a 1 on variability.
5. Decide the scope. It is often wise to automate only the main path and route exceptions to a person, with a well documented ticket.

A caveat: tickets only show what was logged. If part of the work is handled over chat or phone without a ticket, sampling underestimates variability. The walk-through helps spot those gaps.

Data, access and change risk: questions before scoring

Technical factors are scored better with a short list of concrete questions. If the answer to several of them is "I don't know", real complexity is higher than it looks.

• Do the systems involved offer documented APIs or command line interfaces?
• Is there, or can there be, a least-privilege service account approved by security?
• Does the input data have a stable format that can be validated?
• Is there a test environment reasonably similar to production?
• Can the action be simulated before it runs? Tools such as Ansible, for example, offer a check mode that reports what would change without applying changes.
• Is there a tested rollback procedure if the automation fails halfway?
• Can a third party change the interface without notice? Is there an agreement that prevents it?

Change risk in production is not a reason not to automate. It is a reason to design more carefully: small batches, pre and post checks, and the automation treated as one more change within the change enablement process.

From drop to redesign

When a candidate lands in the Drop or redesign quadrant (low value, high complexity), the framework's rule is clear: it is not automated as is. Simplify the process first. That does not mean forgetting it; it means the useful work is process design, not automation engineering.

The most common simplification levers act directly on the complexity factors:

• Standardize the input: a form with mandatory fields instead of free-form emails reduces variability and improves data quality.
• Remove steps that add no control: inherited approvals nobody really reviews, data copied between systems that could be queried at the source.
• Agree interfaces with third parties: a fixed format, a single channel and advance notice of changes reduce dependency.
• Separate the common case from the exceptions: if one type of request concentrates most exceptions, perhaps it should have its own process.

Illustrative example. Take a fictional candidate: monthly inventory reconciliation with a vendor that keeps changing its file format. Its complexity factors were 3, 5, 4, 2 and 5 (systems, variability, data and access, change risk, third parties), for complexity of 3.8 and value of 2.0: the Drop or redesign quadrant.

Suppose the team agrees with the vendor on a file with a fixed structure, always delivered through the same channel, and writes rules for the three most frequent exceptions. On re-scoring, variability drops to 2, data and access to 2 and third-party dependency to 3. Complexity becomes (3 + 2 + 2 + 2 + 3) / 5 = 2.4.

With value 2.0 and complexity 2.4, the candidate moves to Opportunistic fixes. It did not become a quick win, and that is fine: the framework does not promise that every redesign ends in high priority. What the redesign did achieve is that, when there is spare capacity, automating it is a contained task rather than a project that drags on.