Blog | TSG Technology Consulting

The OT testing paradox: The systems that most need testing are the ones you can't touch

Written by TSG | Nov 16, 2026, 4:07:37 PM

Imagine a security team preparing to test the network that keeps a manufacturing plant running. The objective is straightforward: identify vulnerabilities, determine whether an attacker could exploit them, and understand which critical systems are exposed.

Then the operations manager asks a question that changes the conversation.

Can you guarantee the test won't interrupt production?

The security team can't make that promise. An aggressive scan could overwhelm a sensitive device. An unexpected request could cause a legacy controller to behave unpredictably. A test intended to improve security could interrupt the very process it's supposed to protect.

So the scope changes. Certain systems are excluded. Testing is restricted to approved devices and maintenance windows. The assessment proceeds, and the organization receives a report detailing the vulnerabilities discovered.

But the most important question may remain unanswered: Could an attacker reach the systems that were too critical to test?

This is the OT testing paradox. The systems with the greatest operational consequences are often the ones organizations are least willing to probe. And the more those systems matter, the harder it becomes to determine whether they're exposed.

The answer isn't to accept that uncertainty. It's to rethink how industrial security is tested.

 

The problem with treating OT like another IT network

Traditional penetration testing is built around controlled attempts to identify and exploit weaknesses. Security teams probe systems, test defenses, and demonstrate where unauthorized access may be possible.

In many IT environments, that approach is manageable. Systems can be isolated, restored, or tested outside business hours.

Operational technology works differently.

A programmable logic controller may regulate a production line. A supervisory control system may coordinate equipment across a facility. An industrial workstation may provide access to processes where an unexpected command could affect equipment, safety, or service availability.

These systems weren't necessarily designed to tolerate the kinds of probing that modern security tools routinely perform.

NIST addresses this distinction in its Guide to Operational Technology Security, SP 800-82 Revision 3. The guidance recommends evaluating how vulnerability scanning tools might affect OT components, testing those tools offline before introducing them into production, and conducting penetration testing only when it will not adversely affect operations.

That last qualification matters.

For an IT security team, successfully exploiting a vulnerability may demonstrate that a control has failed. For an operations team, the same exercise could introduce unacceptable risk to a process that must remain available.

Neither team is wrong.

The problem arises when organizations use the limitations of active testing as a reason to stop investigating what an attacker could reach.

 

What happens when your most critical systems are off-limits?

Consider a typical industrial security assessment.

The team examines network infrastructure, reviews known vulnerabilities, checks configurations, and tests systems approved for active scanning. The report identifies outdated software, unnecessary services, and weaknesses that need remediation.

It's useful information. But the assessment may tell only part of the story.

A controller excluded from testing doesn't become less vulnerable because nobody touched it. A network segment that cannot tolerate scanning doesn't become unreachable to an attacker. And a successful assessment of accessible systems doesn't establish that the rest of the environment is secure.

The distinction becomes more important when considering how industrial intrusions unfold.

An attacker doesn't necessarily need direct access to a controller. The initial compromise might involve a contractor account, remote access service, engineering workstation, or another system with legitimate connections to operational equipment.

From there, the attacker may attempt to move through the environment using existing permissions and trusted communications.

MITRE ATT&CK for ICS documents these behaviors under lateral movement, including the use of remote services, valid accounts, and compromised systems to reach additional industrial assets.

The danger may not be a single exposed device. It may be the route connecting an ordinary workstation to something capable of affecting physical operations.

If the assessment never examines that route, the organization may have an extensive vulnerability report without a clear understanding of its most consequential exposures.

 

A critical vulnerability isn't always your biggest risk

Security teams are accustomed to prioritizing vulnerabilities by severity. A critical finding gets immediate attention. A medium-severity issue moves further down the remediation queue.

That approach provides structure, but it doesn't always reflect operational consequences.

Imagine two findings.

The first is a critical vulnerability on an isolated device with tightly restricted communications. The second is a lower-rated weakness involving a workstation with access to an engineering system.

The first may be technically more severe. The second could present a more consequential route into the operational environment.

Which should be addressed first?

The answer depends on more than a vulnerability score. It requires understanding network reachability, access permissions, exploit conditions, existing defenses, and the operational importance of the systems involved.

This is where traditional vulnerability management can fall short. It identifies weaknesses, but it doesn't always establish how those weaknesses interact or whether an attacker could use them to reach critical assets.

An attacker doesn't evaluate an industrial environment as a spreadsheet of disconnected vulnerabilities. They look for opportunities to move from one system to another.

Security teams need to understand those relationships, too.

 

What if you could test the path without touching production?

The testing paradox becomes less restrictive when organizations separate two activities that are often treated as interchangeable: testing a live device and evaluating how an attacker might reach it.

One way to approach the second problem is to construct a digital representation of the OT environment using information the organization already maintains.

Firewall and router configurations, routing tables, asset inventories, vulnerability exports, and other approved data sources can help establish how systems are connected, which communications are permitted, and where potential weaknesses exist.

Security teams can then analyze possible attack paths within that modeled environment rather than actively probing production equipment.

Consider a contractor who needs remote access to troubleshoot industrial machinery.

The contractor connects to a maintenance workstation. That workstation communicates with an engineering server. The engineering server has authorized connections to controllers supporting a critical process.

Every connection may have a legitimate purpose. But together, they could create an unintended path into sensitive systems.

By modeling those relationships, security teams can investigate whether a compromise of the contractor's access could create a plausible route toward critical equipment.

They can also examine where segmentation, access restrictions, or configuration changes might interrupt that route.

This is the thinking behind TSG/ControlBurn, which uses digital twin technology and simulated penetration testing to examine potential attack paths through OT environments without conducting those attacks against live production systems. By working with existing infrastructure and vulnerability data, the service helps organizations understand which weaknesses may create routes to critical assets and where remediation efforts could have the greatest operational value.

The distinction is important. A modeled attack path is not automatically proof that an attacker could successfully exploit every system along it. The accuracy of the analysis depends on the completeness of the underlying data, the assumptions used, and the conditions being evaluated.

But the model can reveal relationships and potential exposures that a conventional vulnerability list may never explain.

And it allows organizations to investigate those questions without introducing unnecessary testing activity into production.

 

Your last assessment can't see yesterday's changes

Even a thorough security assessment has a limitation that organizations sometimes overlook: it describes an environment at a particular point in time.

Industrial environments change.

A new vendor receives remote access. A firewall rule is modified to support an integration. A maintenance workstation is replaced. A facility connects equipment that previously operated independently.

Each change may be necessary. Each may also alter the paths available through the network.

An assessment completed six months ago cannot account for a connection introduced last week.

This creates another problem with relying exclusively on periodic penetration testing. Organizations may be using yesterday's findings to make decisions about today's environment.

NIST recommends ongoing vulnerability identification and recognizes passive monitoring as an approach for observing OT environments while limiting operational interference.

Passive monitoring is valuable, but it doesn't necessarily establish whether an attacker could move through a sequence of permitted connections or exploit a particular weakness.

Combining passive observations with configuration analysis, vulnerability context, and attack-path modeling can provide a more complete view.

It also creates an opportunity to reassess exposure when the environment changes, rather than waiting for the next scheduled engagement.

For example, if a new vendor connection introduces access to an engineering workstation, the organization can examine how that change affects modeled attack paths and determine whether additional controls are needed.

The objective isn't to test everything continuously. It's to make security evaluation more responsive to the changes that can alter operational risk.

 

Stop counting vulnerabilities. Start examining consequences.

A vulnerability report can tell a security team what needs attention. It doesn't always tell an operations leader what could happen if a weakness is exploited.

That difference matters.

A CISO may need to understand whether an attacker could move between network segments. A plant manager wants to know whether production could stop. An engineering leader needs to know which systems support critical processes and whether remediation could introduce its own operational risks.

These aren't competing priorities. They're different perspectives on the same problem.

An effective OT security assessment should connect technical exposure to operational consequence.

That means asking more useful questions.

Could a compromised vendor account provide access to an engineering workstation? Could that workstation communicate with critical controllers? Which security controls would interrupt the route? What would need to change to reduce exposure without affecting operations?

The answers can help teams prioritize remediation based on the systems and processes that matter most.

Sometimes that means addressing a vulnerable device. Sometimes it means removing unnecessary access, tightening firewall rules, or changing how vendors connect to the environment.

The most valuable security improvement isn't necessarily the one that eliminates the highest number of findings. It's the one that meaningfully reduces exposure to an unacceptable operational consequence.

 

The future of OT security isn't unrestricted testing

None of this means traditional penetration testing should disappear.

Active testing remains important when it can be performed safely, particularly in representative test environments, during approved maintenance windows, or against systems designed to tolerate it.

Passive monitoring, configuration reviews, vulnerability analysis, and attack-path simulation each provide different forms of evidence. No single method can eliminate uncertainty.

The mistake is assuming that because a system cannot tolerate conventional testing, its security cannot be meaningfully evaluated.

Industrial organizations have spent decades developing practices that protect operational continuity. Cybersecurity needs to work within those realities, not demand that operations accept unnecessary disruption in the name of validation.

That requires a shift in how security teams define a successful assessment.

The objective should extend beyond identifying vulnerabilities or demonstrating individual exploits. It should include understanding how an attacker could move through the environment, which operational assets may be exposed, and where defensive changes can interrupt those paths.

A model won't answer every question. Some findings will still require additional validation, engineering review, or carefully controlled testing.

But those limitations are reasons to combine methods, not reasons to leave critical systems unexplored.

The systems that matter most shouldn't remain security blind spots simply because they're too important to touch.

Organizations can begin by asking a different question: What could an attacker reach, and how much can we learn without putting production at risk?

That's a more practical starting point for industrial cybersecurity, and one that respects the operational realities of the environments it's meant to protect.