When the internet began moving into the center of organizational life, it looked like chaos.
Organizations were connecting systems, customers, suppliers, employees, and markets faster than their controls could comfortably accommodate. Procedures were catching up. Security models were changing. Business models were still forming.
And yet we adapted. Controls developed, security disciplines matured, and institutions learned to operate in a connected world.
That history makes it tempting to treat AI as another version of the same story: disruption, hype, unfamiliar risks, and governance that eventually catches up.
I do not think it is that simple.
The internet expanded connectivity. AI expands agency.
By agency, I mean a system's participation in what an institution notices, considers, recommends, and ultimately decides, not consciousness, and not necessarily autonomy. It does not have to make the final decision to influence the result.
Connecting the environment is different from shaping judgment
Connectivity creates pathways. It allows information, transactions, and instructions to move between people and systems. Governing those pathways means asking who can gain access, where information can travel, and what controls protect it.
Those questions remain essential.
AI introduces another set of questions when it participates in the decision process. A summary can influence what an executive notices. A classification can affect how a case is handled. A recommendation can make certain options appear reasonable while leaving others out of view.
There is a difference between carrying information and shaping the basis on which someone acts.
This is a distinction between functions. Earlier software could influence decisions too, and AI applications differ in how much agency they exercise. The relevant question is what the system is doing in this particular process, and how much the institution relies on it.
An AI system used for decision support may influence judgment without permission to take action. Where a system is authorized to act, the scope and limits of that authority require attention as well.
Either way, access controls do not establish whether a recommendation is supported, whether relevant context has survived, or whether anyone can challenge it.
The internet connected the operating environment. AI can enter the operating logic.
The decision can be shaped before the human approves it
Consider a hypothetical: an AI-generated summary prepared for an executive decision.
The summary may be clear, concise, and easy to use. But if it leaves out a material qualification, the executive may never see what would have changed the assessment. If it presents one recommendation with unwarranted confidence, the executive may mistake clarity for reliability.
The person still approves the decision. The system has helped determine what that person was asked to approve.
That is how agency can enter quietly: through what the system selects, emphasizes, or leaves out.
NIST's Generative AI Profile identifies automation bias and over-reliance as risks in human–AI interaction. It also describes the risk of confidently stated but erroneous content, and notes that automation bias can exacerbate it. These are relevant risks, not proof that every reviewer will defer or every generated summary will mislead.
Human oversight can help. Its value depends on how it works in the actual decision process.
Can the reviewer examine the evidence? Recognize a missing qualification? Question the recommendation? Choose a different course?
If the institution prizes speed and treats review as confirmation, the human may confirm rather than decide. A signature does not tell us which role the person actually played.
This is why "human in the loop" is not enough by itself.
Governance must follow the influence
When AI participates in judgment, governance must ask more than who has access:
- What decisions can this output influence?
- What evidence and assumptions support it?
- What relevant information or options might it leave out?
- Who can challenge or disregard it?
- Who is accountable for the decision and its consequences?
The distinction is practical. If AI summarizes regulatory obligations, someone must determine whether the summary is complete enough for the intended use. If it prioritizes incidents, someone must assess whether the ranking reflects the operating conditions. If it prepares executive analysis, someone must be able to examine the evidence beneath the conclusion.
The NIST AI Risk Management Framework, in GOVERN 3.2 and Appendix C, addresses differentiated human–AI roles and responsibilities. Appendix C also discusses context lost through modeling and the varying results of human–AI interaction, and it treats the degree to which people are empowered and incentivized to challenge AI output as a question requiring further study. That supports examining the arrangement as a whole rather than treating the presence of a human as sufficient evidence of effective oversight. The framework is voluntary guidance; it does not certify this article's distinction.
An institution can connect a system securely and still rely on an unsupported conclusion. It can keep the system available and still lose sight of what its recommendation leaves out.
Decision integrity requires attention to that gap.
Controls can arrive after reliance
Earlier technology cycles offer a warning as well as reassurance. In my reading, from mainframes to PCs, distributed systems, the internet, and cloud services, institutions often adopted first and built controls later: process ownership, data quality, vendor oversight. Adaptation was possible, but it did not make the initial exposure harmless.
AI raises a related concern. Decision controls may arrive after reliance has become routine.
That is a risk to examine, not a forecast. Human–AI arrangements can improve decisions. What matters is whether the institution understands the role it has assigned to the system and can account for the judgment that follows.
Connectivity was not agency.
Once systems participate in what institutions decide, the governance question expands beyond whether they are connected safely:
Can we let systems influence decisions without losing accountability for judgment?
