When I talk to people about agents interacting with websites, the conversation almost always starts with perception: how does an agent “see” a page? Is it screenshots or the DOM or the accessibility tree? In my previous InfoWorld article, “Accessibility is the first-class interface for AI agents”, I argued that the accessibility tree is becoming an increasingly important interface between websites and AI agents. From there the conversation shifts to actuation, and to ways of making it less brittle, which let a site expose its own actions as callable tools instead of forcing an agent to guess at buttons and forms.

That conversation matters. But it leaves out a more consequential architectural question, one that determines who is accountable when an agent gets something wrong: where does the agent live?

There are three plausible homes for an agent that interacts with your business: on-site, in-browser and off-browser. Each one trades control for reach differently. Right now, much of the industry’s attention is focused on the two homes furthest from the business: browser extensions that co-browse alongside a user, and autonomous agents running in the cloud that talk to your APIs or drive a browser on their own.

I think that is the wrong place for most enterprises to start.

If you run a business with real compliance exposure, or a brand you cannot afford to have misrepresented, the pragmatic move is to build the agent that lives on your own site first and treat reach as a problem you solve later, on your own terms.

Reach without governance is a liability

The appeal of in-browser and off-browser agents is obvious. They promise discovery, scale and automation you do not have to build or maintain yourself. An off-browser agent can compare your prices against every competitor and keep working after your support team has gone home. A co-browsing extension can act inside a user’s already authenticated session without requiring any special integration on your side.

But an agent you do not control is still speaking for your business.

An off-browser agent talking to your APIs or scraping your DOM runs on a model you did not choose, follows guardrails you did not write and represents your brand in a conversation you cannot see. If it misquotes a price, misstates a policy or initiates a transaction that creates a customer dispute, the fact that the agent belonged to somebody else may not make the resulting problem disappear.

There is also a deeper security problem. Computer-use agents interact with untrusted content by design. Anthropic’s guidance for browser and computer-use agents notes that webpages and application interfaces can contain adversarial instructions and recommends scoped permissions, human confirmation for high-stakes actions, and monitoring of agent behavior. OWASP’s AI Agent Security Cheat Sheet goes further, identifying agent-specific risks including prompt injection, tool abuse and privilege escalation, data exfiltration and memory poisoning.

In-browser agents are a better deal, because a human is watching the same screen and can catch a mistake before it lands. But the model, the tone and the decision logic are still outside your hands. For  or any other regulated industry, that is not a risk to take lightly just to pursue reach you have not yet learned how to govern.

What you actually gain by building at home

An on-site agent is less glamorous than the idea of your product being discovered by roaming agents across the open web. But it is the only home where the business keeps full governance.

You choose the model. You define the guardrails. You decide the tone, the escalation paths and exactly what the agent is and is not allowed to promise a customer. Because the agent has direct access to your systems, it does not need to reverse-engineer your checkout flow as an outside agent would. It can be given your actual domain knowledge and your actual APIs.

That matters because enterprises have solved this problem before. They did not expose databases directly to the internet; they introduced APIs. APIs were not just about integration. They created contracts, authentication, auditing and policy enforcement. On-site agents represent a similar evolution. Rather than forcing an external agent to infer behavior from a user interface, a business can expose governed capabilities through tools with explicit contracts.

WebMCP is a promising step in that direction. It makes it easier for a site to publish actions an agent can invoke, instead of forcing the agent to guess at the page. The important shift is not that the agent can click a button. It is that every action becomes observable, policy-governed and replayable.

The Chrome team’s WebMCP Sports demo is a good preview of what this can look like done well: an on-site assistant calling the site’s own WebMCP tools, with every call visible in the chat log. That is not a black box you are hoping behaves. It is an auditable interaction, with named tools and defined contracts.

The three homes

The cleanest way to think about this is as a spectrum.

• On-site. The agent lives on your website. You ship it. You own the model, the tools, the tone, the policies and the telemetry. It has first-party access to your systems and your workflows. The trade-off is reach: it is coupled to your site, and it is not meant to carry the user’s journey across the web.
• In-browser. The agent lives inside the user’s browser session, usually as a co-browsing extension or a browser-integrated assistant. It benefits from the user’s logged-in state and can act with the user present. That gives it real reach and a strong human safety net. But the business does not control the model or the guardrails, and the agent can still misrepresent the brand.
• Off-browser. The agent runs elsewhere — in the cloud, on a server or on a user’s local machine and reaches your business by talking to APIs or driving a browser remotely. This is the widest-reach model, but also the least governed from the business’s point of view. It may compare competitors, cross workflows and act at scale, but you do not control the experience.

None of these homes is inherently bad. They solve different problems. But if your first priority is governance, accountability and brand integrity, on-site is the right starting point.

Start with the governed interface

For teams deciding where to put their first investment, three moves make the difference between a real interface and a chatbot glued to a homepage.

First, ship the on-site agent as the authoritative interface, not a fallback. It should have first-class access to your own APIs and domain knowledge — the same systems your support team and checkout flow already use — rather than being handed a scraped version of your own page.

Second, instrument every tool call it makes. If your agent calls a WebMCP tool to check inventory or apply a discount, log it the way you would log an API call from any other client. That log is your audit trail, your debugging tool and eventually your evidence when a regulator or customer asks what happened.

Third, design for human escalation from day one. Even a well-governed agent should not make every decision autonomously. Build explicit handoff paths to human operators for ambiguous or high-risk situations.

Taken together, those choices turn an on-site agent into a governed interface rather than a novelty feature. That distinction matters. The first is something an enterprise can safely operate; the second is something it will eventually have to defend.

When agents start talking to agents

There is another reason to build the on-site agent first: the three homes are unlikely to remain isolated.

Imagine that a user’s browser agent arrives at an airline website. It already knows the traveler’s preferences, calendar constraints, loyalty programs and perhaps their budget. The airline’s on-site agent knows something completely different: routes, seat availability, fare rules, upgrade eligibility and the airline’s operational policies.

Why should the visiting agent reverse-engineer all of that from a website?

A more natural architecture may be for the two agents to communicate.

This is no longer purely hypothetical. The Agent2Agent (A2A) protocol is explicitly designed to allow independent agents built by different vendors and frameworks to discover capabilities, exchange information, delegate work and coordinate tasks.

In that world, the user’s agent brings intent and user context. The site’s agent brings domain knowledge and governed execution.

And suddenly the on-site agent’s limited reach does not look quite so limiting. It does not need to travel across the web if other agents can come to it.

The incentive problem nobody is pricing in yet

There is a harder question underneath the technical one: who does the on-site agent actually work for?

A visiting agent, co-browsing or otherwise, is acting on the user’s behalf. An on-site agent, however well-built, is the business’s employee. And the moment the two negotiate a discount, an upgrade or a substitution, that exchange stops looking like a neutral handoff and starts looking like a sales conversation with an AI on each side of the table.

That tension is going to matter.

Businesses building on-site agents now should design their guardrails with that scrutiny in mind, rather than treating “the agent decided” as a shield. Control is worth having. It is also a responsibility. And building the agent that answers to you is the only way to exercise it.

The first agent should be the one you can govern

The history of enterprise software suggests a familiar pattern. Organizations adopt new interaction models by first bringing them under their own governance. Internal systems become APIs. APIs become platforms. Platforms become ecosystems.

AI agents will likely follow the same path.

That is why I think the first agent your business builds should not be the one that can reach the entire web. It should be the one whose behavior you can understand, measure and improve. Once that foundation exists, expanding outward becomes an architectural choice instead of a leap of faith.

Reach is tempting. But for regulated and brand-sensitive businesses, control has to come first.

Every AI feature you’re building is based on a price that isn’t real. Current AI services are heavily subsidized as model providers seek to expand their market share, and subsidies inevitably end. Just think back to the early days of Lyft and Uber, when a single ride often cost less than taking public transit. As the costs associated with building on cheap AI infrastructure slowly start to reflect reality, the architecture decisions you made to get to market quickly may be the ones hemming you in.

This isn’t unique to AI. IT leaders have always had to weigh speed-to-value against the cost of dependency. What’s different now is how fast those dependencies are being built, and how little visibility most teams have into them.

Across IT, scalability is contingent on portability. And architecting with portability in mind is the best way to minimize lock-in as AI pricing models evolve, giving you leverage in future contract negotiations, and a fighting chance of switching platforms if the costs of one vendor get too high.

The challenge is that AI providers are incentivized to make leaving as painful as possible, which they achieve through deep integrations and proprietary features. You can control some of this through architectural decisions. Other parts are structural dependencies you should at least be aware of going in.

Swapping platforms will never be painless, but it’s less painful than being locked in when your vendor decides it’s time to recoup its losses, as every AI provider eventually will.

The part you control

The foundation for portability is the abstraction layer, an internal service your applications call instead of hitting Claude, GPT, or Gemini directly. It uses a request format defined by you, and translates that to whatever the vendor behind it expects. If you need to change vendors later, it’s closer to a configuration change than a code rewrite.

You don’t need to build this from scratch; open-source gateways like LiteLLM or Portkey already handle the routing and interface. Scoped to what you actually need, it’s a small project that one or two engineers can complete in a few weeks.

From there, audit the parts of your codebase that call a vendor’s API directly and route them through your proxy layer one at a time (have fun with that). Skip the features you’re not using yet, like fine-tuning or batch APIs, and focus on the 80% case. Think: text in, text out, and maybe tool calls.

The parts you don’t control

The API call is the easy part to abstract. Other dependencies may be tougher to account for given the pressure to deploy quickly, but you should at least be aware of them.

Prompts get tuned to a specific model’s characteristics, and those instructions won’t necessarily make sense to another model. If you do switch vendors, expect to re-tune each prompt. Fine-tuning a model will box you in, unless you use an open-weight model that you operate internally.

Vectors generated by one provider’s model won’t be compatible with another’s, so switching means re-embedding your knowledge base. Tool calling and structured output conventions differ by vendor, so code written around one model’s schema will need rewriting for another. Caching and batching mechanics carry their own vendor-specific rules and cost implications that don’t map across providers.

In short, code portability is cheap. Behavioral portability is the real cost, and no gateway tool solves that for you.

Build with your eyes open

Most teams don’t realize how locked into a model they’re becoming until they’re forced to move, which is the worst time to find out. How many parts of your codebase import a vendor SDK directly, versus calling a shared internal function?

If you want a real answer instead of a guess, pick one workflow that isn’t business-critical and try porting it to a different vendor. See what breaks and how long it takes to fix. That will uncover your dependencies better than a checklist.

The bottom line is that you should treat AI providers like any other infrastructure vendor where concentration risk matters. Hold AI providers to the same standard you would apply to any IT or technology vendor. There will likely always be some level of dependency, but you should know where the hooks are and choose them deliberately. That way, you’re prepared when AI price hikes inevitably arrive.

—

New Tech Forum provides a venue for technology leaders—including vendors and other outside contributors—to explore and discuss emerging enterprise technology in unprecedented depth and breadth. The selection is subjective, based on our pick of the technologies we believe to be important and of greatest interest to InfoWorld readers. InfoWorld does not accept marketing collateral for publication and reserves the right to edit all contributed content. Send all inquiries to doug_dineley@foundryco.com.

Frontend teams have become pretty good at monitoring what happens after an application reaches production. We track JavaScript errors, API failures, latency and Core Web Vitals. When something changes after a release, these signals help us determine whether the application is healthy.

But there is one gap I think frontend teams often overlook. An application can look completely healthy in those dashboards while an important workflow is unusable for someone navigating with a keyboard or assistive technology.

Imagine a checkout flow where a recent UI change makes the final button unreachable by keyboard. The page still loads. The API responds successfully. There are no new JavaScript exceptions, and performance metrics look normal. Every traditional signal says the application is working, yet some users can no longer complete the purchase.

That is why I have started thinking about accessibility as more than a compliance or pre-release testing concern. If an accessibility regression prevents someone from completing a critical workflow, that is also a production reliability problem.

Where traditional frontend observability falls short

Most frontend observability focuses on technical health. We want to know whether pages load quickly, requests succeed and deployments introduce errors. These are useful signals, but they mainly tell us whether the software is functioning as the system expects.

Accessibility failures are different because they often do not produce obvious technical errors. Removing an accessible name from an icon button will not necessarily throw a JavaScript exception. Replacing a semantic button with a clickable div might leave mouse interaction working normally. A focus-management regression inside a modal can make the interface extremely difficult to navigate while having no effect on page performance.

This creates a blind spot. Our monitoring can tell us that the button rendered and that clicking it successfully triggered an API request. It may not tell us that a keyboard user cannot reach that same button.

Automated accessibility checks in CI are an important first line of defense, and I use them as an early guardrail. But I would not treat a passing CI check as proof that the production experience is accessible. Production applications contain changing data, feature flags, third-party scripts and combinations of components that may not be represented in isolated tests.

Automated tools also have limits. They can identify many structural problems, but they cannot detect every accessibility or WCAG conformance failure. CI should therefore be an important checkpoint, not the end of accessibility verification.

Monitor accessible journeys, not just healthy pages

One way I think frontend teams can close this gap is by applying the same idea behind synthetic monitoring to accessibility.

Instead of checking only whether a page loads, we can test whether important user journeys remain usable. Can someone navigate the primary controls using only a keyboard? When a dialog opens, does focus move somewhere sensible? Can a form be completed without relying on a mouse? Do important controls still expose useful names and roles?

Consider the checkout example again. Traditional synthetic monitoring might verify that the page loads, products appear and the checkout API responds. An accessibility-aware journey could go further by navigating through the same flow with keyboard interactions and verifying that focus reaches the controls required to complete the purchase.

I would not try to automate every accessibility requirement this way. The goal is to identify a small number of critical workflows and ask a simple question: Can a user still complete the task?

For applications with a design system, I would extend the same thinking to shared components. A regression in a button, dialog, form field or navigation component can spread across many screens at once. Testing keyboard behavior, semantics, accessible names and focus expectations at the component level can catch problems closer to their source instead of discovering the same regression page by page.

This creates two useful layers of protection. Design-system checks help protect the building blocks, while synthetic journeys help verify that those building blocks still work together in real application flows.

Make accessibility part of release health

Frontend teams already look at signals after deployment to decide whether a release is healthy. We watch error rates, latency, failed requests and performance changes. I think accessibility signals can become part of that same conversation.

That does not mean creating another huge dashboard or turning every automated accessibility warning into a production incident. The useful approach is to focus on impact.

If a release introduces a minor issue in secondary content, the team may address it through the normal development process. If the same release prevents keyboard users from completing checkout, submitting a form or navigating a primary workflow, that deserves a different level of attention.

This is where accessibility-aware release health becomes valuable. Teams can define a small set of expectations around critical journeys and shared components, then watch for regressions just as they would watch other frontend reliability signals.

The need for this approach is becoming harder to ignore. The 2026 WebAIM Million analysis found an average of 56.1 detected accessibility errors per home page, a 10.1% increase from 2025. It also found that the average home page contained 1,437 elements, a 22.5% increase in a single year. As interfaces become larger and more dynamic, relying only on occasional accessibility audits becomes increasingly difficult to scale.

If users can’t complete the workflow, the system isn’t healthy

Frontend observability has given us increasingly sophisticated ways to understand what applications are doing in production. But our definition of application health should not stop at whether JavaScript executed, APIs responded or pages rendered quickly.

A system can pass every one of those checks and still fail the person trying to use it.

Accessibility testing during development remains essential, but production-oriented signals can help close the gap. Synthetic user journeys can tell us whether critical interactions still work. Design-system regression tests can protect shared components. Accessibility-aware release checks can help teams recognize when a deployment has affected a critical workflow.

For me, the principle is simple. If users cannot complete an important task because of an accessibility regression, I do not think we should describe the application as healthy just because the traditional dashboards are green.

That is not only an accessibility problem. It is a frontend reliability failure.