Table of Contents
TL;DR
There’s a real line between using Claude as an assistant and letting it act as an agent, one drafts an answer, the other changes state in a live system. Getting this right starts with choosing the right autonomy level for the task (assist, recommend, execute-with-approval, or execute-within-limits), giving the agent a narrow, bounded job instead of a vague objective, and connecting tools without quietly expanding what it’s allowed to touch. Approval should scale with risk, not with how easy an action is to automate, and autonomy should expand only after an agent proves itself against messy, adversarial, real-world cases, not because a demo went well.
Executive summary
Claude-powered applications are increasingly moving beyond answering questions and generating drafts. They can retrieve information, call tools, update systems, and complete tasks across multiple steps. In this blog, we tell you how much authority Claude should have, and under what conditions.
Introduction
There is a quiet line every company crosses with generative AI, and it usually happens when someone says, “Why can’t the model just fix this itself?”
Up until that point, you are using Claude as an assistant.
The moment you let the model take action on its own, everything changes.
When Claude moves from drafting a response to querying three different systems, deciding on a resolution, updating the database, and triggering an external API, it stops being an assistant. It becomes an agent.
An assistant gives you an answer, but an agent changes state in your live environment.
Building that safely used to be a custom engineering nightmare. Today, the plumbing is finally catching up. Anthropic’s Model Context Protocol (MCP) gives models a clean, standardized way to talk to internal tools. And heavyweights like Databricks and AWS provide the guardrails – handling the identity, long-term memory, and observability needed to track what an agent is actually doing.
The technology is finally mature enough to support real autonomy. But how to decide where automated execution adds value, and where you still need a human in the loop?
1. Choose the right level of autonomy
The first decision is not which model or framework to use. It is what the system should be allowed to do.
Assist
Claude generates information, but does not take action.
Examples:
- – Summarising a customer case
- – Drafting an incident report
- – Explaining a data-quality issue
- – Preparing a response for an employee to review
This is the lowest-risk starting point for most organizations.
Recommend
Claude reviews information and proposes what should happen next. A person makes the decision.
Examples:
- – Recommending which support cases need escalation
- – Suggesting remediation steps for a cloud incident
- – Identifying documents that require compliance review
- – Proposing the next action in a sales or operations process
The system should show the evidence behind the recommendation and make it easy for the user to accept, change, or reject it.
Execute with approval
Claude prepares an action, but a person must approve it before execution.
Examples:
- – Updating a customer-service ticket
- – Creating a change request
- – Preparing a purchase requisition
- – Drafting a customer communication for sign-off
- – Submitting a remediation task to an operations queue
This model is useful when the action is valuable but still requires accountability.
Execute within limits
Claude can complete low-risk actions without approval, provided the action stays within defined boundaries.
Examples may include:
- – Applying a support-ticket category
- – Updating non-sensitive metadata
- – Running a read-only diagnostic query
- – Sending an internal notification when a known condition is met
The system still needs monitoring, access controls, and a way to suspend the action quickly.
The mistake is to assume that the most autonomous design is the most advanced design. In an enterprise, the right design is the one that creates value while keeping responsibility clear.
2. Give the agent a narrow job
An agent should not begin with an instruction such as “manage customer operations” or “improve the organization’s cloud environment.”
Those objectives are too broad to govern. They do not specify:
- – Which systems the agent can access
- – Which decisions it can make
- – How it should prioritise competing objectives
- – When it should ask for help
- – What it must never change
- – What success looks like
A better starting point is a bounded task:
Review newly created support tickets, classify them using approved categories, retrieve relevant guidance, draft a response, and route tickets with billing or security indicators to a human specialist.
That definition is narrow enough to implement and test. It also exposes the required controls:
- – A connection to the ticketing system
- – Access to approved knowledge sources
- – A classification action
- – A response-drafting action
- – Escalation rules
- – Human review
The agent should also have explicit non-goals:
- – It must not change account permissions.
- – It must not issue refunds.
- – It must not close a ticket without approval.
- – It must not make claims outside approved sources.
- – It must escalate when information is incomplete or contradictory.
These boundaries should not exist only in a prompt. They should be enforced through tool permissions, application logic, data access, approval flows, and monitoring.
3. Connect tools without expanding authority
An agent becomes useful when it can work with the systems where employees already perform their jobs. It becomes dangerous when every available system is exposed to it without a clear access model.
Every tool connected to Claude should have an owner and a defined purpose.
Define the action
A read-only customer lookup is not equivalent to a tool that can change a customer account.
Tool design should follow the principle of least privilege. Give the agent the smallest action that can complete the task rather than broad administrative access.
Restrict the data
The agent should receive only the information needed for the task. Existing identity, role, and data-governance controls should apply to the agent’s access path.
This is particularly important when the underlying user has broad permissions. A user’s ability to view a system does not automatically mean Claude should receive all of that system’s data in every interaction.
Control invocation
Tool access may need to depend on:
- – User role
- – Customer account
- – Geography
- – Data sensitivity
- – Approval status
- – Type of action
- – Business hours or operational conditions
Record the activity
The system should capture:
- – The user’s request
- – The information made available to the agent
- – The tool selected
- – The parameters passed
- – The result returned
- – The approval decision
- – The final action
MCP can provide a standard way for AI applications to connect with external systems and tools. However, MCP does not replace enterprise security. Each server, connector, and tool still requires authentication, authorisation, monitoring, ownership, and lifecycle management.
A practical AWS and Databricks architecture may combine:
- – AWS for application runtime, identity, agent operations, and infrastructure observability.
- – Databricks and Unity Catalog for governed data access, data products, evaluation, and monitoring.
- – MCP or another integration layer for controlled connections to tools.
- – Claude for language interaction, reasoning, planning, and task execution.
The architecture should make it possible to answer a basic question after every important action:
What did the agent know, what did it decide, what did it do, and who authorised it?
4. Make approval and escalation explicit
An agent will eventually encounter incomplete information, conflicting instructions, unavailable tools, or a request outside its scope.
A production design needs a defined response to uncertainty.
The agent should be able to:
- – Ask the user for missing information
- – State that the available evidence is insufficient
- – Escalate to a named team
- – Stop before taking an irreversible action
- – Return a partial result with the unresolved issue identified
- – Retry a failed tool call under controlled conditions
- – Record the event for review
Approval should be based on risk, not on whether the action is technically easy.
Human approval is generally appropriate when the agent’s action involves:
- – Financial commitments
- – Customer-facing communication
- – Identity or access changes
- – Regulated records
- – Personal, financial, or health information
- – Production infrastructure
- – Legal, employment, credit, or clinical decisions
- – Irreversible changes
The approval experience must also contain enough information for a person to make a real decision. The reviewer should see:
- – The proposed action
- – The reason for the action
- – The information supporting it
- – The systems or records that will change
- – The remaining uncertainty
- – The consequences of approval
Human oversight should not be treated as a button that people press without reading. It should be designed as part of the workflow.
5. Let autonomy expand through evidence
An agent should not receive more tools or permissions simply because it performed well in a demonstration.
Before expanding its authority, test it against realistic situations:
- – Normal requests
- – Ambiguous instructions
- – Missing or conflicting data
- – Unauthorised actions.
- – Prompt-injection attempts.
- – Tool failures.
- – Duplicate requests
- – Unavailable dependencies
- – Actions with unintended consequences
- – Cases where escalation is the correct outcome
The evaluation should measure more than the quality of the final response. It should also examine:
- – Whether the correct tool was selected
- – Whether unnecessary tools were called
- – Whether the agent respected permission boundaries
- – Whether it escalated appropriately
- – Whether it attempted to bypass approval
- – Whether the action created avoidable rework
- – Whether the user accepted the result
Anthropic’s guidance recommends defining measurable success criteria and evaluating AI applications against relevant, real-world inputs. That principle is especially important for agents because a polished answer can conceal an incorrect tool choice or an unsafe action.
A practical progression is:
- Test the agent in a controlled environment
- Give it read-only access to selected tools
- Require approval for every action
- Review traces, exceptions, and user feedback
- Automate only low-risk actions with stable performance
- Expand its scope gradually
- Re-evaluate after every significant change
Autonomy should be treated as a permission that can be earned, reduced, or withdrawn.
What you can do next
Before approving an enterprise agent initiative, answer these questions:
- – What exact task will the agent perform?
- – What can it read?
- – What can it recommend?
- – What can it change?
- – Which actions require approval?
- – What must it never do?
- – What happens when it is uncertain?
- – How will every important action be traced?
- – What evidence will justify expanding its autonomy?
Wishtree Technologies helps enterprises design and implement Claude-powered applications and agentic workflows across AWS and Databricks.
We are part of the Claude Partner Network and help connect governed data, approved tools, enterprise applications, evaluation, and operational controls – so organizations can move from AI assistance to controlled, measurable execution.
Contact us today to get started.
Frequently Asked Questions (FAQs)
Is an agent always better than a conventional workflow?
No. Conventional workflows are often better when the steps are predictable, the rules are stable, and the process can be expressed clearly in business logic. Agents are more useful when inputs vary, several information sources are involved, or the sequence of steps depends on interpretation.
Should enterprises build one general-purpose agent?
Usually not as a first step. A narrow agent with a clear objective, limited tools, and measurable outcomes is easier to secure, test, and operate. Broader or multi-agent systems can be considered after the underlying patterns are proven.
Does MCP make enterprise agents secure?
No. MCP standardises connections between AI applications and external systems, but security depends on the implementation. Enterprises still need authentication, authorisation, data filtering, tool-level permissions, logging, monitoring, and clear ownership.
Where should humans remain involved?
Humans should remain involved wherever an action is high-impact, irreversible, regulated, customer-facing, or difficult to explain. They should also be available when the agent lacks sufficient information or confidence.
What is a suitable first agent use case?
Start with a repetitive, measurable, and bounded process that uses a small number of read or low-risk tools. Examples include support-ticket classification, internal knowledge retrieval, incident triage, or document review with human approval.
