Autonomous AI agents are transforming enterprise automation, but businesses need stronger AI governance, data security, and human oversight to manage the risks of automated decision-making. According to Marinela Profi, Global Market Strategy Lead for AI Agents and Generative AI at SAS, organisations must build clear controls into AI architecture before allowing autonomous systems to execute business-critical actions.

Speaking ahead of AI & Big Data Expo Europe in Amsterdam, Profi outlined why traditional software deployment strategies are insufficient for agentic AI. Unlike conventional applications that primarily provide information or assist users, autonomous AI agents can interact with enterprise systems, initiate workflows, and make decisions with limited human intervention.

Her insights highlight a growing priority for businesses adopting artificial intelligence: balancing AI-driven efficiency with accountability, operational resilience, and responsible automation.

Why Autonomous AI Agents Need Stronger Governance

Traditional software development often follows a rapid deployment model in which teams release features, monitor user behaviour, and improve products through continuous iteration. However, this approach becomes more complicated when AI systems have the authority to take action independently.

An incorrect chatbot response may inconvenience a user, but an autonomous AI agent can trigger a business process, modify information in another system, or execute an unauthorised action before a human notices the error.

According to the SAS Data & AI Impact Report figures cited by Profi, trust in generative AI stands at 76%, compared with 66% for agentic AI. The research also indicates that 89% of agents deployed in production are already taking action rather than simply assisting users, while more than half operate with limited or no human approval.

These figures underline the importance of AI risk management, autonomous AI security, and continuous monitoring as companies expand their use of intelligent automation.

Profi argues that responsible AI deployment should not necessarily mean slower innovation. Instead, businesses need systems that allow them to observe agent behaviour, restrict permissions, interrupt risky actions, and recover when something goes wrong.

AI Governance Must Be Built Into Enterprise Architecture

One of the biggest challenges in enterprise AI implementation is the temptation to prioritise rapid experimentation over long-term system design.

Early development teams may overlook data lineage, grant permissions too broadly, or record AI outputs without documenting which tools and systems an agent used. Governance requirements may also remain in written policies rather than becoming enforceable technical controls.

These shortcuts can create significant compliance risks and technical debt as AI deployments expand.

For example, an auditor or regulator may eventually ask why an AI agent made a particular decision, which data informed its recommendation, what policy governed the action, and who was responsible for the outcome. Without reliable records and traceable decision-making, organisations may struggle to provide satisfactory answers.

The SAS research cited by Profi found that only 17.5% of organisations reported fully optimised data infrastructure incorporating capabilities such as data lineage, governance, validation, and explainability.

This highlights why enterprise AI governance should be treated as a core architectural requirement rather than an administrative task added after development. Organisations need clear accountability, traceable decisions, controlled access, and reliable audit records from the beginning.

Why AI Proofs of Concept Are Not Enough for Production

A successful AI proof of concept does not automatically demonstrate that a system is ready for enterprise deployment.

During a pilot, developers typically work with controlled datasets, predictable permissions, and limited system interactions. Real-world production environments are more complex, with changing data, unexpected tool failures, evolving access rights, and dependencies across multiple applications.

Autonomous agents introduce an additional risk because they do not merely generate responses. They can take actions that affect other systems and trigger consequences beyond the original interaction.

Profi therefore recommends shifting from model testing to comprehensive system testing. Organisations should assess not only whether an AI model produces an accurate answer, but also whether it uses the correct data, respects permissions, responds appropriately to failures, and stays within its authorised boundaries.

This approach makes AI system validation and agentic AI testing essential parts of the deployment process.

Before launching an autonomous agent, businesses should test how it behaves when information is outdated, a connected tool becomes unavailable, or a permission changes unexpectedly. These scenarios help reveal weaknesses that a controlled demonstration may never expose.

The Risks of Connecting One AI Agent to Everything

A centralised architecture in which a single AI model or agent connects to enterprise data and coordinates numerous business systems may appear attractive. It promises streamlined workflows, faster decisions, and greater automation.

However, giving an AI agent broad access without clearly defined restrictions can create substantial operational and security risks.

According to Profi, intelligent systems must understand not only how to complete a task, but also which data they are allowed to access, which actions they can perform, and which systems they can influence.

She points to a major global bank working with SAS as an example of an organisation using generative and agentic AI while recognising the importance of collaboration and system architecture.

The broader lesson is that an AI model should operate within established business rules, data structures, decision-making processes, and security controls. It should not become the central authority responsible for controlling the entire enterprise.

For businesses adopting AI-powered enterprise automation, clearly separated responsibilities and restricted permissions can help reduce the consequences of unexpected agent behaviour.

Continuous Verification Is Essential for Trustworthy AI

Traditional AI validation often focuses on predictive accuracy, bias, model drift, and overall performance. Autonomous AI requires additional checks because an agent’s decisions can lead directly to operational actions.

Organisations must establish whether an agent was authorised to act, whether it used appropriate information, whether it selected the correct tools, and whether its actions complied with business policies.

The SAS Data & AI Impact research cited by Profi found that 66% of organisations classified as having trustworthy AI practices maintain formal verification and validation processes, compared with 15% of organisations at earlier stages of maturity.

The difference highlights the value of structured testing and continuous oversight.

Rather than treating verification as a one-time step before deployment, enterprises should monitor AI systems throughout their operational life cycle. This includes reviewing decisions, checking compliance with established policies, and identifying deviations that require intervention.

Such measures can help businesses maintain confidence in AI systems while expanding automation across increasingly complex workflows.

Data Quality Remains a Major Barrier to Autonomous AI

Even advanced AI agents cannot make reliable decisions when the underlying enterprise data is fragmented, inconsistent, outdated, or difficult to access.

Customer records may be spread across multiple platforms, business definitions may conflict, and ownership of decision-making processes may be unclear. In these circumstances, an autonomous agent may operate with an incomplete understanding of the situation.

Profi argues that many AI challenges are actually problems involving data management and decision architecture.

The SAS research cited in the interview found that only 17.5% of organisations had reached fully optimised data and AI readiness. A life sciences example also illustrated the scale of the problem: scientists at one major biopharmaceutical company spent as much as 80% of their time finding, cleaning, and reconciling data before beginning analysis.

These findings show why data quality for AI, enterprise data integration, and AI readiness are fundamental to successful automation.

Before deploying autonomous agents at scale, businesses should improve data consistency, establish clear ownership, strengthen data lineage, and ensure that systems provide the information agents need to make informed decisions.

How AI Governance Can Accelerate Business Innovation

Although governance is often perceived as a barrier to innovation, Profi argues that well-designed controls can help companies deploy AI more efficiently.

Without reusable policies, every new AI project may require separate discussions among engineering, legal, security, risk, and compliance teams. This can create delays and inconsistent decisions across the organisation.

By contrast, organisations that translate policies into executable controls can establish standard rules for data access, approval requirements, escalation thresholds, audit logging, and permitted AI models.

Once these controls are embedded into the technology environment, teams can reuse them across multiple AI deployments instead of repeatedly rebuilding governance processes.

The SAS research cited by Profi also found that organisations with high AI trustworthiness scores were 15 times more likely to report strong or high returns on AI investment than those with low scores.

This suggests that trustworthy AI practices can support both risk management and business performance. Effective governance enables companies to define where autonomous systems can operate independently, when human approval is necessary, and how exceptions should be handled.

Finding the Right Balance Between AI Autonomy and Human Oversight

There is no single level of autonomy suitable for every AI application. The appropriate balance depends on the potential consequences of an error, the reversibility of an action, the uncertainty involved, and the sensitivity of the decision.

Low-risk tasks that can easily be reversed may be suitable for greater automation. Decisions affecting healthcare, credit, employment, or access to public services may require stronger safeguards and explicit human approval.

Profi distinguishes between requiring human approval for every action and maintaining human supervision over autonomous operations. Excessive manual review can slow down workflows and turn approval processes into routine formalities.

A more practical approach is to define clear operating boundaries, allow AI agents to execute approved tasks within those limits, and escalate exceptions to human supervisors.

This model preserves human visibility and intervention without requiring people to approve every routine transaction.

Ultimately, organisations can delegate execution to AI systems, but accountability for decisions and their consequences must remain with the people and institutions responsible for deploying them.

The Future of Enterprise AI Depends on Trust and Control

As businesses move from generative AI assistants to autonomous AI agents, the focus is shifting from what AI can generate to what it is permitted to do.

Successful enterprise AI adoption requires more than capable models. It depends on reliable data infrastructure, secure permissions, continuous verification, enforceable governance policies, and clearly defined human responsibilities.

The insights shared by Marinela Profi ahead of AI & Big Data Expo Europe reinforce an important message for business leaders: AI governance is not simply a compliance requirement; it is a foundation for scaling automation safely and sustainably.

Organisations that build these safeguards into their architecture from the beginning will be better positioned to expand AI-driven operations while managing operational risk, maintaining accountability, and strengthening trust in autonomous decision-making.

SAS is a key sponsor of AI & Big Data Expo Europe. Profi is scheduled to present “Why the Leaders Who Win the Agentic AI Race Won’t Be the Ones Who Moved Fastest” at the event, with further information available from SAS experts at stand #304.