Who Controls AI When AI Starts Taking Action?

Beyond Data Protection: Rethinking Authority, Accountability and Human Responsibility in the Age of Artificial Intelligence

Synopsis

Artificial Intelligence is entering a new phase. What began as technology designed to process information, generate content and assist human decision-making is gradually evolving into systems capable of accessing business applications, coordinating workflows and executing tasks with increasing independence. This transformation raises a fundamental question that goes beyond traditional discussions about data privacy and cybersecurity: when AI is authorised to take action, who determines the limits of its authority, and who remains accountable for the consequences?

Drawing from his experience in software development, enterprise digital transformation and AI implementation, Ts. Lukas J. Tan explores why the next challenge of AI governance is not simply protecting the information AI processes, but governing what AI is permitted to do with that information. Against the backdrop of Malaysia's proposed Artificial Intelligence and Personal Data Protection Framework, he argues that businesses, technology developers and policymakers must begin preparing for a future in which AI is no longer merely an assistant, but an increasingly active participant in organisational operations.

The Question We Have Not Been Asking

For many years, whenever we discussed digital transformation, cybersecurity or the development of information systems, one question consistently appeared at the centre of the conversation: how do we protect our data?

Businesses invested in databases, servers, firewalls, encryption, access controls and privacy policies. Governments introduced legislation to protect personal information, while organisations developed procedures to determine who could collect, process, store and disclose that information. As technology evolved, the methods became more sophisticated, but the underlying objective remained largely the same. We wanted to ensure that information was handled responsibly and that individuals could trust the organisations holding their data.

Today, Artificial Intelligence is beginning to challenge the boundaries of that conversation.

I have spent many years working in software development and enterprise digital transformation. Throughout that journey, I have witnessed organisations move from manual records to computerised systems, from standalone applications to interconnected platforms, and from traditional automation to increasingly intelligent software. Each technological transition has brought new opportunities, but it has also required businesses to reconsider how they manage operations, responsibilities and risks.

The arrival of Generative AI has accelerated this transformation significantly. In a relatively short period, businesses have begun using AI to prepare documents, analyse information, answer customer enquiries, generate marketing materials and assist employees with daily work. The discussion has largely focused on productivity, efficiency and the ability to complete more work with fewer resources.

These benefits are real. However, I believe we are approaching a more important turning point.

AI is no longer developing solely as a technology that produces answers. It is increasingly becoming a technology that can perform actions.

That distinction may sound simple, but its implications for businesses, governments and society are substantial.

When AI generates an answer, a human usually decides what to do next. When AI is authorised to execute an action, the system may directly influence business operations, modify information or initiate processes before a human reviews the outcome.

The question therefore changes.

Instead of asking only whether AI produces accurate information, we must begin asking whether AI has been given appropriate authority to act.And beyond that, we must ask who remains responsible when its actions have consequences.

From a Tool That Assists to a System That Acts

To understand this transformation, we should consider how software has evolved.

Traditional computer systems operate largely according to predefined instructions. A programmer develops rules, establishes conditions and determines how the application should behave. When a user performs an action, the system executes the relevant instructions. Although complex software can produce unexpected outcomes, the intended workflow is generally designed and controlled through explicit programming logic.

Generative AI introduced a different capability. Rather than relying only on fixed instructions, AI models can interpret natural language, recognise patterns, generate content and produce recommendations based on the information available to them.

For example, when an employee asks AI to summarise a customer report, the system analyses the document and prepares a summary. The employee then reads the result and decides whether to use it. If the summary contains an error, the employee has an opportunity to identify and correct it before taking further action.

Now imagine a different situation.

An AI system is connected to the company's customer database, sales application and internal communication platform. It receives a request to review outstanding customer enquiries, prepare follow-up communications, update relevant records and notify the responsible employees.

The system may retrieve information, identify pending tasks, decide which tools to use and execute several authorised steps in sequence.

This is where the discussion begins moving towards Agentic AI.

The significance of Agentic AI is not simply that the technology becomes more intelligent. It is that AI can be integrated into systems where its outputs may trigger actions within the real business environment.

The same underlying AI model might be used in both scenarios. However, the operational consequences are different.

An incorrect document summary may misinform an employee. An incorrect automated database update may alter business records. An incorrectly addressed communication may expose confidential information. An action taken without appropriate authorisation may affect customers, employees or commercial operations.

In other words, the risk of AI is not determined solely by the intelligence of the model. It is also influenced by the authority we give it, the systems it can access and the consequences of the actions it can perform.

This is an important distinction that I believe business leaders must understand before integrating autonomous AI systems more deeply into their organisations.

Data Protection Remains Essential, but the Conversation Must Evolve

When we discuss AI governance, personal data protection remains one of the most important foundations.

Organisations must continue asking why personal data is collected, how it is processed, who can access it, where it is stored and how long it should be retained. Individuals should be able to understand how their information is being used, while businesses should maintain appropriate safeguards against unauthorised access, misuse and disclosure.

These responsibilities do not disappear because an organisation introduces AI.

In fact, the increasing use of AI makes them even more important.

However, I believe there is an additional governance dimension that deserves greater attention.

Traditional data protection primarily addresses the handling of information. As AI becomes capable of taking action, we must also consider the authority assigned to the system.

For example, imagine that an AI application has legitimate access to a customer's personal information. The organisation has implemented the necessary security controls, and the system is permitted to process the information for an authorised business purpose.Does that authorisation automatically mean the AI should be able to change the customer's information? Should it be permitted to send messages on behalf of the company? Can it combine that information with records from another department? Should it be allowed to initiate a transaction based on its own interpretation of the customer's circumstances?

These are not simply questions of whether personal information is protected.

They are questions about the limits of operational authority.

The distinction between accessing information and acting upon information becomes increasingly important as AI systems gain autonomy.

In my view, future AI governance must address both dimensions. Organisations need to protect the information AI processes while also defining the actions AI is authorised to perform.

One responsibility cannot replace the other.

An organisation may have strong data security but poorly defined AI permissions. Conversely, an AI application may have carefully restricted functionality while still handling personal information inadequately.

Responsible AI implementation requires both data protection and operational control.

The Invisible Transfer of Authority

One of the most interesting developments I observe in enterprise AI adoption is how easily organisations can begin transferring operational authority to technology without fully recognising that they are doing so.

At the beginning, a company may introduce AI to support employees with routine tasks. The technology prepares email drafts, analyses spreadsheets or helps answer customer questions. Management sees improvements in productivity and naturally wants to extend those capabilities.

The next step may involve connecting AI to existing business applications. Instead of copying information into an AI interface, employees allow the system to retrieve records directly. This reduces manual work and improves convenience.

Later, the company may allow AI to update information, schedule activities or coordinate tasks between departments. Each enhancement appears to be a logical extension of the previous implementation.

Over time, however, the organisation may reach a point where AI is no longer simply assisting a workflow. It has become part of the workflow itself.

This transition can happen gradually.

No single decision may appear especially significant. Yet the cumulative effect can be substantial because the organisation has progressively expanded what AI is allowed to do.

I believe this creates a new responsibility for management.

Business leaders need to understand that connecting an AI system to more applications is not merely a technical integration exercise. It may also represent an expansion of operational authority.

The question is not whether such integration should be avoided. Integration is precisely where many of the benefits of enterprise AI can be realised.

The question is whether the organisation has deliberately considered the responsibilities and limitations associated with each additional capability.

An AI system does not need unrestricted authority to provide meaningful value. In many cases, carefully defined permissions and appropriate human oversight can support innovation while reducing unnecessary exposure.

The ability to delegate work does not remove the responsibility to determine how that work should be governed.

If AI Becomes Part of the Workforce, Who Manages It?

In conversations about the future of employment, we frequently hear that AI may become a digital employee, virtual assistant or member of an intelligent workforce.

These descriptions help people imagine the potential of AI. However, they can also create misunderstandings if taken too literally.

AI is not a human employee. It does not possess the same judgment, legal standing or personal accountability as an individual working within an organisation.

Nevertheless, there is a useful management lesson in the comparison.When a company recruits an employee, it normally defines the person's responsibilities, reporting relationships, access permissions and decision-making authority. A junior employee may prepare financial information without being authorised to approve payments. A customer service representative may access relevant customer records without receiving unrestricted access to the company's entire database.

The organisation establishes these boundaries because responsibility must be matched with authority.

Why, then, should the introduction of AI be treated as merely installing another software application?

As businesses begin deploying AI systems capable of performing increasingly complex tasks, management may need to develop clearer processes for assigning, reviewing and supervising AI responsibilities.

Some AI activities may require only basic oversight. Others may influence decisions involving employment, financial matters or sensitive personal information and therefore warrant stronger safeguards.

The appropriate controls should depend on the nature and consequences of the activity rather than the excitement surrounding the technology.

I believe this is where the role of business leadership will become especially important.

The technical team may understand how to connect an AI model to enterprise systems. But management must determine the organisation's objectives, acceptable risks, authority boundaries and accountability arrangements.

Technology teams cannot make every governance decision on behalf of business leadership.

At the same time, management cannot reasonably govern sophisticated AI systems without understanding their practical capabilities and limitations.

Effective AI governance therefore requires cooperation between leadership, operations, technology and relevant compliance functions.

The future of enterprise AI will not be determined by technology alone. It will also depend on whether organisations develop the management discipline necessary to use that technology responsibly.

The Difference Between Having a Policy and Having Control

Another issue deserves attention.

Many organisations measure governance maturity through documentation. They maintain policies, prepare procedures, record approvals and conduct periodic reviews. These activities are important because they establish expectations and provide evidence of organisational responsibility.

However, a written policy does not automatically mean that effective operational controls exist.

An organisation may have a comprehensive AI policy while allowing employees to connect AI applications to business systems with unnecessarily broad permissions. It may require human approval for certain activities without establishing a reliable way to enforce that requirement. It may maintain activity logs without regularly reviewing significant events.

The organisation can appear well governed on paper while important operational weaknesses remain unresolved.

This challenge is not unique to AI. Similar issues have existed for many years in cybersecurity, information technology management and corporate governance.

What makes AI particularly interesting is that the system's behaviour and capabilities may change over time.

An AI application might initially perform a narrow function. After a software update, new integration or change in permissions, it may gain access to additional information or become capable of executing new tasks.

The organisation may not regard these changes as a completely new system deployment, even though its operational risk has changed.

This is why I believe AI governance cannot be treated as a one-time compliance exercise.

It requires ongoing awareness of how the system operates, what permissions it holds, what actions it can perform and whether those capabilities remain consistent with the organisation's original objectives.

Policies should guide behaviour, but effective governance must also be visible in daily operations.Ultimately, the quality of an AI governance programme should be reflected not only in what an organisation says it controls, but in what it can demonstrate it actually controls.

A New Responsibility for AI Developers and Technology Providers

The emergence of increasingly autonomous AI also changes the conversation for software developers and technology providers.

In a traditional software project, responsibilities are often established through system specifications, contractual arrangements and agreed business processes. The developer builds the application, while the customer determines its intended business use and manages the relevant operations.

Modern AI solutions can introduce additional participants.

An enterprise application may use a foundation model developed by an international technology company, cloud infrastructure provided by another organisation, integration services delivered by a local software developer and business data controlled by the customer.

In some cases, these services may operate across different countries and contractual relationships.

When an AI-related problem occurs, identifying responsibility may not be straightforward.

The application developer may control the user interface and integrations but not the underlying foundation model. The model provider may control parts of the model service but not the customer's business workflow. The customer may determine the intended purpose but depend on external technology providers for technical implementation.

Existing legal responsibilities remain applicable, including those under relevant personal data protection laws. Nevertheless, practical guidance on the allocation and coordination of responsibilities can help organisations comply more consistently.

In my view, this is especially important for Malaysia's local technology industry.

Many Malaysian software companies and SMEs are not developing large foundation models from the beginning. Instead, they are building applications, integrating existing AI technologies and helping businesses apply AI to practical problems.

Their ability to inspect or control the internal workings of third-party AI models may be limited.

A practical governance approach should therefore recognise the difference between what an organisation directly controls, what it can reasonably verify and what it must address through contractual assurances and supplier oversight.

This is not an argument for reducing accountability.

It is an argument for making accountability sufficiently clear that organisations can meaningfully implement it.

Malaysia's Opportunity to Shape Responsible AI Adoption

In October 2026, Malaysia's Personal Data Protection Department, or JPDP, issued Public Consultation Paper No. 1/2026 concerning the proposed Artificial Intelligence and Personal Data Protection Framework under the Personal Data Protection Act 2010, Act 709.

The consultation represents an important development in Malaysia's approach to responsible AI adoption.

The proposed Framework addresses the processing of personal data throughout the AI system lifecycle, including development, testing, deployment, external AI services, data subject rights, security, fairness, risk management and compliance documentation.

I welcome this direction.

The protection of personal information is not an obstacle to technological progress. On the contrary, it is one of the conditions that enables individuals and businesses to adopt technology with greater confidence.

As part of my industry feedback, I have highlighted an additional question that I believe deserves further consideration as AI capabilities continue to evolve.

Should the level of autonomy and operational authority granted to an AI system become a more explicit consideration in governance and risk assessment?

The proposed Framework already recognises different levels of autonomy, access controls, human oversight and the importance of reviewing material changes to AI systems. These provide a useful foundation.My observation is that the relationship between these elements deserves further discussion, particularly when AI is permitted to execute actions across connected enterprise systems.

An AI application that generates recommendations and an AI application that independently executes business activities may process similar information, yet present different operational risks.

Recognising this distinction could help ensure that regulatory guidance remains relevant as Agentic AI becomes more widely adopted.

It is equally important that governance expectations remain proportionate.

Not every AI agent should automatically be classified as high risk. A system performing limited internal administrative tasks under appropriate controls is different from one that can independently execute actions with significant consequences for individuals.

Malaysia has an opportunity to develop a practical approach that recognises such differences while maintaining strong personal data protection principles.

This would support not only regulatory clarity, but also responsible innovation among local software developers and SMEs.

The Real Challenge Facing Malaysian SMEs

For many Malaysian SMEs, the challenge of AI adoption is not a lack of interest.

Business owners increasingly recognise that AI can improve efficiency, reduce repetitive work and support better decision-making. However, many do not have dedicated teams specialising in AI engineering, data protection, cybersecurity or technology governance.

They often depend on external vendors and commercially available applications.

As a result, the success of AI governance will depend partly on whether its principles can be translated into guidance that smaller organisations can understand and implement.

If governance becomes excessively complicated, costly or disconnected from operational realities, businesses may struggle to adopt it effectively.

On the other hand, if governance requirements are too vague, organisations may misunderstand their responsibilities or implement controls that do not adequately address the risks involved.

Neither outcome is desirable.

I believe Malaysia should continue pursuing an approach that balances innovation, responsibility and practical implementation.

This includes providing greater clarity on third-party AI services, explaining how risk-based controls can be applied to different use cases, and supporting industry awareness and capability development.

A framework becomes valuable when it can be understood not only by policymakers and legal professionals, but also by the people responsible for developing, operating and managing technology every day.

Responsible AI should not become a privilege available only to large corporations with extensive resources.

It should be achievable for smaller organisations through safeguards appropriate to their activities and risks.

The Human Role Becomes More Important, Not Less

As AI becomes capable of completing increasingly sophisticated tasks, it is natural to question the future role of human beings in organisations.

Will AI replace employees? Will managers become less necessary? Will decisions eventually be made entirely by machines?

I believe the deeper issue is not simply whether AI can perform work previously completed by humans.

It is how human responsibility changes when more work can be delegated to intelligent systems.

For many years, the value of technology has been measured by its ability to automate tasks. A successful system reduces manual effort, improves consistency and enables employees to focus on higher-value activities.

AI expands those possibilities.

But when technology becomes capable of performing more complex work, the importance of defining objectives, evaluating consequences and exercising judgment does not disappear.

In some circumstances, it becomes even greater.A manager who delegates an important task to an employee remains responsible for establishing appropriate expectations and oversight. Similarly, an organisation that authorises AI to perform business activities cannot assume that the technology itself has taken over the organisation's accountability.

The ability to delegate an action and the responsibility for authorising that action are two different matters.

This distinction may become one of the defining management challenges of the AI era.

We will need leaders who understand not only how to use AI, but also when to rely on it, when to question it and when human intervention is necessary.

We will need developers who think not only about functionality and performance, but also about the consequences of system integration and automated actions.

We will need employees who understand how to work productively with AI without surrendering their professional judgment.

And we will need governance practices that recognise both the capabilities and limitations of increasingly autonomous technology.

The future of human leadership may depend less on controlling every individual task and more on exercising sound judgment over the responsibilities we choose to delegate.

Building AI into Business Means Building Responsibility into AI

One of the principles I frequently emphasise is the importance of integrating AI into everyday business operations.

AI should not exist only as an interesting experiment or a tool occasionally used by employees. Its greatest organisational value often emerges when it is appropriately connected to real business needs, workflows and decision-making processes.

But meaningful integration also requires responsibility.

The more deeply AI participates in business operations, the more carefully organisations must consider its authority.

Who decides which information the system may access? Who determines which activities it can execute? Who monitors important outcomes? Who intervenes when the system behaves unexpectedly? Who answers when its actions affect another person?

These are management questions as much as technical questions.

I believe the companies that succeed in the next phase of AI adoption will not necessarily be those that deploy the largest number of AI tools.

They may instead be the organisations that understand how to combine technological capability with clear responsibilities, sensible controls and sound human judgment.

The objective is not to slow down innovation.

It is to develop the confidence and organisational discipline required to innovate sustainably.

When governance is understood and implemented properly, it can become an enabler of innovation rather than merely a compliance obligation.

Who Ultimately Controls AI?

Artificial Intelligence will continue to evolve. New models, applications and autonomous systems will introduce capabilities that may be difficult to imagine today.

The technology will become faster, more capable and increasingly integrated into how businesses operate.

However, technological advancement alone does not determine how society should use those capabilities.

Those decisions remain matters of human values, organisational responsibility, governance and judgment.

We should welcome AI's ability to improve productivity and support innovation. We should continue protecting personal information and strengthening confidence in digital services. We should encourage businesses, especially SMEs, to explore how AI can contribute to sustainable growth.

At the same time, we must recognise that the ability of AI to take action introduces responsibilities that go beyond conventional discussions about data processing.

The next stage of AI governance must consider not only what AI knows, but what AI is authorised to do.

Not only how information is protected, but how actions are controlled.

Not only whether a system is capable of making decisions, but who remains accountable for the consequences.

Perhaps the defining question of the next decade will not be whether AI can perform a particular task.We already know that its capabilities will continue expanding.

The more important question is whether we are prepared to exercise sound judgment over the authority we give it.

We have spent years teaching machines how to process information. We are now developing machines that can take action. The responsibility ahead is to determine what they should be allowed to do, under whose authority, and with what safeguards.

Because ultimately, the future of AI should not be defined only by what technology is capable of achieving.

It should also be defined by the wisdom of the people who choose how that technology is used.

And that brings us back to a question every business leader, developer and policymaker should consider:

When AI starts taking action, who is really in control?

Ts. Lukas J. Tan is the Founder & CEO of OPERiON Ecommerce & Software Sdn Bhd. He writes and speaks about Artificial Intelligence, enterprise digital transformation, technology leadership and responsible AI implementation.

This article reflects his independent perspective as an industry practitioner. It draws on themes from his October 2026 position paper, "Beyond Data Protection: Governing AI Autonomy, Accountability and Real-World Deployment", prepared in connection with Malaysia's JPDP Public Consultation Paper No. 1/2026. The views expressed do not represent an official position of JPDP.