The Hidden Responsibility Behind Giving Intelligent Systems Access to Our Digital World
Synopsis
As Artificial Intelligence becomes increasingly integrated into business operations, organisations are connecting intelligent systems to email platforms, customer databases, financial applications, internal documents and enterprise software. These connections make AI more capable, allowing it to retrieve information, coordinate activities and execute tasks across different environments. Yet every additional connection introduces an important question that is often overlooked in the excitement surrounding automation: just because an AI system can access something, does that mean it should? Drawing from his experience in software development, enterprise transformation and AI implementation, Ts. Lukas J. Tan explores the distinction between technological capability and authorised responsibility. He argues that the next stage of responsible AI adoption requires businesses to think beyond data security alone and examine how permissions, authority, human judgment and organisational trust must evolve alongside increasingly autonomous systems.
The Moment Technology Becomes More Powerful Than We Intended
There is a particular satisfaction that comes from connecting two systems that previously operated independently. For someone who has spent many years developing software, I have always appreciated the possibilities created when information begins flowing more efficiently between applications. A customer management system can communicate with an accounting platform, a website can connect directly to business operations, and information that once required manual handling can become available almost immediately. Every successful integration offers the possibility of improving productivity, reducing errors and allowing people to concentrate on work that creates greater value.
Artificial Intelligence has introduced an entirely new dimension to this process. Instead of connecting applications only through predefined workflows, we can increasingly build intelligent systems that interpret instructions, retrieve information and decide how to use the tools made available to them. An AI assistant may begin by summarising documents, but with additional integrations, it can become capable of examining customer records, coordinating administrative activities, preparing communications and supporting operational decisions. For a software developer, these possibilities are exciting because they offer new ways to solve problems that previously required considerable technical effort and human coordination.
However, I have noticed that enthusiasm for increasing technological capability can sometimes cause us to overlook a more fundamental question. As a system becomes capable of accessing more information and performing more activities, are we deliberately expanding its authority, or are we simply enabling whatever functionality is technically available?
The distinction is important because technology does not become responsible merely by becoming more capable. An AI model may be able to interpret a request, identify a relevant database and execute a particular action, but that does not establish whether the action is appropriate for the organisation or consistent with the permissions that should govern its operations. An intelligent system can perform a technically valid action while still producing an outcome that management never intended.
I believe this is one of the defining challenges emerging from the development of Agentic AI. The question is no longer limited to whether an application functions correctly. We must increasingly ask whether the system possesses the appropriate authority to perform each function, whether the organisation understands the boundaries of that authority and whether those boundaries remain effective as the technology evolves.
From Information Access to Operational Authority
In traditional business software, access permissions have long been an essential aspect of system design. Employees may receive different privileges depending on their responsibilities, while administrators establish boundaries intended to protect information and maintain operational control. An accounting employee may require access to financial records, but that does not automatically mean the employee should be authorised to approve every payment. A salesperson may need customer information to manage relationships, but it would rarely be appropriate to provide unrestricted access to confidential employee records.
These distinctions seem natural when we discuss people because organisations understand that the ability to access something is not the same as having legitimate authority to use it for every purpose. Yet when AI applications are introduced, the same principles can become less visible. Business owners may focus on connecting the necessary systems and enabling the desired automation without examining whether the permissions granted to the AI are broader than the task actually requires.
Consider an AI application designed to help a sales team prepare customer follow-up communications. Initially, the system may need only limited access to relevant customer information and historical correspondence. If the company later allows the same application to modify customer records, create quotations or send messages automatically, its responsibilities have expanded considerably. The change may appear to be a routine improvement in functionality, but it represents a meaningful increase in operational authority.
The implications become more significant when AI is connected to multiple enterprise systems. An intelligent agent with access to email, internal documents, customer information and business applications may be capable of combining information that was previously separated across departments. Such integration can improve efficiency, but it can also create new pathways through which information may be used, disclosed or acted upon in ways that the organisation did not originally anticipate.
In my view, this is why AI governance must extend beyond the conventional question of whether data is secure. Security remains essential, but organisations must also examine whether the system is authorised to use available information for a particular purpose and whether the actions it performs remain within appropriate business boundaries.
The fact that information is accessible does not automatically make every possible use of that information legitimate or appropriate. Responsible implementation requires a clear distinction between what a system is technically capable of doing and what the organisation has deliberately authorised it to do.
The Invisible Expansion of Permission
One of the most interesting characteristics of modern software development is how easily capabilities can accumulate over time. An application rarely remains exactly as it was when first deployed. Developers introduce improvements, businesses request new functions, and integrations are added as operational requirements evolve. These changes are often beneficial because they allow software to adapt to the needs of an organisation.
With AI, however, the consequences of incremental changes deserve particularly careful consideration. A system that initially performs a limited task may gradually receive additional permissions without undergoing a complete reassessment of its operational role. New APIs may be connected, access to further databases may be granted, and automated actions may be introduced as confidence in the technology increases. Each enhancement appears reasonable when considered independently, yet the combined result may be a system with substantially greater authority than originally intended.
This creates what I consider an important management challenge: the invisible expansion of permission. Organisations may recognise that their AI has become more functional without fully recognising that it has also become more influential within their operations.
In software development, a permission setting may appear to be a small technical detail. In business governance, however, that setting can determine whether a system is allowed to observe information, modify records, communicate externally or initiate an action with real consequences. The same technical connection may therefore carry very different levels of responsibility depending on how it is configured and used.
My experience has made me increasingly sensitive to these distinctions. I enjoy building systems that can perform more work, but I also recognise that the ability to create an integration does not remove the need to consider its consequences. A well-designed system should not require unlimited authority to provide meaningful value. In many circumstances, its usefulness can be preserved while limiting access and maintaining appropriate human oversight.
This is not an argument against technological progress. It is an argument for ensuring that progress remains deliberate rather than accidental. When an organisation expands the capabilities of an AI system, it should also understand how that expansion changes the system's responsibilities, risks and relationship with the people whose information it processes.
The Data Behind Every Digital Action Belongs to Someone
As developers, we frequently work with databases, records, identifiers and information structures. These elements are necessary for building applications, and they allow organisations to manage large volumes of information efficiently. Over time, it can become easy to think about data primarily in technical terms: fields, records, tables, access permissions and processing requirements.
Yet I believe one of the most important habits in responsible software development is remembering that personal data is not merely a technical resource. Behind many records is an individual whose circumstances, expectations and privacy may be affected by how that information is handled.
A customer database may contain information about personal preferences, purchasing behaviour and communication history. An employee system may contain confidential employment details. A recruitment application may contain information about education, experience and career opportunities. When AI is introduced into these environments, the organisation must consider not only whether the technology can process the information accurately, but also whether the processing is appropriate for the intended purpose.
This sensitivity has become increasingly important to me because modern AI systems can bring together information from multiple sources with extraordinary convenience. The ability to combine information may create useful insights, but it may also introduce risks when the combined information reveals details that were not apparent in the original records. Data that seems ordinary in one context can become more sensitive when connected to other information.
I do not believe businesses should avoid using data simply because it carries responsibility. Information is essential to modern operations, and intelligent analysis can help organisations make better decisions. However, the ability to generate additional insights does not automatically justify every possible use of personal information.
Responsible AI development requires an understanding of purpose, proportionality and the expectations of the people affected by the system. These considerations should influence how we design applications and how we determine which information an AI system is allowed to access.
For me, protecting data is ultimately about protecting trust. A technically successful system can still damage confidence if people believe their information has been used beyond reasonable expectations. Rebuilding that confidence may be considerably more difficult than correcting a software defect.
The Human Consequences of a Technically Correct Action
Not every problem created by automation involves a security breach or an incorrect database operation. Sometimes, an AI system may perform exactly the action it was instructed to perform while creating an outcome that is inappropriate in the human context.
Imagine an AI assistant that is authorised to send follow-up messages to customers who have not responded to earlier communications. The application may accurately identify pending enquiries, prepare professional messages and deliver them according to a schedule. From a technical perspective, the automation works as intended.
However, a customer may be dealing with circumstances that are not visible to the system. A previous reply may have suggested hesitation or dissatisfaction. The relationship may require a more personal approach, or the timing of the communication may be inappropriate.
I have always been sensitive to these aspects of human interaction. Sometimes, the wording of a message makes me reconsider how a situation should be handled. A particular phrase or an unexpected response may prompt an inner feeling that there is more to understand. I do not treat that feeling as evidence, because human interpretation can be mistaken, but I regard it as a reason to examine the context more carefully.
This distinction matters when AI is authorised to communicate or act on behalf of an organisation. The system may be capable of producing convincing language, but persuasive wording does not guarantee an understanding of the relationship behind the conversation. An automated response may be accurate and grammatically correct while still creating an impression that the organisation is insensitive or unwilling to listen.
Such outcomes remind me that governance involves more than controlling technical operations. It also involves understanding the consequences of allowing technology to act within relationships shaped by human expectations, uncertainty and emotion.
The objective should not be to require human approval for every routine message or administrative activity. That would undermine many of the advantages of automation. Instead, organisations should recognise that different actions carry different consequences and that some situations require more careful treatment.
Good judgment involves knowing when speed is appropriate and when a person deserves greater attention.
Why Governance Must Move Alongside Innovation
There is a tendency to view governance as something introduced after technology has been developed. A team builds an application, demonstrates its functionality and then considers the policies and procedures required to operate it responsibly. This approach may appear efficient because technical development progresses without interruption, but it can create difficulties when governance concerns are discovered only after the system has become deeply integrated into business operations.
I believe governance should be considered alongside development rather than treated as a separate activity that follows it. This does not mean every software experiment requires extensive documentation or formal approval. Exploratory work is essential to technological innovation, and developers need room to test ideas, learn from mistakes and discover possibilities that were not initially anticipated.
However, when a system begins interacting with real business information or receiving authority to execute operational actions, its responsibilities change. The developer and the organisation should understand those changes before treating the system as a routine part of daily operations.
This principle reflects how I prefer to approach technology. I want to move quickly, experiment and develop practical solutions, but I also want to remain conscious of the conditions under which those solutions will operate. The objective is not to create unnecessary bureaucracy. It is to recognise that different activities require different levels of care.
A reversible experiment using non-sensitive test information presents a different situation from an automated workflow that modifies live customer records. An internal assistant preparing draft material carries different implications from a system sending messages to external parties. The appropriate governance arrangements should reflect these differences.
In my view, effective governance makes innovation more sustainable because it allows organisations to understand the boundaries within which technology can operate confidently. When permissions, responsibilities and escalation processes are clear, developers and business users can move forward without relying entirely on assumptions.
The strongest implementation is not necessarily the one with the fewest restrictions. It is the one where the controls are proportionate to the risks and sufficiently clear that everyone involved understands what the system is intended to accomplish.
The Business Owner Must Remain in Control
As intelligent systems become more capable, business owners may increasingly rely on technology providers to determine how AI should operate. This is understandable because many organisations do not possess extensive internal expertise in software engineering, cybersecurity or artificial intelligence. SMEs in particular often depend on external developers and commercial technology platforms to introduce new capabilities.
Nevertheless, the decision to delegate work to AI remains a business decision as well as a technical one. A developer can explain what the system is capable of doing, but the organisation must understand why those capabilities are necessary, which activities are appropriate and what responsibilities accompany their use.
This does not mean every business owner must become a software engineer. In fact, one of the responsibilities of technology practitioners is to translate complex technical possibilities into practical choices that management can understand. A responsible implementation process should help leaders recognise the implications of different levels of automation and make informed decisions about the authority being granted.
For local software developers and technology consultants, this creates an opportunity to contribute more meaningfully to digital transformation. Our role should not be limited to delivering requested functionality. We can also help organisations examine their processes, identify unnecessary complexity and understand how emerging technologies may affect their operations.
I believe this is particularly important as Malaysia continues developing its AI ecosystem. The country's proposed Artificial Intelligence and Personal Data Protection Framework provides an opportunity to examine how existing data protection responsibilities apply to increasingly capable AI systems. As the technology evolves, industry dialogue can help clarify practical issues surrounding external AI providers, access permissions, human oversight and accountability.
The ultimate objective should be responsible innovation that businesses of different sizes can understand and implement. Governance should not exist only as a theoretical framework. It should help organisations make better decisions about how technology is used.
The Authority We Give AI Reflects the Judgment We Exercise
When I look towards the future of enterprise AI, I do not believe the most successful organisations will necessarily be those that connect the greatest number of applications or deploy the largest number of agents. These measurements may indicate technological ambition, but they do not automatically demonstrate responsible management or meaningful business value.A more important measure may be whether the organisation understands the authority it has delegated and can explain why that authority is appropriate.
AI should be allowed to perform tasks that create genuine value, and businesses should remain willing to explore increasingly sophisticated applications. I believe strongly in innovation, practical development and the courage to move beyond familiar limitations. At the same time, I recognise that every additional capability deserves to be considered in relation to its purpose, consequences and effect on the people involved.
For me, the relationship between building and governance has become increasingly inseparable. I want systems that work, but I also want to understand what they are permitted to do and how their actions will be supervised. I want businesses to move quickly, but not so quickly that they lose awareness of the information, relationships and responsibilities affected by their decisions.
This is the perspective I bring to conversations about enterprise AI implementation. Before discussing how many applications can be connected, I am interested in understanding the organisation's objectives, the information required and the decisions that should remain under human control. Often, the quality of these early questions determines whether an AI project becomes a meaningful business improvement or simply another layer of technical complexity.
I welcome discussions with business leaders, technology practitioners and industry organisations that are exploring how intelligent systems can be integrated into operations responsibly. The opportunities are significant, but so is the responsibility to ensure that technology serves the organisation without exceeding the authority it has been given.
As AI becomes more capable, we should continue asking what it can do. That question drives innovation and helps us discover possibilities that were previously unavailable. But we must also develop the discipline to ask what it should be allowed to do, because that question determines whether innovation remains aligned with human judgment, organisational responsibility and trust.
The future of responsible AI will not be defined by unlimited access. It will be defined by the wisdom with which we grant authority.
And that leads me to the question I believe every business leader should consider before connecting another system to an AI agent: if the technology can access almost everything, have we taken enough care to decide what it actually needs?
Ts. Lukas J. Tan is the Founder & CEO of OPERiON Ecommerce & Software Sdn Bhd. A software development and enterprise digital transformation practitioner since 2007, he writes about Artificial Intelligence, responsible technology implementation, business leadership and the relationship between innovation, governance and human judgment.
For discussions on enterprise AI implementation, software development and responsible AI governance, visit [www.lukastan.com](http://www.lukastan.com/) or contact [email protected]_._this is article for social media manager