ICO tech futures: Agentic AI

Key Take Aways

• Agentic AI is rapidly evolving, combining generative AI capabilities with external tools, increasing autonomy, contextual understanding, and open-ended task automation relevant to financial services.
• The development and deployment of agentic AI pose significant data protection and privacy risks, including risks related to transparency, accountability, data minimisation, and the potential for unintended inferences, especially in high-risk sectors.
• Organisations are responsible for ensuring compliance with data protection laws, even as agents operate with increasing autonomy and less human oversight.
• Potential use cases in financial services include automated customer interactions, complex transactional planning, fraud detection, and enhanced cybersecurity, but deploying in high-stakes environments demands rigorous governance.
• There are four future scenarios of agentic AI development, ranging from low adoption and capabilities with limited risk to ubiquitous, high-capability systems which could significantly strain regulatory oversight.
• Technical developments, such as multimodal agents, multi-agent systems, embedding into IoT, and self-improving agents, will expand operational possibilities but also complicate privacy and security controls.
• The risk of cascading inaccuracies (‘hallucinations’) and the creation or inference of sensitive data can lead to substantial harms, especially if not effectively mitigated.
• Organisations should prioritise privacy by design, transparency, and effective governance systems, including real-time monitoring and documentable decision logs, to manage increasing complexity.
• A variety of business models, especially those relying on extensive personalisation or embedded data, could centralise vast quantities of personal information, heightening surveillance and data breach risks.
• The evolving role of Data Protection Officers (DPOs) may include overseeing ‘agentic AI’ governance through emerging ‘DPO agents’ and enhanced oversight tools.
• The report underscores the importance of collaborative, international regulatory engagement and proactive scenario planning to manage technological and societal uncertainties.
• Innovation opportunities exist in developing data protection compliant agents, privacy-enhancing controls, and effective benchmarks for measuring agentic AI compliance and performance.

Key Statistics

• AI companies raised over £14 billion revenue in 2023, indicating rapid market growth.
• Venture capital funding for agentic AI startups in 2025 was approximately USD $2.8 billion, with 10% of all AI funding focused on agentic applications.
• The cost of training AI models is expected to escalate to billions of dollars by 2027, risking power concentration among large providers.
• Analyses estimate that the physical compute required for AI applications is decreasing at three times per year due to efficiency gains.
• One in six UK organisations uses at least one type of AI in the workplace, with the largest adoption in IT, legal, and healthcare sectors.
• Gartner predicts that over 40% of agentic AI projects could be cancelled by 2027 due to unclear business value or risk management challenges.
• The UK Government estimates that AI could create around 6,500 jobs in the sector if investments continue at current rates.
• 40% of agentic AI projects may be subject to failure because of hype-driven misapplication, according to Gartner.

Key Discussion Points

• The increasing autonomy of agentic AI systems elevates data protection challenges, notably around transparency and control.
• Organisation responsibilities persist regardless of agentic AI’s decision-making autonomy; human oversight remains critical.
• The potential for agentic systems to process, infer, or generate sensitive personal data in unexpected ways raises compliance and ethical considerations.
• Future scenarios range from limited, cautious adoption to widespread deployment of high-capability, highly autonomous agents, influencing regulatory and operational landscapes.
• Developers must embed data protection principles such as purpose limitation and data minimisation into agentic AI architectures.
• Risks of hallucinations and cascading errors in complex agentic systems could lead to significant harms, demanding improved technical safeguards.
• The role of Data Protection Officers may evolve towards managing ‘agentic AI’ governance, monitoring, and automated compliance.
• Organisations need flexible, evolving governance frameworks capable of handling multi-agent ecosystems and complex decision logs.
• The proliferation of agent-to-agent communication introduces transparency and oversight challenges, especially in privacy-sensitive contexts.
• There are recognised opportunities to innovate in privacy-compliant agent design, including privacy-enhancing technologies and secure, localised processing.
• Cross-jurisdictional regulatory coherence and international collaboration are vital to managing agentic AI risks at scale.
• Strategic scenario planning will be essential to prepare for multiple possible futures, ranging from conservative to optimised, privacy-respecting deployments.

Document Description

This article is an in-depth exploration of emergent agentic AI technologies and their implications for data protection, privacy, security, and regulation. It provides a comprehensive overview of current capabilities, technical developments, potential use cases across sectors including financial services, and future scenarios ranging from low to high adoption and capability. The report highlights the risks associated with increasing autonomy, complex data flows, and multi-agent ecosystems, alongside opportunities for innovation in responsible AI design. It encourages proactive governance, international cooperation, and scenario planning to ensure that innovation aligns with legal and ethical standards, safeguarding personal rights in an evolving technological landscape.