Only a few months ago, I wrote about why AI and the mainframe could become a powerful combination.
Since then, something interesting has happened. The conversation is beginning to change. The question is becoming less about whether Artificial Intelligence has a place on the mainframe and more about how organisations can actually use it in production.
That distinction matters.
For the last few years, almost every organisation seems to have been experimenting with AI. Proofs of concept appeared everywhere. Chatbots were created. Developers experimented with coding assistants. Operations teams investigated AIOps, and organisations started looking at what generative AI could do with their data.
Experimentation is relatively easy. Production is different.
When AI becomes involved in systems processing payments, financial transactions, insurance claims or other critical business processes, suddenly performance, security, governance, resilience and trust matter considerably more. And this is where the mainframe becomes particularly interesting.
AI Where the Transactions Happen
IBM has been moving in this direction for some time. IBM z16 introduced the ability to perform AI inferencing directly on the platform using the first-generation Telum processor.
IBM z17 takes this considerably further. Its Telum II processor includes a second-generation integrated AI accelerator designed to perform high-volume AI inferencing close to the applications and transactions being processed.
IBM states that z17 can perform up to five million inference operations per second, with response times of less than one millisecond under its tested configuration.
Think about what that means. Rather than processing a transaction and subsequently moving information elsewhere for analysis, an organisation can potentially apply an AI model while that transaction is taking place:
- A payment can be evaluated for fraud.
- A financial transaction can be assessed for risk.
- An insurance claim can be analysed.
- Suspicious behaviour can be identified.
And this can happen close to the applications and data involved in the transaction. This isn’t simply about making AI faster. It changes the architecture.
Telum II Is Only Part of the Story
There is another development that makes the current generation of IBM Z particularly interesting. IBM Spyre Accelerator is now available on IBM z17. Telum II and Spyre address different parts of the AI problem.
Telum II is particularly suited to high-volume, low-latency inferencing integrated with transactional workloads.
Spyre extends the platform’s AI acceleration capabilities towards larger generative and agentic AI workloads.
Together, they make it possible to consider AI workloads that would previously have been assumed to belong somewhere else.
That opens some interesting possibilities. Imagine an operations engineer investigating a production problem and being able to ask questions about system information using natural language. Imagine an AI assistant analysing operational data, logs and configuration information without requiring sensitive information to be sent to an external AI service. Or a security analyst using AI to help correlate activity across enormous quantities of system information.
The important point is that AI is no longer necessarily something sitting outside the mainframe. It can increasingly become part of the platform itself.
But There Is a Bigger Question: Trust
Technology isn’t actually the most interesting part of this story. Trust is.
The recently published 2026 BMC Mainframe Survey provides an interesting indication of where the industry currently stands. The survey, which included more than 1,300 mainframe practitioners and decision-makers, indicates that organisations are moving from AI experimentation towards operational use.
But there is an important detail. Mainframe professionals appear considerably more comfortable allowing AI to identify problems and recommend actions than allowing it to perform those actions autonomously. That doesn’t surprise me.
There is a very big difference between an AI system saying:
“I believe you should make this change.”
and:
“I made the change.”
Particularly when we are talking about production environments supporting some of the world’s largest banks, insurers, retailers, governments and other critical organisations.
This is where the discussion about agentic AI becomes particularly important. An AI assistant that can analyse information is useful. An AI agent that can take action is potentially much more powerful.
But it also introduces an entirely different level of risk:
- What authority should an AI agent have?
- Which commands should it be allowed to execute?
- Which datasets should it be able to access?
- How do we authenticate an AI agent?
- How do we authorise it?
- How do we audit what it has done?
- What happens when it makes the wrong decision?
These aren’t theoretical security questions. They are questions organisations will increasingly need to answer.
Security Cannot Be an Afterthought
There is another reason why running AI close to mainframe data is attractive. Data doesn’t necessarily have to leave the environment.
For organisations operating in heavily regulated industries, this can be extremely important.
Financial information, customer records, authentication data and operational information may all contain sensitive data that organisations don’t necessarily want to send to an external AI platform.
Keeping AI closer to the systems holding that information can simplify some aspects of security, governance and data sovereignty. It doesn’t eliminate the risks. AI introduces its own attack surface.
Prompt injection, excessive permissions, sensitive-data leakage, model manipulation and poorly controlled autonomous agents all create new security challenges. The security model therefore needs to evolve alongside the AI model.
And this is where decades of experience with mainframe security become extremely relevant:
- Authentication.
- Authorisation.
- Least privilege.
- Separation of duties.
- Logging.
- Auditing.
- Change control.
These aren’t new ideas. We simply need to make sure we don’t forget them because AI is new.
Meanwhile, Another Security Challenge Is Approaching
AI isn’t the only technology forcing organisations to rethink security.
Quantum computing is doing the same.
IBM has continued developing quantum-safe capabilities around IBM Z, including IBM Z Crypto Discovery and Inventory.
One of the first problems organisations face when preparing for post-quantum cryptography is surprisingly simple:
Where are we using cryptography today?
In a large organisation, answering that question can be extremely difficult.
Certificates, TLS connections, SSH, applications, databases, APIs, middleware and decades of application development may all contain cryptographic dependencies.
Before organisations can migrate them, they need to find them.
IBM Z Crypto Discovery and Inventory can help organisations discover cryptographic usage and produce a Cryptographic Bill of Materials, providing a clearer picture of the cryptographic assets that may eventually need to change.
For large enterprises, that migration isn’t going to happen overnight. Preparation needs to start long before cryptographically relevant quantum computers become available.
Perhaps We Need to Redefine Mainframe Modernisation
For many years, the word “modernisation” was frequently associated with moving applications away from the mainframe.
I have never believed that was the only definition:
- Modernisation can mean APIs.
- It can mean DevOps.
- It can mean automation.
- It can mean better observability.
- It can mean modern development tools.
- It can mean integrating mainframe applications into hybrid architectures.
- And now it can mean AI.
The platform doesn’t have to disappear for an application to become modern. Sometimes the better answer is to modernise what already works.
The 2026 BMC Mainframe Survey provides an interesting indication of this. Ninety-four percent of respondents see the mainframe as either a long-term platform or a platform for new workloads, while 69 percent report growth in general-purpose capacity.
Those numbers don’t describe a platform waiting to disappear. They describe one that continues to evolve.
The Next Step Is the Interesting One
AI on the mainframe is no longer just an interesting experiment. The hardware exists. The accelerators exist. The software ecosystem is developing. The use cases are becoming clearer.
Now comes the difficult part:
- Governance.
- Security.
- Trust.
And deciding how much control we are prepared to give AI over the systems that run some of the world’s most important businesses.
Perhaps the next big mainframe security question won’t be:
“Who has access to the system?”
It might be:
“Which AI has access to the system, what is it allowed to do, and who is accountable when it does it?”
That is a conversation I believe the mainframe community needs to start having now.
Be the first to comment