AI and the Mainframe: Practical Applications

Artificial Intelligence is becoming increasingly relevant to the mainframe, with applications ranging from fraud detection and security monitoring to systems management, database administration and application development. But beyond the announcements and promises, what can we actually do with it?

Artificial Intelligence is everywhere these days. Hardly a week goes by without another announcement about a new AI model, an agent capable of performing increasingly complex tasks, or a product promising to transform the way organisations operate.

Naturally, the mainframe industry is no exception. IBM has been investing heavily in AI capabilities for IBM Z, introducing new processors, accelerators and software designed to bring AI closer to enterprise applications and data. Other mainframe software vendors, including BMC, have also been introducing AI capabilities into products used to develop, manage and operate these environments.

All very interesting, but I think we should be asking a more practical question.

What can we actually do with AI on the mainframe, and where does it make sense to use it?

After all, the mainframe has been supporting business-critical workloads for decades. It is a platform recognised for its reliability, security, resilience and ability to process enormous volumes of transactions. Any new technology introduced into these environments needs to deliver genuine benefits without compromising those characteristics.

AI is no different.

There are several areas where I believe AI can make a meaningful contribution. Some already have commercial solutions available, while others represent opportunities that organisations can explore using their existing applications and operational information.

Let’s look at a few of them.

AI on the Mainframe: A Few Things Worth Understanding

Before getting into practical applications, it is worth clarifying what we mean by running AI on the mainframe.

We aren’t necessarily talking about installing a large language model on z/OS and allowing it to interact directly with production applications.

There are different ways of integrating AI into a mainframe environment.

An application running on z/OS can invoke an AI service hosted on another platform. AI workloads can run on Linux on IBM Z. Alternatively, supported models can use the AI acceleration capabilities provided by IBM Z processors and associated software.

Each approach has its advantages and limitations, particularly when it comes to performance, latency, security, data movement and integration with existing applications.

With the IBM z16, IBM introduced the Telum processor, which includes an integrated AI accelerator designed for supported inference workloads. IBM z17 builds on this with Telum II, providing a second-generation integrated AI accelerator.

IBM has also introduced the Spyre Accelerator, providing additional capabilities for AI inference, including generative and agentic AI workloads.

These technologies aren’t interchangeable. They support different types of workloads, and the software and model requirements need to be understood before deciding on an architecture.

There is also a difference between training an AI model and using that model to make predictions.

Training involves developing or updating a model using data. Inference involves using a trained model to produce a result, whether that is a prediction, a classification or a generated response.

The integrated AI accelerators in Telum and Telum II are primarily designed to accelerate supported inference workloads, rather than train large language models.

IBM provides more information about these capabilities in its AI on IBM Z overview.

It is also worth remembering that hardware acceleration and application-level AI are different parts of the overall architecture.

A mainframe software product might use machine learning to analyse operational data or generative AI to explain source code without necessarily executing its models on the Telum or Spyre accelerators.

The deployment architecture depends on the product, its supported configurations and the organisation’s requirements.

This is particularly important when comparing capabilities from different vendors. A product that uses AI to investigate mainframe performance problems isn’t necessarily running its AI model on the mainframe itself.

With that out of the way, where can we put AI to practical use?

Real-Time Fraud Detection

Fraud detection is probably one of the most obvious applications of AI in a mainframe environment.

Banks and financial institutions have been using statistical analysis, business rules and machine learning to identify potentially fraudulent transactions for many years.

What makes the latest IBM Z capabilities particularly interesting is the possibility of performing supported AI inference as part of the transaction-processing flow, without necessarily sending the transaction information to an external scoring service.

Imagine a customer making a payment.

The application receives the transaction and collects the relevant information, such as the amount, account details, transaction type and other available contextual data.

An AI model analyses the information and produces a fraud-risk score. The application can then use that score, together with existing business rules and controls, to decide whether the transaction should be approved, challenged or referred for further investigation.

The interesting part is that the scoring can take place while the transaction is being processed.

For high-volume environments, avoiding unnecessary data movement and external service calls can be particularly valuable.

IBM provides software capabilities for this type of architecture through Machine Learning for IBM z/OS, which supports deploying and invoking machine learning models close to z/OS applications and transactional workloads.

Depending on the model format and software configuration, supported inference workloads can use IBM Z’s integrated AI acceleration capabilities. Not every model or execution path automatically benefits from hardware acceleration.

However, the fact that a processor can execute an AI model quickly doesn’t mean that the complete transaction will automatically be faster.

The application still needs to prepare the information required by the model, execute the inference and process the result. All of this needs to happen within the transaction’s response-time requirements.

There is also the question of what happens when the model is unavailable or produces an unexpected result.

Does the application continue using its existing business rules? Does it reject the transaction? Does it request additional verification?

These are business and architectural decisions that need to be made before the solution goes into production.

I find this application particularly interesting because it demonstrates how AI can complement existing transaction-processing capabilities rather than replace them.

The AI model provides additional information. The application remains responsible for enforcing the business rules and making the final decision.

Intelligent Operations and Performance Management

Anyone who has worked in mainframe systems programming or operations will be familiar with the amount of information these environments generate.

SMF records, RMF measurements, console messages, subsystem statistics, application logs and performance metrics provide an enormous amount of detail about what is happening across the system.

The difficulty is not necessarily collecting the information. It is analysing it and identifying what really matters.

Consider a CICS region where transaction response times have started to increase.

CPU utilisation appears normal. Storage consumption is within acceptable limits. No obvious errors have occurred, and none of the traditional monitoring thresholds has been exceeded.

An experienced systems programmer might eventually identify the problem by examining historical performance information and looking for changes in workload behaviour.

But what if a machine learning model could identify those changes automatically?

Instead of simply checking whether a particular metric has exceeded a predefined threshold, an anomaly detection system can compare current behaviour against historical patterns.

For example, it might identify that response times are increasing relative to transaction volume, even though both measurements remain within their configured limits.

This could give the operations team an earlier indication that something deserves investigation.

There are already commercial solutions using this approach.

IBM provides AI-driven operational analytics through IBM Z OMEGAMON AI Insights.

Similarly, BMC AMI Ops Insight uses machine learning and multivariate analysis to identify anomalous system behaviour by comparing current operational measurements with learned patterns.

Although their implementations and supported monitoring capabilities differ, both illustrate how AI can complement established performance monitoring.

BMC has also been extending these capabilities through BMC AMI Assistant, combining machine learning, domain-specific knowledge and generative AI to help engineers understand probable causes and investigate operational problems.

Of course, detecting unusual behaviour isn’t the same as identifying the cause of a problem.

An unexpected increase in transaction volume could indicate an application issue, but it could just as easily result from legitimate business activity.

Even when a system identifies a probable cause, the finding still needs to be validated against the available evidence.

This is why I see AI as a useful addition to traditional monitoring and performance management, rather than a replacement for them.

It can help identify patterns that might otherwise go unnoticed, allowing engineers to concentrate their efforts where they are most needed.

And in an environment where identifying a problem early can make the difference between a minor performance issue and a major production incident, that is certainly worth exploring.

Mainframe Security: Looking Beyond Access Controls

Security is another area where AI has considerable potential.

On z/OS, security managers such as RACF, ACF2 and Top Secret, working through SAF, provide mechanisms for controlling access to protected resources.

These controls are fundamental to mainframe security and aren’t going anywhere.

However, there is an important distinction between an action being authorised and that action being legitimate.

Imagine a privileged user ID with permission to access sensitive datasets.

Under normal circumstances, that identity accesses a relatively small number of datasets as part of its regular activities.

Now imagine that the same identity suddenly begins accessing hundreds of unrelated datasets, at unusual times and in a pattern that differs considerably from its established behaviour.

The security manager may authorise every access because the identity has the necessary permissions.

From an access-control perspective, nothing has necessarily gone wrong.

From a security-monitoring perspective, however, the activity might deserve immediate investigation.

This is where behavioural analytics can be useful.

By analysing security-relevant events and identifying deviations from established patterns, machine learning can help security teams recognise potentially suspicious activity that traditional access-control mechanisms weren’t designed to detect.

IBM has introduced capabilities in this area through IBM zSecure Detection, including AI-driven access anomaly detection.

This doesn’t mean AI can determine whether every unusual activity is malicious.

There will be false positives. Legitimate operational activities can produce unexpected patterns, and attackers may attempt to avoid detection.

The quality of the results will also depend on the information available to the detection system, how the models are configured and how effectively the findings are investigated.

It is also important to distinguish between AI-based behavioural analytics and traditional security monitoring. Rule-based detection, event correlation and automated response remain valuable, but they aren’t necessarily applications of AI.

Nevertheless, I believe this represents an important development in mainframe security.

We have always needed to understand who can access our systems and resources.

Increasingly, we also need to understand whether the activities performed by those identities are consistent with what we would normally expect.

Both perspectives are necessary.

And as mainframe environments become more interconnected with distributed systems, cloud services and enterprise security platforms, that additional visibility becomes even more valuable.

AI-Assisted Systems Programming

This is probably one of the applications that interests me most from a technical perspective.

Mainframe systems programming requires a considerable amount of knowledge and experience.

A systems programmer investigating a production problem might need to examine system messages, review dumps, analyse configuration parameters, check maintenance levels and consult several IBM manuals.

Quite often, the difficult part isn’t finding a particular message description or command syntax.

It is understanding how different components interact and determining which information is relevant to the problem being investigated.

Generative AI can potentially make this process more efficient.

Imagine an assistant that can help explain a system message, retrieve the relevant technical documentation and suggest possible areas for investigation.

An engineer could ask about a particular abend or subsystem message and receive an explanation supported by the appropriate IBM documentation.

More sophisticated implementations could incorporate approved internal procedures, configuration information and operational records.

One approach is Retrieval-Augmented Generation, commonly known as RAG.

Rather than relying entirely on the information learned during model training, the AI system retrieves relevant material from an approved knowledge base and uses it to support its response.

That knowledge base might contain IBM manuals, internal operating procedures, technical standards and troubleshooting documentation.

IBM has been developing capabilities in this area through watsonx Assistant for Z, including support for generative and agentic AI workflows.

BMC provides related capabilities through BMC AMI Assistant, which brings conversational and contextual AI into supported mainframe development and operational workflows.

The interesting part isn’t simply having a chatbot that can answer technical questions. It is the possibility of combining natural-language interaction with relevant operational information and specialist mainframe knowledge.

This could be particularly useful for engineers who are less familiar with a particular subsystem or are investigating a problem they haven’t encountered before.

But there is a problem we cannot ignore.

AI models can generate technically incorrect information, sometimes with remarkable confidence.

Anyone who has worked with complex z/OS configurations knows how dangerous an incorrect recommendation can be.

A parameter that is perfectly valid in one environment might be inappropriate in another. A command that resolves one problem could create a considerably more serious issue if executed under different circumstances.

For that reason, I would expect an AI assistant used in systems programming to provide references to the documentation supporting its recommendations and to make clear when an explanation is uncertain.

There is also a considerable difference between an assistant that recommends an action and an AI agent that is authorised to execute it.

I would be very cautious about allowing an AI system to make production changes without appropriate controls and approval mechanisms.

AI can help engineers find information, understand problems and investigate possible solutions.

But the responsibility for validating those solutions remains with the people operating the systems.

Application Development and Modernisation

Let’s consider another familiar situation.

An organisation has a business-critical COBOL application that has been running for decades.

Over the years, the application has been modified to accommodate new business requirements, regulatory changes and additional functionality.

Different teams have maintained different parts of the application. Some of the original developers have retired, and the documentation may no longer reflect everything the application does.

The system continues to perform its business function reliably.

But understanding the application has become increasingly difficult.

Now imagine that the organisation needs to introduce a new requirement.

Before making any changes, developers need to understand the relevant programs, their dependencies, the data they process and the potential impact on other applications.

AI-assisted development tools can help with some of this work.

IBM watsonx Code Assistant for Z provides capabilities for application understanding, code explanation and generation.

BMC offers related capabilities through BMC AMI DevX Code Insights, with generative AI code explanations through BMC AMI Assistant.

BMC AMI DevX Code Insights also provides application analysis capabilities, including program structure, logic flow, data flow and dependency information. These are useful in their own right and should not all be confused with generative AI functionality.

Depending on the supported functionality and release, these tools can assist with code explanation, documentation, application analysis and development activities.

This can be particularly valuable when dealing with applications that have evolved over many years.

However, explaining a program isn’t necessarily the same as understanding the complete application.

A COBOL program might depend on copybooks, Db2 tables, CICS resources, IMS databases, external interfaces and runtime configuration.

Analysing the source code in isolation may not provide enough information to understand the full business process.

There is also a difference between explaining what a program does and understanding why it was designed to behave that way.

A particular calculation might exist because of a regulatory requirement introduced many years ago. An unusual condition might handle a business exception that occurs only a few times a year.

An AI model might explain the code correctly without understanding the reason behind those decisions.

This is why generated explanations and proposed changes need to be validated against business requirements, actual application behaviour and appropriate testing.

Nevertheless, I see considerable value in using AI to help organisations better understand the applications they already have.

Modernisation doesn’t always mean replacing existing applications.

Sometimes it means improving their maintainability, extending their capabilities or making them easier to integrate with other technologies.

And understanding what we already have is a good place to start.

AI-Assisted Database Administration

Database administration is another area where I believe AI can make a practical contribution, particularly in environments running large Db2 or IMS workloads.

Anyone who has worked with Db2 for z/OS knows that maintaining good performance involves more than simply checking whether the database is available.

SQL performance, access paths, locking, data organisation, statistics and utility execution can all influence how applications behave.

Consider a Db2 tablespace where application response times have gradually deteriorated.

The database administrator might need to examine access paths, SQL workload characteristics, statistics, reorganisation indicators and recent changes before deciding what action is appropriate.

Some of these activities can be supported by established monitoring and tuning tools. Others involve analysing several sources of information and applying considerable specialist knowledge.

AI can potentially help by bringing that information together, explaining unusual behaviour and suggesting areas worth investigating.

In October 2026, BMC announced BMC AMI Agentics for Data, introducing agentic capabilities intended to assist with Db2 and IMS administration.

The announced functionality includes an AI Reorg Advisor for investigating Db2 reorganisation indicators, SQL workload analysis capabilities and assistance with selected database performance and exception scenarios.

These are interesting applications because database maintenance decisions often depend on understanding several related factors rather than a single measurement.

However, an AI recommendation to reorganise a database shouldn’t automatically result in a REORG being scheduled.

The decision still needs to consider the expected benefit, available maintenance windows, utility requirements, resource consumption and potential impact on applications.

The same applies to SQL tuning recommendations.

A suggested change to an access path or SQL statement needs to be evaluated against actual workload behaviour, not simply accepted because an AI system has identified it as an improvement.

The October 2026 BMC announcement describes the direction and scope of these capabilities. Organisations should confirm the availability, prerequisites and supported functions of the particular releases they intend to use.

I see this as a useful example of AI supporting specialist technical decisions, rather than attempting to automate every aspect of database administration.

Generative AI and Enterprise Data

Generative AI introduces another set of possibilities.

Consider a financial institution that wants to provide employees with an AI assistant capable of answering questions about customer accounts, explaining financial products or helping investigate transactions.

Much of the information required might reside in Db2 for z/OS, IMS databases, VSAM datasets or applications accessed through CICS.

One approach would be to retrieve the relevant information and send it to an AI service running on another platform.

Depending on the organisation’s requirements, that might be entirely appropriate.

But it raises questions about data movement, confidentiality, latency, connectivity and regulatory obligations.

Another option is to run supported generative AI inference within the IBM Z environment.

With IBM z17 and the IBM Spyre Accelerator, organisations have additional options for deploying AI workloads closer to their enterprise applications and data.

IBM has also introduced on-platform generative AI capabilities through watsonx Assistant for Z, using supported Granite models and Spyre acceleration.

This doesn’t mean that a language model automatically gains access to all the information stored on the mainframe.

Applications still need appropriate integration mechanisms, and access to enterprise information must remain subject to established security controls.

Nor does running a model within the mainframe environment automatically guarantee that sensitive information is protected.

Model inputs, generated responses, access permissions, logging and administrative interfaces all require consideration.

The choice of model and deployment location also affects how information is handled. Organisations need to understand whether prompts and retrieved data leave their environment, where inference takes place, and what information is retained by the supporting services.

There is also the question of whether generative AI is actually the right solution.

If an application needs to retrieve a specific account balance, a conventional database query is likely to be more appropriate than asking a language model to generate the answer.

But if the requirement involves interpreting complex information, summarising documents or providing a natural-language interface to enterprise knowledge, generative AI may offer real benefits.

The important thing is choosing the right technology for the problem, rather than introducing AI simply because it is available.

Predictive Analytics for Batch Processing

Despite the attention given to real-time transactions, batch processing remains essential to many organisations.

Financial reconciliation, settlement, billing, reporting and numerous other activities depend on scheduled processing.

And anyone who has supported a complex batch environment knows how disruptive a failed overnight processing cycle can be.

Now consider what predictive analytics might contribute.

Batch scheduling and operational systems already contain information about job execution times, dependencies, completion codes, resource consumption and workload patterns.

A machine learning model could analyse this information and estimate the likelihood that a particular job or batch stream will exceed its expected completion window.

Imagine a job that normally completes in twenty minutes.

Over several weeks, its execution time gradually increases.

Initially, the change might appear insignificant.

But if the job is on the critical path of an overnight processing cycle, the accumulated delay could affect downstream processing and ultimately cause the batch window to be missed.

A predictive system might identify the trend and provide an early warning.

Of course, historical execution times alone may not be sufficient to make reliable predictions.

The model might also need information about input volumes, workload priorities, resource availability and exceptional processing periods.

It would need to distinguish between a legitimate month-end increase in processing time and an unexpected deterioration in performance.

And, as with any predictive solution, its effectiveness would need to be measured against existing scheduling and monitoring techniques.

Enterprise workload automation products such as BMC Control-M already provide orchestration across mainframe and distributed environments, including dependencies between applications and processing workflows.

BMC has also been extending Control-M with AI-related capabilities, including natural-language troubleshooting and support for coordinating application, data and AI workflows.

That is relevant to the broader discussion, although orchestrating an AI workflow or explaining a job failure is not the same as using a predictive model to forecast a batch-window breach.

The predictive example described here would still require appropriate analytics, operational data and validation.

I think this is a good example of how AI could help improve an area of mainframe operations that doesn’t always receive the same attention as newer technologies.

AI Agents and Operational Automation

AI agents are receiving considerable attention, and I expect we will see more of them being introduced into enterprise IT environments.

Unlike a conventional assistant that primarily generates answers, an AI agent can be designed to interact with tools, retrieve information and perform actions as part of a workflow.

In a mainframe environment, this could include collecting diagnostic information, investigating job failures or helping operations teams follow established procedures.

Imagine an agent that receives an alert about a failed batch job.

It retrieves the relevant output, identifies the messages associated with the failure, consults approved troubleshooting documentation and prepares a summary for the operations team.

That could be useful.

Now imagine allowing the same agent to resubmit jobs, modify system parameters or execute privileged commands without human approval.

That is a very different proposition.

This is an area where mainframe software vendors are beginning to introduce more specialised capabilities.

IBM’s watsonx Assistant for Z provides capabilities for AI-assisted operational workflows and agents, with supported on-platform generative AI inference using Spyre.

In October 2026, BMC announced BMC AMI Agentics, introducing purpose-built agentic capabilities for mainframe operations and data management.

For operations, BMC described capabilities for investigating CPU utilisation problems, explaining REXX automation and identifying probable causes associated with operational alerts.

For database administration, the announced capabilities include assistance with Db2 and IMS investigations and optimisation decisions.

The precise functionality, integration requirements and availability depend on the individual offering and release.

BMC also announced a beta integration using a Model Context Protocol (MCP) server for BMC AMI DevX Code Pipeline, allowing supported AI coding assistants to interact with mainframe development workflows under established permissions and audit controls.

That is an interesting development, but beta functionality shouldn’t be confused with a generally available production capability.

What makes these technologies interesting is the possibility of moving from an AI assistant that explains information to one that can coordinate parts of an investigation or development workflow.

However, AI agents introduce risks that need to be understood, particularly when they have access to production systems.

An incorrect interpretation, an unexpected tool response or malicious instructions embedded in retrieved information could result in actions that were never intended.

For that reason, any implementation needs clearly defined permissions, restricted interfaces, appropriate approval mechanisms and comprehensive audit trails.

An AI agent should not be able to bypass established security or change-management procedures simply because it has generated what appears to be a reasonable justification for an action.

I believe there is considerable potential for AI-assisted automation in mainframe environments.

But there is no reason to give an AI system more authority than it needs to perform its intended function.

We have spent decades developing controls to protect these systems. Introducing AI shouldn’t become an excuse to weaken them.

Making AI Work in Practice

Looking at these examples, it is easy to see why organisations are interested in introducing AI into their mainframe environments.

But implementing AI successfully involves considerably more than installing a product or deploying a model.

The organisation needs to understand the problem it is trying to solve, the information required, the integration with existing applications and the operational implications.

It also needs to determine how success will be measured.

For fraud detection, this might involve detection rates, false positives and transaction response times.

For operational monitoring, it might involve earlier identification of problems or reduced investigation time.

For application development, it could involve improvements in the time required to understand, document and safely modify existing applications.

And there is another question that I consider particularly important.

What happens when the AI gets it wrong?

AI models aren’t infallible.

They can produce inaccurate predictions, misunderstand technical information or generate incorrect recommendations.

In a business-critical environment, those failures need to be anticipated and handled appropriately.

This means having suitable fallback procedures, validation mechanisms, monitoring and clearly defined responsibilities.

It also means recognising that some problems are better solved using traditional programming, established business rules or conventional automation.

Not everything needs AI.

There is also the question of model governance.

A model that performs well today may become less accurate as workloads, application behaviour or business conditions change. Organisations need to understand how models are versioned, validated, monitored and updated.

For generative AI, this includes controlling which information can be retrieved, which tools can be invoked and how generated responses are evaluated.

For predictive models, it includes monitoring model performance, false positives, false negatives and changes in the underlying data.

These are operational responsibilities, not simply tasks for a data science team.

My preference would be to start with a well-defined use case, establish how its benefits will be measured and introduce the technology in a controlled manner.

An anomaly detection solution could initially operate without initiating corrective actions.

A fraud detection model could run alongside existing systems, producing predictions that are evaluated without affecting live transaction decisions.

An AI assistant could initially be restricted to retrieving and explaining technical information rather than executing commands.

These approaches allow organisations to understand how the technology behaves before introducing it into more critical processes.

And they provide something that no marketing presentation can deliver: evidence that the solution actually works.

Final Thoughts

Having spent much of my career working with mainframe technology, I have seen the platform evolve considerably.

New processors, operating system capabilities, middleware, development tools and security technologies have continuously expanded what organisations can do with their mainframes.

AI is another development in that evolution.

What interests me isn’t simply the fact that we can now run more AI workloads on IBM Z.

It is what we can achieve by combining those capabilities with the applications, data and operational knowledge that already exist in mainframe environments.

There are genuine opportunities to improve fraud detection, identify performance anomalies, strengthen security monitoring, support systems programmers, improve database administration and make complex applications easier to understand and maintain.

The combination of IBM’s hardware and AI infrastructure with software capabilities from IBM, BMC and other vendors gives organisations different ways to explore these opportunities.

Some of these capabilities are already available through commercial products. Others are emerging, while some will require organisations to develop and validate solutions suited to their own environments.

Either way, I don’t believe the objective should be to introduce AI into every mainframe application or operational process.

The objective should be to use AI where it makes a measurable difference, without compromising the reliability, security and predictability that organisations expect from the platform.

And we shouldn’t underestimate the importance of the people who understand these systems.

AI may help engineers find information faster, identify patterns more effectively and automate certain activities. But that doesn’t eliminate the need for technical expertise, sound judgement and a proper understanding of the environment.

If anything, those qualities become even more important when we introduce technologies whose behaviour isn’t always entirely predictable.

The question isn’t whether AI has a place on the mainframe. I think it clearly does.

The more interesting question is where it genuinely improves what we already do, and whether the benefits justify the additional complexity.

That is where I believe our attention should be.

References and Further Reading

The following technical resources provide further information about the technologies and capabilities discussed in this article.

IBM

BMC

Disclaimer: The views and opinions expressed in this article are my own and do not necessarily reflect those of my employer.