Finance leaders are eager to step up AI adoption, but they’re prioritising tools that make it easy to verify and validate outputs, new research from Sage reveals. In doing so, finance is building a governance structure that will help to generate confidence and value across the whole business.
By David Prosser
In finance, almost right is completely wrong. Executives cannot make critical business decisions about, say, asset allocation or strategic investment based on information that may not be wholly correct. The regulatory and reputational costs of filing inaccurate reports and accounts are potentially disastrous. And working with misleading financial data may undermine relationships with partners ranging from customers to suppliers.
New research from Sage pinpoints these concerns as a major challenge for finance leaders as they look to exploit advances in artificial intelligence. While AI has the potential to be a transformative technology, finance leaders are simply not prepared to take its outputs on trust. Rather, they want to understand how AI tools reach their conclusions – and then to verify the data for themselves.
The research is based on a survey of almost 2,300 senior financial decision-makers in organisations worldwide, conducted by the market analysis firm IDC. It found 71% of finance leaders would reject an AI tool with 99% accuracy if they could not explain its answers. More than half would pay more for an AI tool with transparency and traceability so that they could reach those explanations.
The findings underline the difficulties that finance leaders face when asked to work with so-called “black box” AI tools, where the algorithms used to generate outputs are not visible to users. Such tools leave finance leaders completely in the dark about how the tool has reached a specific conclusion – and therefore uncomfortable about making use of the output.
The stakes are getting higher, warns Aaron Harris, chief technology officer at Sage. “As AI takes on more complex financial workflows, the cost of uncertainty is simply too high,” he argues.
“The next era of AI won't be won on raw model intelligence alone; it will be won on trust infrastructure. Finance teams cannot afford to spend hours playing detective with black box AI outputs. They need solutions that bring transparency, control and traceability into the systems behind its outputs, so they can execute with absolute confidence.”
That’s going to be especially important for SMBs, where research suggests there is particular concern about these issues – and where there may be only limited capacity to develop AI accountability. Indeed, research just published by the Federation of Small Businesses reveals that in the UK, 92% of small firms are now concerned about the risks inherent to AI, up from 73% in 2023.
“There is a healthy realism among small businesses when it comes to AI,” says Tina McKenzie, policy chair at the FSB. “Business owners can see the potential, but they are also asking sensible questions about how their data is used, who is responsible when things go wrong, and how they can adopt the technology safely.”
In which case, Sage’s new research offers a way forward. Finance leaders have begun to build a “trust architecture” that will serve as the foundations for responsible AI adoption and usage across the whole business.
Many finance teams are already making considerable investments of time into verification. IDC’s research shows finance professionals now spend nearly 13 hours every week reconstructing, validating and defending AI outputs. Almost half of those taking part in the research spend more than 15 hours a week on verification; almost one in five spend more than 30 hours.
What does this mean for finance leaders, including those SMBs, for whom AI can be a powerful way to augment resources and drive greater financial sophistication? The danger, argues Kevin Permenter, IDC’s research director, is that finance leaders are deterred from using AI tools by issues around transparency and verification. Instead, he suggests, there is an opportunity for finance to develop a new approach to AI that emphasises trust – to develop a template that the whole organisation can make use of as it embraces innovation.
“The organisations that will achieve the most durable AI advantage are those that reframe trust infrastructure not as a constraint on AI deployment, but as the foundation on which scalable AI is built,” says Parmenter. “Organisations have a choice: act early to operationalise trust or risk becoming overwhelmed by verification overhead.”
In practice, this first means selecting “glass box” AI tools, where the process of reaching an output or recommendation is transparent; this makes it possible for the organisation to audit that process – to decide for itself whether the tool works in a way that is appropriate for the task at hand.
In addition, finance teams will want to continue verifying and validating outputs; even where they are happy with the approach taken by the AI, they will want to go on checking for accuracy. That doesn’t mean double-checking every single data point that a tool produces – but finance leaders need to do enough work on verification to feel they can rely on the application.
To get this right, finance will need professionals with different types of skills and experiences than in the past. Many organisations are already rethinking their talent needs accordingly. In IDC’s research, finance leaders said they were prioritising risk, governance and decision judgment when recruiting; strikingly, they currently value knowledge in this area nearly twice as highly as deep technical accounting.
The good news is there is time to turn theory into practice. The research suggests most businesses are still at a relatively early stage in their adoption of AI in finance, with just 4% describing their finance function as largely autonomous.
It makes sense to start with AI tools where finance can review and validate outputs quickly. This will ensure finance leaders can develop governance structures that deliver accountability and confidence – to build validation processes they can apply consistently as AI usage expands, particularly as finance starts to use tools for higher-value work.
In time, finance teams that work in this way will develop a governance model that the rest of the business can adopt as other functions embrace AI too. Effectively, finance is now building the roadmap for the whole organisation as it seeks to adopt AI responsibly and practically – and to deliver genuine value.
