How trust in financial AI is shaped by design, governance and regulation

AI is becoming crucial to credit scoring, robo-advice, trading, compliance, reporting and customer service, but the safeguards needed to prevent blind reliance, opaque accountability and systemic trust failures are still fragmented across the field.

A new study, titled “Trust in human–AI collaboration in finance: a bibliometric–systematic literature review” and published in AI & Society, reviews 114 finance-specific studies published from 2018 to 2025 and proposes a micro–meso–macro framework showing that trust in financial AI depends not only on users and algorithms, but also on corporate governance, auditability, regulation and infrastructure.

Financial AI is expanding faster than trust frameworks can keep up

The rapid expansion of AI in finance has turned trust into a major governance problem rather than a narrow technology adoption issue. In financial services, trust failures do not merely slow the use of new tools. They can weaken accountability, hide bias, obscure decision rationales, enable fraud, worsen reputational risks and create systemic vulnerabilities.

The authors highlight a clear concern: the literature on trust in human-AI collaboration in finance has grown quickly, but it remains conceptually scattered. Researchers often use terms such as trust, reliance, confidence, acceptance, assurance and adoption in overlapping ways. This makes it difficult to compare findings across credit risk models, robo-advisors, chatbots, blockchain systems, explainable AI and corporate governance studies.

To address that gap, the review defines trust as a latent evaluative belief held by a human trustor toward an AI system or AI-enabled socio-technical arrangement under uncertainty. The trustor may be a client, analyst, advisor, compliance officer, organizational decision-maker, board member or regulator. The trustee may be an AI model, interface, vendor, financial institution or the broader system in which the technology is deployed.

This definition matters because the authors separate trust from reliance. A person may rely on an AI recommendation without justified trust, or may distrust a system that is technically reliable. Trust is therefore not simply adoption. Nor is it the same as model accuracy. In finance, justified trust depends on competence, integrity, transparency, accountability, fairness, robustness, compliance and fiduciary alignment.

The review screened 430 Scopus records and retained 114 finance-specific studies. The authors then used bibliographic coupling, weighted Leiden clustering and centrality metrics to identify six major research clusters. These clusters show where the field has concentrated its attention and where integration remains weak.

The first cluster focuses on AI governance in finance, where trust is linked to accountability, transparency, responsible AI and oversight. The second centers on explainable AI, or XAI, for finance, where trust is tied to interpretability, auditability, fairness and regulatory compliance. The third examines anthropomorphism in financial AI agents, including the ways human-like design cues can shape user trust, trust repair and emotional response.

The fourth cluster focuses on user-interface design for financial human-AI interaction, especially chatbots and fintech interfaces. The fifth examines robo-advisors and financial decision-making, where trust affects advice acceptance, loyalty, overreliance and investment behavior. The sixth cluster covers infrastructural trust technologies such as blockchain, traceability systems, smart contracts and verification mechanisms, where trust is partially embedded in technological and institutional infrastructure.

The review finds that each cluster talks about trust differently. Governance and XAI studies usually frame trust as cognitive and procedural, grounded in transparency, explainability, accountability and assurance. Anthropomorphism and interface studies treat trust as partly emotional and social, shaped by warmth, competence, empathy, social presence and human-like interaction cues. Robo-advisor studies often treat trust as a behavioral mediator between design features and user decisions. Infrastructural studies frame trust through verification, traceability and decentralized accountability.

This fragmentation is not only academic. It has practical consequences. A bank deploying AI for credit scoring may focus on explainability and regulatory auditability. A fintech chatbot provider may focus on social cues and user comfort. A robo-advisor platform may focus on user confidence and retention. A regulator may focus on systemic risk, accountability and standardized benchmarks. Without a shared framework, these actors may optimize different forms of trust while missing the wider risks of miscalibrated reliance.

The study asserts that finance cannot treat trust as a feature built into an AI model alone. Trust emerges across the interaction between people, organizations, interfaces, governance systems and regulatory infrastructures. That is why the authors propose a multi-level framework that treats trust as a distributed property of the financial AI ecosystem.

Explainability, human-like design and robo-advice create both trust and risk

Explainable AI has become one of the strongest themes in financial trust research. XAI is often promoted as a way to make complex machine-learning models understandable to users, auditors and regulators. In finance, this is especially important because AI systems may influence loan approvals, insurance decisions, fraud alerts, investment recommendations and compliance judgments.

However, the study suggests that explainability is not automatically enough. Transparency can support trust only when it is linked to auditability, accountability and real oversight. A technical explanation that users cannot understand, challenge or act upon may have limited value. Similarly, an explanation that satisfies a developer may not satisfy a regulator, consumer, board member or compliance officer.

The authors find that many XAI studies emphasize tools and methods designed to reduce opacity, support fairness and strengthen confidence. But the research field still lacks standardized finance-specific benchmarks for explainability. This creates a major gap. Without shared standards, firms may claim transparency in different ways, making it hard to compare tools across credit scoring, robo-advisory, trading, insurance and reporting.

The review also highlights the double-edged nature of human-like AI design. Financial chatbots, robo-advisors and virtual agents increasingly use conversational tone, social presence, empathy cues, avatars, responsiveness and personalized interaction. These features can make users feel more comfortable and increase initial trust. They can also support service recovery after errors, especially when users feel the system is responsive or helpful.

Yet human-like design can also produce overreliance. If a chatbot appears empathetic, competent or socially aware, users may assume it has judgment, responsibility or concern that it does not actually possess. In finance, that risk is especially serious because users may be making decisions about savings, debt, insurance, investments or transfers. A warm interface can make a system feel trustworthy even when its scope, uncertainty or limits are not clearly shown.

The study describes this as a calibration problem. The goal should not be to maximize trust. The goal should be to align user trust with the system’s actual capabilities and limitations. Too little trust can prevent useful AI from being adopted. Too much trust can lead users to accept poor recommendations, ignore warnings or delegate decisions without enough scrutiny.

Robo-advisors provide a clear example. These platforms can lower the cost of financial advice, personalize recommendations and widen access to investment guidance. But trust in robo-advisors depends on more than algorithmic performance. Users respond to clarity, perceived competence, design cues, commitment, self-efficacy, cultural expectations, risk tolerance and emotional reactions. Some users may accept advice too quickly. Others may reject good advice because they distrust automation.

The same issue applies to AI-driven credit scoring and compliance. A bank may trust an AI system because it performs well statistically, but that does not guarantee fairness, explainability or contestability. A customer denied credit may need a clear basis for the decision. A compliance officer may need to understand whether a suspicious transaction alert is meaningful. A board may need evidence that model risk is being governed properly.

The review also points to methodological limits in the research base. Survey-based structural equation modeling dominates several clusters, while scenario experiments, interviews, eye-tracking, qualitative studies and technical evaluations appear less often. Much of the literature remains cross-sectional, meaning it captures trust at a single point in time rather than showing how trust develops, breaks, repairs or becomes miscalibrated through repeated use.

That is a serious limitation because trust in financial AI is dynamic. A customer may trust a robo-advisor after several good experiences, then lose confidence after a market shock. A compliance team may rely on an AI model until one high-profile failure exposes blind spots. A firm may believe its governance is strong until regulators or customers challenge a decision. The study calls for more longitudinal, experimental and real-world field research to capture these dynamics.

Another gap is cross-cultural evidence. Trust in financial AI may differ across countries because of different regulatory traditions, levels of institutional trust, financial literacy, cultural attitudes toward automation and expectations about human advice. A design that works in one market may not translate smoothly to another.

The authors also identify the need for better research on emotional, ideological and demographic factors. Trust in financial AI is not formed only through rational evaluation of accuracy. It is shaped by fear, anxiety, confidence, social influence, perceived risk, ideology and prior experience with institutions. This matters because finance is not a neutral setting. Financial decisions are personal, high-stakes and often tied to uncertainty, vulnerability and long-term consequences.

Trust in financial AI is a governance problem, not a model feature

At the micro level, trust is shaped by individual users’ perceptions, including whether they see the AI system as competent, fair, reliable and understandable. This level includes clients, investors, employees, analysts, advisors and compliance officers. It also includes the risk of overreliance or underreliance.

At the meso level, trust is shaped by organizational design and corporate governance. This includes workflow design, human-in-the-loop controls, escalation protocols, model risk oversight, audit trails, explainability artifacts, validation reports and responsibility mapping. The study argues that this level is crucial because organizations translate regulatory expectations and technical systems into daily financial practice.

At the macro level, trust depends on regulatory and infrastructural conditions. This includes audits, standards, compliance regimes, public oversight, payment systems, blockchain traceability, certification systems and broader institutional confidence. In this view, trust is not only a user feeling. It is a system-level condition shaped by the credibility of institutions and infrastructure.

The authors propose eight propositions to help future research test how trust works in financial AI. These include the idea that subjective trust does not improve decision quality unless it is calibrated to system competence; anthropomorphic design can increase initial trust but also raise overreliance risk; transparency works best when embedded in governance mechanisms; mature corporate AI governance reduces unreliable reliance behavior; institutional assurance shapes decision acceptance; standardized benchmarks are necessary for regulatory trust; trust repair requires clear responsibility attribution; and correlated adoption of similar AI systems can increase systemic trust fragility.

If many financial institutions use similar AI models, vendors or governance templates, a failure in one area could spread more widely. Similar models may generate similar errors, similar biases or similar responses to market conditions. This can create correlated behavior, amplify shocks and weaken confidence across the sector.

The paper shifts the debate from whether users trust AI to whether financial institutions can govern AI responsibly. AI systems cannot be moral agents. They cannot carry legal or ethical responsibility in the way boards, managers, firms and regulators can. Yet their outputs can shape decisions that affect customers, investors and markets. That makes governance central.

Corporate governance occupies the middle layer of the framework because it links individual trust and institutional assurance. Boards and senior managers must ensure that AI systems are not only accurate, but accountable, auditable and contestable. They must specify who is responsible when an AI-assisted decision causes harm, when human review is required, how exceptions are escalated, how models are monitored and how stakeholders can challenge outcomes.

The study warns that transparency without governance can become superficial. A model explanation may look reassuring but fail to provide real accountability. A chatbot may seem responsive but fail to reveal limits. A robo-advisor may be easy to use but fail to prevent overreliance. A blockchain system may provide traceability but still leave questions about governance, sustainability, interoperability and stakeholder responsibility.

In payment verification, individual users confirm transfer details, payment providers conduct real-time checks and institutional rulebooks support security. In financial education, individuals seek guidance, organizations deploy digital tools and public policies promote inclusion. In stock market participation, investors interpret signals, platforms use algorithmic analytics and regulators oversee transparency. In financial reporting, users rely on disclosed data, firms deploy automated systems and standards bodies enforce accountability.

These examples clearly show why trust in financial AI cannot be assigned to a single actor. It is produced through coordinated relationships among users, platforms, institutions and regulators. When one layer fails, the others may not be enough to preserve confidence.