# Trust in AI Is a Question of Governance **By:** FRANCESCA ROSSI **Published:** 2026-08-11T13:03:40+00:00 **Source:** [Substack](https://francescarossiai.substack.com/p/trust-in-ai-is-a-question-of-governance) --- ### Trustworthiness is built into AI systems. Trust is earned through four layers of governance. We talk a great deal about how to make AI trustworthy and far less about how to make the ecosystem around AI trustworthy: the organizations that build it, govern it, adopt it, and use it. However, it is that ecosystem, much more than the technology alone, that determines whether AI can actually be trusted and therefore adopted in a beneficial way. When we ask whether we can trust AI, we usually assume the answer lies in the properties of the technology itself: how accurate it is, how robust, how fair, and how transparent. Those properties matter, and they are necessary, but they are not sufficient. Trust arises from the relationship between AI systems and everything around them: the organizations that deploy them, the people who use them, and the institutional and regulatory context in which they operate. We can design systems that are more and more trustworthy, but what turns technical trustworthiness into earned trust is governance: trust in AI is what emerges when several layers of governance come into alignment, each creating some of the conditions under which AI can be developed, adopted, and used responsibly. The first layer is the governance built into the AI systems themselves. These are the mechanisms of control, oversight, and verification designed inside the systems so that they operate within the limits and toward the goals set by their developers and users. For many years this discussion centered on data quality, bias mitigation, transparency, privacy, and security. Those remain essential, but the move toward increasingly autonomous and agentic systems requires a further step. AI agents no longer merely generate content: they plan actions, use tools, interact with other systems, and take initiative. This calls for systems that carry governance inside themselves: the ability to monitor their own behavior, to recognize situations of uncertainty, to respect the constraints their developers have set, to bring a human supervisor into the loop when needed, and to stop when they cannot act safely. Governance, in other words, is not only something organizations do around a system, but has to become a property of the system itself. The second layer is organizational governance. Adopting an AI system means taking on the responsibility of judging whether it fits your context, defining roles and accountabilities, putting control processes in place, monitoring its effects, and intervening when problems appear. This requires governance structures, leadership, skills, processes, incentives, and a clear allocation of responsibility. A third layer is often overlooked: the governance of the interaction between people and AI. More and more decisions are made with the support of AI, and the quality of those decisions depends on how people use what the systems produce. Too much trust leads us to accept wrong answers automatically, while too little trust keeps us from capturing the benefits. What we need instead is calibrated trust: the capacity to understand what AI can and cannot do, to know when to verify a result and when to rely on our own judgment. This is why AI literacy is not merely a technical skill, but an essential component of governance. The fourth layer is external governance. Laws, technical standards, certifications, independent audits, scientific research, standards bodies, and dialogue among institutions, companies, universities, and civil society all help create an environment in which trust can develop. Laws are fundamental, but on their own they are not enough and are too slow compared to the pace of the technology. This is why external governance has to be accompanied by the other three layers if principles are to become concrete and effective practice. These four layers are not independent but rather reinforce one another. Good external governance gives organizations a reason to be more responsible, mature organizations build systems that are safer and better governed, AI systems designed to be governable make for a more calibrated interaction with their users, and competent users better understand limits, risks, and opportunities, feeding the continuous improvement of the whole ecosystem. One feature runs through all four layers: governance cannot be static, but must evolve as fast as AI. New models, new capabilities, and new ways of using the technology continually reshape both the opportunities and the risks. Governance has to be able to adapt, to learn from experience, and to update itself over time. It is not a set of rules written once and for all, but a continuous process of observation, assessment, and improvement. Governing AI means designing AI systems that carry control and oversight within them, building organizations able to manage them responsibly, fostering a calibrated interaction between people and AI, and cultivating an institutional ecosystem that supports innovation. Trust in AI is the result of these four layers of governance coming into alignment. When the four layers support each other, trust is well calibrated and sustainable, and this lets innovation develop in a way that is reliable, responsible, and directed toward the common good. In your experience, which of these four layers is currently the weakest in practice? _Cover image: original drawing by the author._ Thank you for reading. If this reflection resonated with you, consider subscribing to receive future essays. — Francesca Rossi Thank you for sharing your thoughts, Francesca. Your argument that trust should be calibrated rather than simply increased made me wonder about a difficult feedback problem. If governance has to learn continuously from how AI systems behave in practice, how do we distinguish evidence that our governance is working from evidence that people and organizations are simply becoming accustomed to the system? I suspect this may already be happening in many organizations today. A widely used AI system could produce few visible failures because its risks are well governed. But couldn't the same observation emerge because users have normalized its limitations, stopped questioning its outputs, some harms remain difficult to observe, or people no longer have the expertise to judge the outputs well? At what point, then, can experience with an AI system legitimately increase our trust in it rather than merely our familiarity with it? The line about calibrated trust stayed with me: too much trust and we accept wrong answers, too little and we lose the benefit. If I had to pick the weakest layer, I would say the human one. Most people meet AI long before anyone teaches them how far to trust it. Thank you for laying out the four layers so clearly.