In political philosophy, legitimacy concerns the normative justification of authority: whether an institution has a defensible right to exercise power over those subject to its decisions [25,26]. It is distinct from legality, effectiveness, ethical compliance, and sociological acceptance. Authority may be lawful and effective yet remain illegitimate if its exercise lacks a publicly defensible justification [26]; conversely, public acceptance does not itself establish legitimacy [25].

This distinction is particularly important in clinical AI because health institutions increasingly exercise authority through computational systems as well as through clinicians and formal policies. AI systems stratify risk, prioritize patients, recommend treatments, flag diagnoses, and allocate clinical attention [27,28]. In doing so, they shape the distribution of treatment opportunities, risks, responsibilities, and other socially important goods [27-29]. Institutions must therefore be able to justify to affected publics why they are entitled to exercise clinical authority through a particular system, for a particular purpose, and under particular conditions [26,28,30]. Moreover, AI design and deployment embed value choices that technical validation cannot resolve: they concern how institutional power should be exercised and therefore require public justification [26,28,31].

Current clinical AI governance may treat legitimacy as a presumed consequence of institutional approval. A device may receive regulatory clearance [32], its local deployment may be classified as nonresearch quality improvement or receive a favorable institutional determination [33], and a health system committee may approve its integration [34]. Together, these processes may establish technical, legal, and organizational permissibility [28,32-34], but they do not by themselves establish public authorization to govern through the technology [26,29].

The deficit is not an absence of governance. Patients, communities, and frontline clinicians may nonetheless have limited, late-stage, or advisory-only influence over a system’s purposes, embedded trade-offs, target populations, or conditions for modification or withdrawal [8,35,36]. Those most affected by AI-mediated authority may therefore have little meaningful opportunity to shape, contest, or reauthorize its exercise. Existing oversight mechanisms remain necessary, but they do not by themselves establish why an institution is entitled to make particular deployment-level choices on behalf of those subject to them [26,29].

This problem is best understood as a jurisdictional gap. Regulatory mandates, institutional accountability, professional judgment, and affected-public participation each provide a partial warrant for clinical AI deployment, yet responsibility for publicly justifying deployment-level decisions about purpose, threshold, population, priority, and acceptable trade-offs may remain unclear [28,34,37]. The concept of the unclaimed decision introduced above captures this gap in responsibility. Accumulating existing forms of approval does not resolve the gap: regulatory, institutional, professional, and participatory processes answer different questions and provide complementary rather than substitutable warrants [26,28,29,34].

Individual consent remains important but cannot authorize every systemic consequence of clinical AI. Organizational uses may reshape access, workload, and resource allocation beyond the consenting encounter [28,34,38,39], particularly when AI-mediated processes are embedded in routine care and difficult to avoid. We distinguish 3 complementary mechanisms: voice, meaning influence over governing purposes and conditions; contestability, meaning the ability to challenge a consequential decision and obtain meaningful reconsideration; and exit, meaning a practical opportunity to avoid the AI-mediated pathway. A patient may challenge an AI-mediated decision without being able to influence deployment policy or decline an encounter-level AI tool while remaining subject to an AI-managed referral queue.

The legitimacy deficit is not merely conceptual. Research identifies concerns about trust in institutional AI use and the inclusion of affected communities in governance [35,40]. Insufficient public justification may compound these concerns, although its effects on resistance and inequities require empirical evaluation. We propose that stronger public authorization could help identify value conflicts and distributional concerns before deployment arrangements become embedded in routine care [8,35]. Legitimacy does not guarantee acceptance or better outcomes; it provides a basis for institutions to assess whether a proposed exercise of deployment authority is justifiable despite disagreement [25-27,29].

The legitimacy question becomes more consequential as AI is embedded in clinical workflows [28,34]. As validated tools become integrated into professional practice, clinicians may face expectations to use them, making nonuse as consequential as use [28,41], while AI-mediated care may become increasingly difficult for patients to avoid. The question is therefore not only whether AI is safe, effective, or trustworthy, but also on what grounds institutions subject patients to consequential algorithmic processes [26,29,34]. Patients may not have chosen these processes and may have limited ability to contest them [27,28].

Clinical AI governance must account for 2 constituencies simultaneously. Patients may be subject to AI-mediated prioritization, recommendations, communication, and access decisions, while frontline clinicians may experience changes in professional discretion, workload, accountability, and standards of care [34,37,42,43]. Public authorization therefore cannot be reduced to patient acceptance alone [35,40]. Legitimate governance must account for both those who receive AI-mediated care and those who remain professionally answerable for delivering it, particularly because clinical authority is embedded in fiduciary and relational obligations [44]. When AI redistributes judgment, attention, and responsibility, the authority exercised through it must remain publicly justifiable, contestable, and renewable [26,29,37,42,44].

Our framework connects political theories of legitimacy and public value theory [22,25,26,29] with governance concerns discussed in the clinical AI literature. These include limited affected-party influence over deployment decisions [8,35,36], insufficient public justification of purposes and distributive trade-offs [28,30,31], and the need for mechanisms to support contestation and reauthorization as systems and uses change [45,46]. We interpret these concerns as aspects of the unclaimed decision. This normative-conceptual synthesis is not a systematic review or a consensus exercise.

On this basis, we propose 3 interdependent dimensions of public authorization: procedural authorization, substantive justification, and relational renewal (Figure 1). Each addresses a distinct question: Who can meaningfully influence deployment decisions? Which purposes and distributions of benefit and burden can be publicly justified? How does authorization remain contestable and justified over time?

Procedural authorization concerns how AI deployment decisions are made and whether those affected have meaningful influence over them. Affected constituencies, including disproportionately affected groups, should be involved before the purposes, trade-offs, and conditions of use are established [35]. Participation should involve deliberation or co-design where appropriate, with documented institutional responses showing how input influenced decisions [35]. Input should address choices that remain open to influence; product-level constraints may require engagement with developers or vendors.

In a national survey of US adults conducted in 2023, 62.7% rated as “very true” the statement that notification of AI use in their care is important [47]. Two UK citizens’ juries, each involving 18 participants over 5 days, favored accuracy over explainability in the health care scenarios considered [48]. These findings inform notification and deliberation but do not establish legitimacy or clinical benefits. Broader research identifies context-dependent expectations concerning public benefit, privacy, commercial involvement, and institutional trustworthiness [49-52]. Participation should therefore shape governing decisions rather than merely secure acceptance [35,51].

We propose a 4-step process for mapping the affected public for each deployment. First, identify whose care, access, work, or responsibilities may change. Second, trace indirect effects, including queue displacement, caregiver burdens, consequences for data-contributing communities, and unequal exposure to error. Third, identify groups at risk of underrepresentation because of language, disability, digital access, geography, or social disadvantage. Fourth, publish the proposed boundary, the reasons for inclusion and exclusion, recruitment routes, and omissions; invite corrections; and revisit the map after material changes. This engagement-informed heuristic is not a validated sampling method [35,43,53].

For a hospital-wide risk model, relevant groups may include patients whose care is intensified or deprioritized, staff receiving alerts, and subgroups with unequal error burdens. Referral triage also affects people displaced within the queue and referring services. Ambient documentation affects patients whose encounters are recorded and clinicians whose documentation changes; secondary data uses may warrant wider inclusion. The boundary should reflect distributive reach rather than current use or ease of recruitment.

Institutions should specify participants’ influence before engagement. For low-impact, readily avoidable uses, input may be advisory but should be accompanied by a reasoned response. For consequential uses, affected representatives should help set conditions, monitoring priorities, and review triggers. For high-impact, rights-sensitive, or effectively unavoidable uses, we propose formally delegated powers to require reconsideration or to pause deployment, subject to applicable law and institutional authority. These collective powers are distinct from an individual veto over a shared service.

A named accountable institutional body should resolve disagreement through documented deliberation, disclose conflicts of interest, publish reasons and dissent, and provide escalation to a body outside the deploying team. Delegated powers should be explicitly defined, while the deploying institution retains accountability. Majority support cannot waive safety obligations or protections against discrimination. Unresolved concerns about fundamental interests should prompt modification, restriction, or suspension pending review [19,28,45].

Substantive justification concerns whether the purposes and distributive consequences of AI deployment can be publicly defended. It requires explicit scrutiny of what a system is designed to optimize, which trade-offs it privileges, and whose interests it prioritizes [28,31]. A legitimate deployment should pursue a clearly stated clinical or social purpose, generate benefits proportionate to its risks and burdens, and avoid creating or exacerbating inequities [27,28,31]. Efficiency and cost reduction may be relevant benefits, but they do not by themselves justify adverse effects on autonomy, access, workforce burden, or relational care, nor do they justify commercial extraction of public health data without adequate safeguards [28,31,51].

This dimension draws on the Belmont principles of respect for persons, beneficence, and justice [54], adapting them to clinical AI deployment beyond research settings. It requires health systems to assess and publicly report how AI-related benefits, burdens, errors, and opportunity costs are distributed. Health AI assurance laboratories can provide essential evidence on validity, safety, fairness, and real-world performance [7]. However, technical assurance cannot by itself establish the public justification of a deployment’s purpose or distributive trade-offs. Technical metrics can inform but cannot resolve normative choices about tolerable errors, the time available for relational care, or the allocation of scarce attention [7,31].

For data-intensive or commercially developed systems, substantive justification should also address reciprocity: whether data-contributing communities receive proportionate benefits and whether commercial value extraction is consistent with the stated public purpose [28,51]. Where technically reasonable options remain normatively contestable, value pluralism itself becomes part of the governance problem [26,29,31].

Relational renewal concerns the continuing answerability of institutions that exercise authority through AI. Legitimacy cannot be secured once at procurement or deployment because systems, workflows, institutional incentives, and public expectations change over time [35,45,46]. Trust in clinical AI is relational rather than a fixed property of the system [40,49,55]. It should be treated as a possible consequence of legitimate governance, not as evidence of legitimacy itself [25,26,29].

Declining trust, rising complaints, repeated override patterns, or persistent subgroup concerns should therefore function as warning signals that prompt review, rather than as direct measures of legitimacy [45,46,56]. Relational renewal requires accessible mechanisms for patients and clinicians to identify AI involvement, challenge consequential outputs, and obtain human reconsideration, explanation, or remedy [28,45,56]. It also requires transparent communication, continued professional responsibility, and predefined review triggers when models, vendors, data sources, populations, purposes, workflows, or degrees of autonomy materially change [28,42,45,46]. Authorization should remain capable of renewal, restriction, suspension, or withdrawal when its original conditions no longer hold [45,46].

The 3 dimensions are jointly necessary but individually insufficient. Participation without substantive safeguards can create an appearance of legitimacy while leaving harmful deployment choices inadequately justified; technical fairness without meaningful participation can preserve paternalistic authority; and initial authorization without relational renewal can become obsolete. Public authorization therefore depends on the continuing alignment of fair participation, publicly defensible purposes, and accountable relationships.

The 5-question public authorization test (Table 1) translates the tripartite framework into 5 governance questions addressing purpose, distribution of benefits and burdens, exclusion, meaningful choice and contestability, and renewal. Table 1 links each question to minimum evidence and warning signs. These domains reflect established health AI governance priorities concerning public benefit, equity, accountability, human oversight, contestation and remedy, monitoring, and change management [3,28,45,57]. The test is a normative governance diagnostic, not a validated assessment instrument or a substitute for regulatory assessment.

It complements existing technical, ethical, legal, regulatory, and risk management assessments by examining the public justification of a specified deployment [26,28]. Its domains span procedural, substantive, and relational concerns. The test should be applied before deployment, at predefined review intervals, and after material changes in systems or deployment conditions [3,45,46,57]. Each answer should identify publicly defensible evidence, unresolved concerns, and any further justification, modification, or review needed.

The test addresses a distinct governance question: whether an institution is justified in exercising authority through a particular AI system, for a particular purpose and population, and under particular conditions [26,29]. It is noncompensatory and should not be reduced to an additive score: strength in 1 domain cannot offset an unresolved failure in another. Technical performance cannot compensate for authority that affected people cannot meaningfully challenge, nor can participation justify substantively unacceptable distributions of benefit and burden. Prior authorization cannot substitute for renewed justification when the system or its conditions materially change. An answer that reveals an authorization gap or leaves a material concern unresolved should therefore trigger further justification, modification, or review.

The test identifies where authorization remains insufficient and what evidence or governance response is needed before deployment proceeds or continues. The following recommendations translate these requirements into institutional governance mechanisms.

The tripartite framework supports a recurring, risk-proportionate governance cycle of deliberation, legitimacy impact assessment, conditional authorization, disclosure and monitoring, and renewal, restriction, or withdrawal (Figure 2). The 5-question public authorization test applies across the cycle, while the 3 minimum auditable conditions provide a procedural foundation. The cycle specifies governance functions; it does not require a new committee or a separate assessment document at every stage.

Translating the tripartite framework into practice requires mechanisms that support, document, and renew public authorization. We propose 3 minimum auditable conditions for consequential clinical AI deployments: visibility, affected-party input, and conditional reviewability. Visibility means that a deployment is discoverable by those it affects. Affected-party input must be structured and provided before the purposes and governing conditions are established. Conditional reviewability requires at least 1 predefined review trigger that can prompt formal reconsideration and be invoked by a named actor outside the deploying team. In operational terms, deployments should be locatable, claimable, and revisable.

These are minimum procedural conditions, not a comprehensive standard or proof of legitimacy. Together, they make AI-mediated authority identifiable, allow affected parties to influence choices before institutional embedding, and permit authorization to be reopened when circumstances change or its justification weakens. They are consistent with governance principles emphasizing transparency, stakeholder engagement, contestation, and redress [29,35,45].

Deliberation should address the purposes and major trade-offs of proposed deployments [28,51]. It should occur early enough to influence procurement, workflow design, conditions of use, and the scope of authorized uses [35,45], and should produce reasoned findings that receive documented institutional responses. Public deliberation should complement, not replace, technical, regulatory, ethical, and clinical review.

We propose 4 determinants of governance intensity: clinical consequence; distributive reach across access, attention, opportunity, and burden; difficulty of reversal; and the absence of genuine, informed choice. Meaningful exit requires proportionate disclosure of AI involvement, an accessible non-AI pathway without unreasonable disadvantage in cost, waiting time, quality, or access, appropriate language and disability accommodations, and an opportunity to reconsider the choice. Where AI use is invisible or effectively unavoidable, stronger affected-party influence, accessible contestation, and more frequent review are warranted. Where a non-AI alternative is infeasible, institutions should explain why and justify continued AI use under clearly specified safeguards [28,38,45].

Every health care organization using consequential AI should identify an accountable owner, make deployment visible, obtain timely affected-party input, publicly explain purposes and trade-offs, and maintain contestation and invocable review. These are minimum functions within the proposed framework; their organizational form can vary with risk and resources. They do not necessarily require dedicated panels, stand-alone assessments, bespoke registries, or in-house audit teams. Shared technical assessments can be reused, but local judgment is still needed to assess effects on workflows, identify excluded populations, and determine whether patients have a practical opportunity to opt out of AI-mediated care [9,34,45].

A community clinic or smaller behavioral health care organization could use its patient advisory body, sampled interviews, including interviews with underrepresented patients, and a clinician working group; document decisions using standardized assessment and disclosure templates; publish information on an existing service page; and designate an existing safety lead to receive concerns. Where a patient body is absent, trusted community partners can support recruitment. Regional networks or professional associations could share assessment expertise, contract templates, and independent escalation mechanisms. The deploying health care organization retains responsibility for its local decision.

Low-impact, readily reversible, and genuinely optional uses may receive abbreviated review. Consequential uses require documented affected-party input and conditional authorization; high-impact, broadly distributive, or effectively unavoidable uses warrant stronger representation and independent scrutiny, which can be obtained through shared services where necessary. To limit participation fatigue, institutions can reuse engagement where contexts are comparable and combine related reviews while preserving local input; participation should be accessible, compensated where feasible, and followed by an explanation of how input affected decisions. Periodic reassessment should continue, with additional review triggered by material changes, complaints, or emerging risks. Resource constraints should change the mechanism, not excuse failure to perform a necessary function [35,45,53].

Legitimacy impact assessment should ordinarily be integrated into, rather than duplicate, existing algorithmic, privacy, clinical safety, ethical, or fundamental rights impact assessments [3,45,57]. Beyond bias, fairness, and technical performance, evaluation should consider the meaningful involvement of affected publics, the public defensibility of the deployment’s clinical or social purpose [25,26], the distribution of benefits, burdens, and opportunity costs, and the availability of mechanisms for contestation and remedy [3,28,31,45,57].

A public summary should document major concerns, institutional responses, and how affected-party input influenced the final decision. It should include a concise public reason and value statement specifying what the system is intended to optimize, the principal value trade-offs accepted, why AI is preferred to feasible alternatives, and which uses fall outside the scope of authorization. Relevant trade-offs include whose outcomes are prioritized, which errors are considered acceptable, how efficiency is balanced against relational care, and how burdens are distributed across affected publics [31,57].

For consequential clinical AI deployments, we recommend conditional authorization with predefined review intervals rather than presumptively indefinite approval. Authorization should specify the approved purpose, population, workflow, degree of autonomy, human oversight requirements, accountable institutional owner, and conditions for continued use.

Authorization should remain valid only while its specified conditions continue to hold. Formal reassessment should occur at predefined intervals and after material changes to the system, its intended use, population, vendor, or deployment context [3,45,46,57]. For AI-enabled devices regulated by the US FDA, relevant modification procedures should be aligned with any authorized predetermined change control plan [58]. Review should permit continuation, modification, restriction, suspension, or withdrawal of authorization.

At least 1 predefined review trigger should be invocable by a named actor outside the deploying team, such as a patient or community governance body, clinical safety office, ethics committee, or independent review body. Each authorization should specify the trigger, evidence required, and permissible actions. Review should not depend solely on self-initiation by the deploying team.

Publicly accessible deployment records should be maintained for consequential clinical AI through a registry or an existing service page, with the format adapted to the health care organization’s scale and resources. These records should identify the system, vendor, intended purpose, deployment context, and accountable institutional owner, and summarize local validation, population and equity analyses, affected-party input, key findings from the legitimacy impact assessment, contestation and remedy pathways, authorization conditions, and the date or trigger for subsequent review [3,45,59,60].

These records should also report material updates and aggregated or deidentified information on incidents, complaints, corrective actions, and significant subgroup concerns [3,45,57], with proportionate safeguards for privacy, cybersecurity, and legitimate commercial confidentiality [3,57]. They should include the public reason and value statement, allowing affected publics to assess the stated purpose, accepted trade-offs, and institutional justification for deployment. Together, these records should support risk-proportionate public scrutiny, contestation, and review of AI-mediated institutional authority, without requiring a public referendum on every algorithm.

Health systems should provide accessible pathways for patients and clinicians to challenge consequential AI-mediated decisions, request meaningful human reconsideration, and obtain documented responses and corrective action [3,28]. These pathways should include independent review beyond the deploying team for unresolved concerns or systemic problems.

Accountability must also extend across organizational and contractual boundaries. Legal counsel and procurement teams, working with clinical and governance leads, should negotiate contractual provisions that make conditional reviewability operational. Contracts should specify notification requirements for material changes affecting performance or authorized use; access to version histories, audit logs, and relevant validation and subgroup evidence; investigation and remediation responsibilities and timelines; and audits by qualified independent reviewers.

Health systems should secure contractual rights and operational means to pause, restrict, or disable use, revert to a prior version where feasible, and terminate the contract and transition to an alternative service when essential authorization conditions cannot be maintained. Access should be sufficient for oversight, with proportionate confidentiality safeguards, without necessarily requiring unrestricted source code disclosure. Urgent safety or security updates should follow an expedited notification and review process. These are proposed contractual safeguards, not universally available legal rights [3,45,57].

Deployment records should distinguish locally configurable choices from vendor-controlled constraints, including optimization targets, thresholds, updates, data dependencies, and modification limits. Outsourcing can deepen the problem of the unclaimed decision when an institution remains responsible for care but lacks the information, authority, or capacity to investigate and correct failures [37,41]. Shared contract templates and pooled procurement expertise may help smaller health care organizations secure these review conditions. Where essential review conditions cannot be secured, institutions should document the gap and restrict or decline the proposed use [45,57].

Human oversight should be specified rather than presumed. Requirements for human oversight are insufficient unless institutions define the conditions for meaningful review. These include the identity of the person responsible for reviewing AI-mediated decisions, when review occurs, what information and scope for independent judgment are available, what authority reviewers have to override recommendations or reverse decisions, and how review is documented [45,61]. A nominal human checkpoint may preserve procedural compliance while leaving algorithmic influence largely unchanged [3,45].

Accountability must also be relational rather than merely documentary. Patients and clinicians should be able to identify who remains answerable for a consequential AI-assisted decision, request meaningful human reconsideration, understand the reasons for the resulting decision, and obtain correction or remedy when a decision produces avoidable harm or unjustified disadvantage [28,44,45].

The framework is normative and has not been empirically validated. Its dimensions, test, proposed governance arrangements, and process indicators require evaluation across applications, populations, and organizational settings. The examples illustrate proposals rather than demonstrated implementation effects.

No participation procedure can guarantee that all affected interests are represented or eliminate disagreement. Cultural and legal contexts, resource disparities, and institutional power may distort participation [28,35,53]. Mapping and publicly explaining representation decisions can expose these limitations but cannot resolve them mechanically. Advisory input should not be described as delegated decision authority.

Governance requirements may delay beneficial deployment, create participation fatigue, favor large organizations or established vendors, duplicate existing oversight, and widen inequalities between health care organizations [53,62]. The proportionate pathways proposed here are intended to mitigate those risks, but their costs and effectiveness remain uncertain.

Measurement and construct validation. Studies should develop measures of perceived legitimacy distinct from trust, technology acceptance, satisfaction, perceived usefulness, and willingness to use AI [25,26,49,63]. Conventional acceptance frameworks address perceived usefulness, behavioral intention, and technology use [64-66]. Measures of perceived legitimacy should assess the perceived rightfulness of authority, the opportunity for procedural voice, the adequacy of public justification, distributive fairness, contestability, and access to remedy. Validation should examine reliability, construct validity, and measurement invariance across populations and jurisdictions. Perceived legitimacy cannot, by itself, establish normative legitimacy.

Participatory governance and implementation. Comparative studies should examine whether participation changes deployment purposes, conditions, or safeguards; whether legitimacy assessments identify concerns missed by conventional review; and whether procurement provisions and review mechanisms enable corrective action [3,35,45,51,60]. Comparisons across health care organization sizes should assess abbreviated, integrated, and shared governance arrangements, including costs, staff time, participation burden, deployment delays, and opportunity costs [53,62]. Evaluation should examine whether affected-public mapping and specified participation powers change whose concerns influence decisions and whether shared resources preserve local influence and institutional responsibility.

Authorization processes and process indicators. Three candidate indicators warrant evaluation: authorization coverage, the proportion of consequential deployments with documented evidence for all 3 minimum auditable conditions; time to affected-party input, measured from formal initiation of deployment consideration to the first structured input from a materially affected group; and invocable review trigger rate, the proportion of consequential deployments with a predefined trigger invocable by a named actor outside the deploying team. The review trigger rate provides a component-specific measure that may identify weaknesses obscured by overall authorization coverage. These indicators should be assessed alongside evidence of whether decisions remained open to influence, meaningful choice, actual participation influence, and the timeliness and consequences of review. They should not be combined into a numerical legitimacy score.

Comparative and jurisdictional governance. Research should examine how regulation, professional standards, ownership, commercial incentives, financing, and Indigenous and community governance shape public authorization [3,28,45,60,67]. Indigenous data governance frameworks foreground collective benefit, authority to control, responsibility, and ethics [67]. Studies should include both patients and frontline clinicians.