Simon Spichak is the cofounder of Resolvve, a low-cost virtual therapy clinic. He has no financial or commercial relationship with APA Labs.

Many around the world are turning to AI-powered chatbots for mental health support. These tools aren’t necessarily designed to be safe and effective clinicians and may leak sensitive information to third parties.

The American Psychological Association (APA) is hoping to pave the way for safe, evidence-based mental health tech with the rollout of the APA Labs Digital Badge Program and the Digital Badge Solutions Library of apps that have gone through the program. While some libraries, like MindApps.org, provide prospective users with safety and privacy information, and MindBench.ai provides an independent audit of AI systems, the Digital Badge Program also allows companies to earn certification and shore up the quality of their apps.

Based on the evaluation, which assesses criteria such as regulation and safety, data protection and privacy, and usability and accessibility, APA Labs provides a bronze, silver, or gold badge.

JMIR Publications spoke with Tanya Carlson, MBA—managing director of APA Labs—about the program, how it works, and whether it’s improving the quality of digital mental health apps. The interview was edited for concision and clarity.

How would you describe the Digital Badge Program and the Digital Badge Solutions Library?

TC: The Digital Badge Program is an evaluation program that provides insight and clarity on the critical aspects of digital mental and behavioral health tools—like evidence, ethics, safety, and privacy. The Digital Badge Solutions Library is a public, open-access resource where anyone can search and filter the products that have earned a badge through the digital badge evaluation.

Is a badge an endorsement from APA Labs?

TC: It is not an endorsement. It’s not an accreditation. It’s not a certification. It’s a signal that a product has demonstrated sufficient alignment to our proprietary criteria that are built into the evaluation.

Do companies apply for this badge, and do you provide feedback to the companies?

TC: It’s totally voluntary participation. They initially come through and do an evaluation with a team made up of APA experts and our operational partners, the Organisation for the Review of Care and Health Apps (ORCHA). They together conduct an initial evaluation and come up with a preliminary judgment. That preliminary judgment is shared with the company, along with a gap analysis, so they can see exactly how they scored and why.

If the company chooses to accept that judgment, then it goes to our internal panel for a final review. But we have had some companies say “We earned a silver, we can see from the gap analysis we need to do XYZ to get to gold. We’d like to do those things before you publish.” We allow that, because our goal is to have the best possible products. We are supportive of them taking our feedback, improving the product, and then earning that higher badge.

How was the evaluation criteria created, and who’s included on the panel to judge for certification?

TC: We have two main subject matter experts internally who lead all the expert groups that we pulled together. The expert groups consisted of 12 experts from across the intersection of psychology and technology.

What percent of companies that apply are able to earn the badge?

TC: I don’t believe we’ve had anyone ultimately not earn a badge. Companies come in maybe thinking they’re a little more shored up than they are. Those gap analyses really point out to them where the weaknesses are, and everyone to this point has addressed them to align with the criteria enough to earn a badge. We consider that a really big win.

In addition to providing the Badge Solutions Library, you also help the developers shore up what they have.

TC: Exactly. We had two primary purposes when establishing this program. The first purpose was to help people make better decisions among the literal thousands of tools that are out there, where there has been very little actionable guidance to date. The second purpose was to support innovation by providing developers with the information they need to develop products in the right way. They are just as lacking for actionable guidance.

There are a lot of really authentic, well-intentioned people trying to build better solutions without a playbook, and so we’re hopeful that the Digital Badge Program can address some of those knowledge gaps and provide some direction.

What are some of the risks that developers aren’t always aware about until that gap analysis?

TC: Because so many tools are built on open models, we have seen some developers thinking they know more about what’s under hood than they do. It’s critically important for developers to understand the risks that are inherent in how they’re applying AI, and understand what the expected mitigations ought to be.

The way that we evaluate AI with our proprietary framework really drills down on the type of risk. It’s not just a checklist. It really says “You’re using it in this way and therefore you should expect this risk and mitigate this way.” There’s been a lot of focus not just on red-teaming but on monitoring as well.

Since the digital badge launch, have you seen an increased trend of people applying to the program?

TC: I think the demand has been steady. People are curious. They ask “Who’s already been in it?” “Are my competitors there already?” “Can I be first in my category?” Launching the library has increased the pace of the conversations that we’re having.

What we’ve heard from founders is one of the advantages of the badge is to have a credible differentiation among competitors. We’re supportive of that because we want the apps that are built on the evidence, that are built and implemented ethically, to be the ones that are getting the most attention in the space.

How do you see the mental health tech space evolving in the next few years?

TC: We are watching, with some concern, the gravitation of users to especially AI technologies that are not purpose-built for mental and behavioral health. We are wanting to, number one, make sure that there are legitimate, credible, safe alternatives readily available and accessible out there. We are thinking about and coming together to try to establish some expectations around how AI needs to be handled in this context responsibly and ethically.

What is your take-home message for health care professionals, psychologists, and others in the mental health tech space?

TC: I think that we have a tremendous opportunity to leverage technology to address some of the persistent barriers to care. We’re in the position to lead on how we do that responsibly. Leading with science and leading with the expectation of ethical implementation will allow us to ensure that these tools are built in the right way.