Can AI Really Care? The Ethics of Digital Mental Health Support

Written by: Kealyn McDowell & Daisy Hu

· Mental Health Concerns and Awareness,Expert Opinions and Research Articles,Mental Health in the Digital Age

Based on an interview with Dr. Anita Ho.

Artificial intelligence (AI) is increasingly being presented as a solution to the challenges facing mental health care. From chatbots that provide conversational support to apps that track moods and behaviours, digital technologies promise greater accessibility and convenience. For students navigating academic pressures, financial constraints, and long wait times for counselling, these tools can be particularly appealing.

But can AI meaningfully support our mental health? What happens to the personal information we share with these applications? And could technologies designed to help us inadvertently leave us feeling more isolated?

These questions are central to the work of Dr. Anita Ho, a bioethicist and AI ethicist at the University of British Columbia and University of California, San Francisco. Her interdisciplinary research examines the ethical implications of emerging technologies, particularly their impact on healthcare, health equity, and marginalized communities. Her perspective offers a critical lens through which to consider the growing role of AI in mental health care, particularly how these technologies shape autonomy, power, and human wellbeing in an increasingly digital world.

Can a Chatbot Truly Care?

One of the most important distinctions in digital mental health is the difference between feeling supported and receiving genuine human connection.

AI chatbots can simulate empathy through reassuring language and seemingly attentive responses. For someone who feels uncomfortable discussing their emotions with another person, these interactions may offer a low-pressure space to explore difficult questions or articulate feelings they struggle to express.

However, Dr. Ho cautions against equating simulated empathy with genuine human care. Although a chatbot may respond as though it understands or cares about a user, it does not experience empathy or concern for the person it is interacting with.

This distinction becomes especially important for people experiencing loneliness or isolation. While an interaction with AI may feel comforting, it does not necessarily provide the reciprocal understanding and connection associated with a relationship with someone who genuinely cares.

There is also a risk that digital companionship could become a substitute for human interaction rather than a bridge toward it. If users increasingly turn to AI for connection, they may find themselves engaging more frequently with technology without developing the human relationships that could help address their loneliness.

For Dr. Ho, the concern is not simply whether AI companionship feels real, but whether it ultimately helps people build meaningful relationships outside the digital environment.

When Support Becomes a Business Model

These concerns become more complicated when considering the commercial incentives behind many digital platforms.

Many commercial AI applications are designed to encourage continued engagement. The longer users interact with a platform, the more opportunities companies may have to collect data, refine their products, and generate revenue. Yet the goal of maximizing engagement does not necessarily align with what is best for a user's mental health.

While therapy typically involves establishing goals and boundaries to support clients’ wellbeing and independence, some AI companions encourage ongoing interaction without a clear endpoint. A chatbot is always available, never complains about repeated questions, and can continually respond in ways that encourage users to return.

This constant availability may initially feel reassuring, it may also encourage dependency and contribute to unrealistic expectations about human relationships. People may become accustomed to interactions in which attention is immediate and seemingly unlimited, making the complexities of real relationships feel more difficult by comparison.

Dr. Ho sees a fundamental tension here: mental health support should help people develop the confidence and skills to manage challenges independently, yet engagement-driven applications may benefit financially from keeping users dependent on their services.

Commercial incentives also raise concerns about how users' personal information is handled. Users may disclose intimate details about their emotions, relationships, and mental health without fully understanding how their data is collected, stored, or used.

For Dr. Ho, these concerns extend beyond data privacy to the broader issue of surveillance. Apps that monitor our moods, behaviours, and daily activities may appear to offer personalized support, even as they continuously collect information about us. Whether or not users feel that they are being watched, these technologies can still function as forms of surveillance, raising questions about how much control individuals truly have over their personal information.

Convenience and accessibility, therefore, should not be mistaken for evidence that an application is safe, effective, or designed with users’ best interests in mind.

Who Gets Left Behind?

AI mental health tools are often promoted as a way to address gaps in access to professional care. For people who cannot afford therapy or face lengthy wait times, digital applications may offer a more accessible source of support.

However, Dr. Ho cautions that expanding access to technology does not automatically produce equitable care.

AI systems are shaped by the data used to develop them and the priorities of the organizations responsible for their design. When certain populations are underrepresented in these datasets, technologies may be less accurate or effective for those groups. The assumption that AI is inherently objective can therefore obscure the biases and inequities that these systems may reproduce.

Beyond biases in the data, these technologies may also introduce new forms of power imbalance. Although AI is sometimes presented as a way to reduce the biases associated with human healthcare providers, Dr. Ho emphasizes that it does not necessarily eliminate these concerns. Instead, power may shift toward the companies developing and operating these technologies. AI systems can make inferences about users' mental health based on their personal information, potentially categorizing individuals without their knowledge and leaving them little opportunity to challenge these classifications.

There is also a risk that people already facing barriers to health care will be directed toward digital tools as a substitute for professional support. Although an application may provide benefits when professional care is unavailable, this does not necessarily make it an adequate substitute.

These concerns extend beyond individual applications to the broader health care system. If institutions invest heavily in digital solutions without also improving access to qualified professionals, existing health inequities may persist.

For Dr. Ho, the question is not simply whether AI can provide support, but whether the resources devoted to these technologies could also be used to make human care more accessible and equitable.

Using AI Without Losing Our Independence

Despite these concerns, Dr. Ho recognizes that digital tools can have meaningful benefits when developed and evaluated responsibly. However, establishing these benefits requires more than promising initial results. She emphasizes the importance of evaluating their effectiveness over time, including whether users experience lasting improvements in wellbeing and whether potential harms emerge with continued use. Without rigorous research and ongoing evaluation, it can be difficult to distinguish technologies that genuinely improve mental health from those that simply encourage engagement.

Applications designed with the involvement of licensed therapists and supported by evidence of their effectiveness may help users practise therapeutic exercises between sessions. Other tools can help people better understand mental health conditions, identify appropriate resources, and prepare to communicate their concerns to a health care provider.

The key is to treat technology as a supplement to human care rather than an unquestioned replacement for it.

For students, this also means developing greater digital literacy. Before relying on an application, users should consider who developed it, what evidence supports its benefits, and how their personal information may be used.

Beyond mental health applications, Dr. Ho also encourages us to reflect on how AI shapes our everyday thinking. These tools can help us articulate ideas, organize information, and approach unfamiliar tasks with greater confidence. However, overreliance on these tools may come at the expense of developing our own independent thinking, and creativity.

As Dr. Ho suggests, one practical approach is to begin with our own ideas before turning to AI. Whether drafting an assignment, reflecting on a personal challenge, or preparing for a difficult conversation, we can first consider our own thoughts and then use technology to explore alternative perspectives.

This allows AI to support our judgement rather than replace it.

Rethinking What Helpful Technology Looks Like

AI has the potential to improve access to information and supplement existing mental health services. But its value should not be measured solely by convenience, efficiency, or the amount of time users spend interacting with it.

As Dr. Ho’s work highlights, ethical digital mental health care requires attention to privacy, equitable access, evidence of effectiveness, and the power that technology companies hold over users' information and choices.

For students navigating an increasingly digital world, the challenge is not necessarily to reject AI, but to engage with it critically. As we embrace these technologies, it is worth considering not only what they offer, but also what we may lose in relying on them.

Ultimately, AI should be designed and used in ways that prioritize human wellbeing, respect our autonomy, and strengthen rather than replace the human connections that make us feel understood.

To learn more about Dr. Anita Ho’s work and her research exploring the ethical implications of AI and emerging technologies, please visit her website.