Health, Wellbeing, and Consumer Protection in the Era of AI

Technology products and software have become deeply embedded in daily life, especially in high-income countries where smartphone access is nearly universal. With smartphones come a wide array of apps, including social media platforms and ‘health apps’. Unsurprisingly, the market for smartphone applications is booming. Many of these apps integrate artificial intelligence, offering features like “AI health insights” and other automated functionalities.

Within this landscape, a specific group of health apps, termed medical apps, may qualify as medical devices and thus fall under the scope of certain regulations, e.g. the Medical Devices Regulation in the EU. However, many other apps that can influence users’ health, particularly those labelled as wellness apps, often operate outside this regulatory framework. These types of apps can be downloaded from the app store by any smartphone user and general consumer law applies. However, there are concerns that the current regulatory framework is insufficient, particularly since such apps may be used by vulnerable groups, including young people, migrants, and others.

Below are a few examples of some of these apps:

Apps for mental health

The market for mental health apps has grown significantly in the past years, particularly since the COVID-19 pandemic. Aimed at ‘promoting wellbeing’, these apps often include chatbot-like solutions to provide the user experiences which may feel like talking to a therapist.

However, despite benefits including facilitating easier access to mental health support, risks from the use of such apps are numerous and include receiving misleading information or even possible misdiagnoses, which can directly harm mental health and delay access to appropriate mental healthcare. Questions raised include those relating to trust, data privacy, and more.

See our blog and video on AI & mental health and policy brief for more.

Apps associated with wearables

With the advent of wearable technology, such as fitness trackers, there is an increasing number of technologies which promote healthy living and/or better health outcomes. These technologies or apps use data gathered from wearables, such as heart monitors, smartwatches, and others.

These apps may well present themselves as medical. Some offer possibilities to track glucose, and advertise themselves as assisting with diabetes management. However, further information on the app shows that they are not intended for medical use (see e.g. GluHealth).

Existing wearable devices may themselves also promote applications that appear to be for medical purposes. The wearable device Whoop offers users the possibility to get estimates on their blood pressure in order to better understand how blood pressure affects “performance and wellbeing”. Its information page also contains a disclaimer stating that “not a medical device and cannot diagnose or manage medical conditions”. The company received warnings from the FDA that its product was being promoted as an unregulated medical device.

In these cases, such apps promote their technologies as wellness-related, rather than medical. However, by promoting their apps for users living with diabetes, they put people dependent on adequate blood glucose monitoring at risk of severe health impacts if the app is erroneous. This should not be the responsibility of the user alone.

Period tracking apps

Period tracking apps, also known as cycle tracking apps, are apps that help users track their menstrual cycle. Like wearables, period tracking apps collect data on users’ reproductive cycles.

The health implications of such apps are obvious, particularly if users rely on them for contraception or make use of them for fertility purposes. However, there are serious concerns as to the privacy of the sensitive health data contained within them. Although EU apps need to follow regulations on data protection, research has found that many of these apps are not in compliance. Future regulations on consumer protection must take into account the specific nature and health risks of period tracking apps.

Read our blog on period tracking apps for more.

A need for better consumer protection

The above demonstrates that clear regulatory gaps remain when it comes to the oversight of wellness and health-related apps, and more must be done to protect consumers from the associated risks.

Health Action International is currently involved in processes relating to the upcoming Digital Fairness Act at the EU, including submitting input to the Consumer Agenda 2030. The health of citizens must remain at the core of all EU discussions, even on dossiers and files beyond health.