A new method for evaluating apps’ behavior change potential

In the worlds of digital health, sustainability, and insurance, true success isn't about how often users engage with a product — it's about whether the product delivers real, lasting behavior change. Yet, many companies still rely on engagement metrics as their core measure of success, mistaking activity for achievement. As pointed out by a recent WSJ article, even Behavioral Scientists can fall into the trap of prioritizing short-term engagement with long-term change.

While engagement metrics might be easier to measure, they're fundamentally limited. So, how can we better assess an app's potential to drive long-term product adoption and behavior change?

Introducing the Behavior Change Score (BCS)

The Behavior Change Score (BCS) is a new method designed by Nuance Behavior to evaluate an app or digital intervention's behavior change potential. By integrating both the insights of academic studies and theory with practical knowledge from years of hands-on experience, the BCS provides a comprehensive and systematic measure of an app's effectiveness in achieving behavior change targets. This isn't just about usage metrics — it's about real-world impact.

A New Method

Our goal with the BCS is to offer a systematic and comprehensive evaluation of digital behavior change interventions and apps, integrating both theoretical insights with practical evaluations. The BCS is typically calculated over an extended evaluation period by two experts in behavior change who assess an app using a 100+ item scale based on the fundamental behavior building blocks. They evaluate how the app does in areas such as motivation, reward, and self-determination, along with critical aspects of user experience, such as interface design, onboarding, and prompting. The score is further calibrated based on available data and user metrics.

This approach can help product teams with three issues they often struggle with:

1. Evaluate. Assessing a product’s real-world behavior change potential (beyond mere engagement)

2. Benchmark. Compare relative behavior change strength using industry or competitor benchmarking

3. Target. Identifying key areas for improvement, ensuring the right features and product changes are targeted to solve for the right underlying bottlenecks

When an app aims to change behavior, we believe that Behavioral Science must be integrated into the very foundations of the app. It cannot consist of just a few nudges sprinkled on top of a ‘Little e’ sundae. Instead, digital products need to execute the principles of behavior change in a way that is coherent and in line with science. By focusing on foundational principles rather than “nudges” or “biases”, which are often too simplified and too little generalizable, BCS is able to give a good account of where apps are excelling, and where they may need improvement in order to promote lasting behavior change outcomes.

100+ Items, 14 Mechanisms, 1 Journey

Our goal with BCS is to offer a systematic yet adaptable methodology that makes it easier for product teams to capture the important details necessary for effective behavior change. To allow for that, we have chosen to focus on 14 Behavioral Science mechanisms as opposed to focusing on individual nudges which may or may not generalize to the unique context.

These 14 mechanisms, grounded in rigorous research, allow us to evaluate an app's performance across different dimensions and stages of the behavioral journey. No app needs to excel in all areas, and some apps may choose to focus more on some mechanisms over others. For example, a medical adherence app may focus on developing a personal medication habit, whereas an app that facilitates physical exercise focuses on leveraging social factors. However, any app that scores low on each of the dimensions will be very unlikely to change behavior.

The BCS not only assesses how well an app scores on each mechanism but since various sub-scales correlate to stages in the behavioral journey, we can also get a glimpse at how well apps do at each stage in the behavioral journey. Shown below, the journey progresses from onboarding, to activation, to engagement, to retention.

In the long term, we also hope this approach can better help end-users compare apps on what really matters – their behavior change potential, not just their app store ratings.

Learn more

For inquiries about using the Behavior Change Score for your project or to learn more, book a call here or reach out to us at hello@nuancebehavior.com.

We're also excited to share our recently published Behavior Chance Score report – download it here to ensure you don't miss out on our findings.