An AI journaling study from MIT Media Lab researchers can be summarized as proof that AI journaling lowers depression scores, but the paper itself describes something narrower, and reading it closely makes you a better judge of every similar claim you’ll meet in an app ad.
That claim travels well because it sounds exact: a randomized trial, 55 people, a standard questionnaire. Those details are real. But “AI journaling helps” and “this specific research prototype changed this specific score for these specific volunteers over two weeks” are different sentences, and the gap between them is where hype can live.
This guide walks through what the study did, what it measured, what the numbers say, what the authors themselves list as limits, and how to use the same reading habits on any health-flavored claim in an app’s marketing. It also includes a prompt that pulls the key facts out of a paper without adding opinions. This is general information about reading research, not medical advice, and nothing here says whether any journaling app will help you.
Contents
What Was Actually Tested
The paper is titled “Resonance: Drawing from Memories to Imagine Positive Futures through AI-Augmented Journaling,” and it’s available on arXiv. Here’s the setup as the authors describe it:
- Resonance is a research journaling tool, not a commercial app you can download and compare to others.
- When a user logs an experience, the system uses GPT-4 to suggest an activity grounded in their earlier entries, then prompts them to spend about 30 seconds imagining doing it and to describe that imagined scene.
- The suggestions are designed around three ideas: personalization using similar past entries, novelty by avoiding repeats, and positive framing.
So the AI wasn’t just reading and reflecting. It was suggesting activities and asking people to picture themselves doing them. That’s a specific design, and the results belong to it.
The Design in Plain Numbers
- People recruited. 71, through a university behavioral research lab
- People who finished. 55
- Length. Two weeks
- Groups. Treatment (using Resonance’s suggestions) and control
- Control group. Used the same journaling interface with no AI suggestions or imagination prompts (27 people)
- Treatment group. 28 people
- Ages. 18 to 57, average about 27
- Starting point. Most had mild baseline depression, per the authors
Two quick calculations are worth doing yourself. First, 55 of 71 completing means about 77% finished, so about 23% dropped out before the end. Second, 27 plus 28 equals 55, which matches. Always check that a study’s group sizes add up to its reported total.
The control group matters. Because they used the same interface without the AI features, the comparison isolates something about the suggestions and prompts rather than about journaling in general. That’s a strength of the design.
What the Depression and Mood Numbers Say
What the Depression Numbers Say
The main outcome was the PHQ-8, a questionnaire that screens for current depression symptoms. Scores on it run from 0 to 24, with higher meaning more symptoms.
For the treatment group, the paper reports an average of about 6.11 before and about 4.96 after, and the authors report that the control group showed no significant change. Here’s the arithmetic: 6.11 minus 4.96 is 1.15 points. On a 24-point scale, that’s about 4.8% of the range.
Does that sound like a lot or a little? Honestly, both readings are fair, and that’s the point. The authors call the effect significant statistically, which means it’s unlikely to be a coincidence in this sample. Whether a drop of a bit over one point matters in someone’s actual life isn’t something the number alone can say, and the starting scores were already fairly low.
The Part That Gets Left Out
The study also measured daily positive affect, a rating of how positive people felt each day. The paper reports that the treatment group’s average moved only slightly, from about 3.12 to about 3.17, while the control group’s average went from about 3.30 to about 3.44.
In other words, on that measure the control group’s numbers rose more. The authors do report that within the treatment group, people who were more likely to act on the suggestions showed a greater increase in positive affect. That’s a more careful finding than “AI journaling boosts mood,” and it’s good to see the paper say so.
When a summary mentions one outcome and skips another, it’s worth opening the paper to see what else was measured. These figures come from the paper’s results: the treatment group’s PHQ-8 average fell from 6.11 to 4.96, while the control group’s moved from 6.11 to 6.14.
What the Authors Say About Their Own Limits
The researchers list several limitations, in their words or close to it:
- Two weeks is a short time to judge effects on mental health, especially long-term.
- The improvements could partly reflect novelty effects, meaning the newness of the tool rather than lasting benefit.
- Affect was measured with a single-question scale, and they suggest more comprehensive instruments.
- The system doesn’t learn from whether users actually acted on its suggestions.
- The design couldn’t separate the effect of the imagination step from the effect of the suggestions.
- Tools that collect personal data raise privacy concerns because of the risk of data leaks.
That last one connects to something you can act on: even a study showing benefits tells you nothing about how any particular app protects your entries. Our guide to AI journaling encryption covers that side.
Reading Any App’s “Research Shows” Claim
Here’s a set of questions to ask whenever an app cites a study. The Resonance paper answers most of them, which is what makes it a good example.
| Question | Why it matters | In this study |
|---|---|---|
| Is the tool in the study the same as the product? | Results belong to the tested design | No. A research prototype |
| How many people, for how long? | Small, short studies are early evidence | 55 completers, two weeks |
| Was there a control group? | Without one, improvement may be coincidence | Yes, same interface without AI features |
| Which outcomes were measured, and which were reported? | Selective reporting misleads | Two outcomes, with different results |
| Who were the participants? | Results may not transfer to everyone | Volunteers from a university lab, mostly young adults |
| What do the authors say are the limits? | They know their study closely | Several, listed above |
If the tool in the ad isn’t the tool in the study, “backed by research” is a stretch.
Pull the Key Facts With a Prompt
You can have a chatbot extract a paper’s key facts, as long as it can’t wander off. Paste the abstract and methods sections and use:
I'm trying to understand a research paper before I rely on a claim about it. Below is text copied from the paper. Please use only this text.
[paste the abstract, methods, results, and limitations]
Fill in this table, quoting the text for each item. If the text doesn't say, write "NOT STATED."
1. What tool or intervention was tested?
2. How many people were recruited, and how many finished?
3. How long did the study last?
4. What was the control condition?
5. What outcomes were measured?
6. What numbers are reported for each outcome, for each group?
7. What limitations do the authors state?
Then show any arithmetic (for example, completion rate or point differences) step by step.
Rules:
- Use only the pasted text.
- Do not say whether the tool works or whether I should use it.
- Do not give medical advice.
- If the text reports different results for different outcomes, list all of them.The prompt above is my own wording, not an official or published template.
The prompt asks for the arithmetic step by step, so each number can be followed. And if you can, open the paper’s tables rather than relying on a summary of a summary.
A Worked Example With an Invented Ad
Suppose an invented app, Daybook, says: “Clinically proven to lift your mood in two weeks.” You find the study it links to. Running the prompt, you learn the tested tool was a different prototype, 40 people took part, there was no control group, and only a self-rated mood question was used. Completion was 40 of 50, or 80%.
Each of those facts weakens the claim. A different tool, a small sample, and no control group don’t mean the app is bad. They mean the ad says more than the study does. You’d weigh the app on its other qualities, and treat the claim as marketing. The app, study, and numbers in this example are made up.
What This Study Can and Can’t Tell You
It can tell you that, in one early test with a research prototype, people who got AI-generated suggestions and imagination prompts showed a small, statistically significant drop in PHQ-8 scores, while the control group didn’t. It suggests that the design is worth studying further.
It can’t tell you that journaling apps in general improve depression, that this design works for people unlike the participants, that effects last beyond two weeks, or that it substitutes for professional care. If you’re dealing with persistent low mood or anything that worries you, a licensed professional is the right person to talk to, and an app isn’t a replacement.
Habits Worth Keeping
- Read the abstract, then the limitations. In my view, the limitations section is among the most honest parts.
- Check whether both outcomes were reported. One good number can hide a flat one.
- Do the arithmetic. Completion rates and differences are quick to check.
- Notice who the participants were. Volunteers may differ from you.
- Treat one study as one study. Look for others before drawing conclusions.
Questions About AI Journaling Study
What did the MIT AI journaling study find? In a two-week trial of 55 people, the group using a research prototype with AI suggestions showed a significant reduction in PHQ-8 depression scores, while the control group showed no significant change.
Is the AI journaling study about a real app I can use? The tool tested, Resonance, is a research prototype described in the paper, not a commercial app.
Does the study prove AI journaling treats depression? No. It’s a small, short trial of a specific design, and the authors list limitations. It doesn’t test treatment of a condition.
Did the control group improve more on anything? According to the paper’s reported averages, the control group’s daily positive affect rose more than the treatment group’s.
What is the PHQ-8? A questionnaire that screens for current depression symptoms, scored from 0 to 24. It’s a screening tool and doesn’t diagnose.
How can I check whether an app’s research claim holds up? Compare the tool in the study with the product, then check sample size, length, control group, outcomes, and the authors’ stated limits.
Open One Study and Read It Yourself
Take any research claim from a journaling or wellness app, find the paper, and fill in the table above. An AI journaling study like this one rewards a careful read, and the habit works on every “clinically proven” line you’ll see afterward.
This article offers general information about reading research, not medical advice. It doesn’t say whether any app will help you. If you’re experiencing persistent or serious emotional difficulty, please talk with a licensed professional. The example app, study, and numbers marked as invented are fictional.
Sources:
Leave a Reply