Leaving Facebook and news knowledge: what the Allcott experiment measured

What ‘news knowledge decreased’ means

In Hunt Allcott and colleagues’ experiment, four-week Facebook deactivation before the 2018 US midterm elections was associated with reduced factual news knowledge. Researchers also found changes in other outcomes, including increased subjective well-being. Understanding this result first requires examining whom they compared and what counted as news knowledge.

A specific quiz lies behind that wording. It does not cover a person’s entire awareness, reasoning ability, or the quality of every source they use. A lower score cannot automatically become ‘people understand the world less well without Facebook’.

The design and measurement details below refer to the primary author version of The Welfare Effects of Social Media dated November 8, 2019, hosted on Stanford’s website. The main result directions also appear in the 2020 American Economic Review publication abstract. These sources support explaining the experiment; this discussion does not assume the author version matches the final journal text word for word.

Who was recruited and what was randomized

The researchers recruited 2743 participants through Facebook advertisements. These were people responding to recruitment rather than a random sample of the US population. Recruitment itself limits generalization: people willing to participate may differ from others.

In the author version, effect comparisons used participants willing to deactivate Facebook for compensation below 102 dollars. Within that group, compensation offers of 0 or 102 dollars were randomly assigned. The total recruited participant count therefore must not be presented as the size of a randomly selected representative group of Americans.

Randomization helps distinguish the experimental condition’s influence from pre-existing differences between people. Simply comparing people who independently quit Facebook with those continuing to use it could reflect interests, free time, or habits present before quitting. Random assignment of the compensation offer provides a basis for causal comparison within the study design.

Random assignment of conditions does not make recruitment representative, however. Causal inference within the studied comparison and applicability to all users are separate questions.

How news knowledge was measured

Section 2.3 of the author version describes a quiz with 15 statements. Participants could answer ‘true’, ‘false’, or ‘unsure’. ‘Unsure’ received half a point. The measure therefore depends on both the statements and the scoring rule for uncertainty.

• Ten statements concerned real news or altered news headlines. This set was used for the news knowledge measure.
• Five statements concerned specific false stories. A separate fake news knowledge measure was calculated for them.
• Separate measures do not allow one set’s result to be automatically attributed to the other. A reported decrease in factual news knowledge does not itself mean participants became more likely to believe false stories.

This measurement clarifies which answers researchers compared. But it leaves many other aspects of being informed untested: local-event knowledge, depth of understanding, ability to check a primary source, and long-term retention. These are boundaries of this quiz rather than established losses after deactivation.

The separate false-story measure should not be called a universal media-literacy test either. Answers to five selected statements do not equal an assessment of recognizing any misinformation. This review does not report its effect direction: available confirmation of reduced factual news knowledge does not replace separately checking the fake news knowledge result.

Measured outcome, result, and limitation

The published abstract reports several directions of change. Read them separately: an improvement in one outcome does not cancel a deterioration in another or create an overall ranking of Facebook’s value.

• Factual news knowledge. Result: decreased. Limitation: this is a specific study measure; it is not equivalent to all personal awareness or the separate false-story measure.
• Fake news knowledge. Result: the effect direction is not reported here. Limitation: the author version confirms a separate measure, but conclusions about its change require their own verification.
• Online and offline activities. Result: online activity decreased and offline activities increased, including watching television alone and socializing. Limitation: more offline time does not mean all that time became work, equally restorative rest, or useful interaction.
• Subjective well-being. Result: increased. Limitation: this outcome does not prove treatment of a psychological disorder or predict each individual’s state.
• Political polarization. Result: decreased. Limitation: the inference concerns the studied comparison in the US pre-election context.
• Facebook use after the experiment. Result: decreased. Limitation: this does not prove every participant permanently abandoned the platform.

These are effect directions rather than magnitudes. They cannot determine how strong a change a particular reader would experience or which outcome matters more to them.

Where the experiment’s inference ends

The authors highlight the nonrepresentative sample, intervention duration, and pre-election period. Four weeks before elections form a particular information environment. Effects in an ordinary month or after a longer period without use may differ.

The experiment concerns Facebook in 2018. It does not test the platform’s current algorithms, other social networks, or quitting every app at once. Nor does it test how well any alternative channel replaces news from the feed.

The results also do not confirm the effectiveness of Sinqly or another planning tool. The study does not justify promises of dopamine restoration, ADHD treatment, or guaranteed attention improvement after deleting an app.

An editorial example: free time while retaining a news channel

Imagine a fictional reader, Anna. She wants to open the feed less often but notices that it informs her about events she considers important. Anna chooses a separate news newsletter and decides to assess later whether that channel is sufficient.

This is an editorial illustration of a personal decision rather than a research result. The Allcott experiment does not prove that a newsletter will preserve Anna’s previous knowledge level or that this replacement suits other people. The example simply shows two independent questions: how much space the feed occupies and which information function it serves.

For practical digital-tool choices, consult the separate Sinqly guide ‘Digital minimalism: choosing the tools you need’: https://sinqly.ru/blog/cifrovoy-minimalizm-gid. When reading research, distinguish a measured effect, its limits, and your own decision about information channels.

Sources

• Allcott, Braghieri, Eichmeyer, Gentzkow. The Welfare Effects of Social Media. Primary author version dated November 8, 2019
• The Welfare Effects of Social Media. American Economic Review, 110(3), 2020, 629–676. Published abstract
• Sinqly: Digital minimalism — choosing the tools you need

Frequently asked questions