Reddit AI Search Favors Popular Comments, Audit Reveals

Reddit’s AI search appears more likely to pull formal, already-upvoted comments into its answers than comments carrying stronger signs of personal experience, according to a new preprint from researchers at the University of Illinois Urbana-Champaign.

The team ran 10,000 questions through the feature three times and analyzed 30,000 answers, tracing them back through 14.68 million comments. The questions came from 20 advice and support subreddits, so the findings apply to that slice of Reddit rather than the platform as a whole, and the paper has not yet been peer reviewed.

On top of that, the answers stripped out almost all of the first-person language used in the comments they quoted, which the authors describe as turning personal testimony into generic advice.

Formal Language Correlated With Higher Selection Odds

The researchers compared comments that an answer quoted or listed as a source against other comments in the same discussion that the answer skipped over.

Formality turned out to be the strongest language predictor of selection. When a comment was one standard deviation more formal, its odds of being selected rose by 49% (odds ratio 1.488), with formality scored by a text classifier. Comments using words like “should” and “must” also had higher odds of being picked (1.070).

Comments with more markers of personal experience had lower odds of selection (0.789), as did supportive language (0.924). The paper labels the first measure experiential voice and scores it from features such as first-person pronouns and past tense.

Formality also tracked with higher community voting, while experiential voice tracked with lower voting. Once the researchers controlled for a comment’s score, position, and age, the associations weakened to 1.213 for formality and 0.860 for experiential voice.

None of this shows that writing formally gets a comment selected. As the authors put it, “Our study is observational and should not be interpreted causally.”

Comments That Were Already Visible Had An Edge

A comment’s vote ranking within its thread showed the largest association in the selection model: a one-standard-deviation increase multiplied the odds of being chosen by 2.88.

The median selected comment sat at the 91st percentile for score in its thread, versus the 45th percentile for comments that were not chosen. Among selected comments, 92% were direct replies to the original post, which also made up 53% of all comments. Selected comments also appeared a median of 1.2 hours after the post, while non-selected comments appeared after a median of 5.9 hours.

A one-standard-deviation increase in comment length carried an odds ratio of 1.79, and comments containing an external link had an odds ratio of 2.25.

Comments scoring zero or below accounted for 0.53% of selected comments and 5.1% of all comments collected.

Written Answers Dropped Most First-Person Language

In an analysis of 1,000 queries, the researchers compared Reddit’s quoted comments with the responses built around them. Use of words like “I” and “my” fell from 3.3% in the quoted comments to 0.06% in the answers.

The same queries were run through GPT-4o-mini and GPT-5 via OpenAI’s API with web search enabled. Both models used less first-person language than the Reddit comments, and Reddit’s own responses showed the largest reduction.

Across 4,932 citations, 18 referenced Reddit under both models and search configurations. That figure comes from API calls for advice queries, which differs from the setup used in the ChatGPT app.

How The Audit Was Run

The researchers used an LLM to rewrite real posts from the 20 subreddits into short search queries. Ten are large communities such as r/personalfinance and r/AskDocs, and ten are smaller ones such as r/UKJobs and r/AusLegal.

The three runs were spaced about five hours apart so that differences would reflect the system rather than changes on Reddit. When two runs pulled from the same source posts, the responses came out almost identical, which suggests most of the variation came from which posts the system chose to retrieve.

A typical answer drew on about seven different subreddits. The subreddit where the question was first asked contributed 17.7% of retrieved posts in the large-community set and 15.7% in the small-community set. The authors note they don’t treat a higher share as better.

The paper refers to the feature as Reddit Answers, its name when I first reported on its launch in December 2024. Reddit’s announcement page now shows a May 26, 2026 update stating, “Reddit Answers is now merged into Reddit search for one unified search experience.”

The questions were built from posts dated through July 2026, so these runs took place after that change, though the paper doesn’t specify the collection dates. Reddit’s Help page now describes the feature as AI search, reachable through an Ask button in the search bar.

Why This Matters

Reddit’s value for audience research comes from people describing their own experience with a product or a problem. In these 20 communities, comments with more markers of that experience were less likely to reach the AI answer, and most of the first-person wording was gone by the time they did.

Reddit CEO Steve Huffman told investors in February that Reddit is particularly good at questions where “the answer actually is multiple perspectives from lots of people.” This audit measured which of those perspectives actually reached the AI answer.

The source threads are listed beneath each answer, so read them before you put the summary into a report.

Looking Ahead

The authors caution that these selection patterns may not carry over to other kinds of communities or to other AI search systems. This story will be updated if a revised or peer-reviewed version changes the findings.

Featured Image: Accogliente Design/Shutterstock