Introduction: Why Reddit Comment Data Is a Goldmine (and a Headache)
Reddit hosts over 10 billion public comments spanning every niche imaginable—from real-world software debugging sessions and hardware benchmarks to unvarnished customer feedback and market sentiment. If you are building AI agents, Retrieval-Augmented Generation (RAG) knowledge bases, social listening monitors, or automated research bots, knowing how to get Reddit comments as JSON is an indispensable superpower.
At first glance, extracting comments looks deceptively simple. Most developers quickly learn that adding .json to the end of any public Reddit post URL returns a payload of data.
However, taking that raw response into production reveals an architectural maze designed in the late 2000s:
- Orphaned
MoreCommentsobjects: Reddit truncates conversation branches into stub objects containing only opaque comment IDs. The comment bodies, scores, and timestamps are stripped out entirely. - Exhaustive pagination hurdles: Fetching truncated replies requires dozens of sequential round-trips with arbitrary
aftertokens and rate-limited endpoints. - Brittle schema inconsistencies: Reddit represents an empty reply list as an empty string
""rather than an empty array[], immediately crashing strongly-typed parsers in TypeScript, Go, and Python Pydantic. - Aggressive IP blocking: Requests from cloud providers like AWS, Vercel, and GCP trigger instant
429 Too Many Requestsor Cloudflare challenges without residential proxy rotation.
The Problem with Reddit's Native Comment JSON
To understand why automated parsing breaks, examine what happens when you curl Reddit's raw URL directly:
Instead of returning an intuitive object with post and comments keys, Reddit returns a two-element array where index 0 contains post metadata and index 1 contains the top-level comment listing:
[
{
"kind": "Listing",
"data": {
"children": [
{
"kind": "t3",
"data": {
"id": "18xyz9",
"name": "t3_18xyz9",
"title": "Best local LLMs for code generation in 2026?",
"author": "dev_architect",
"score": 420,
"num_comments": 85
}
}
]
}
},
{
"kind": "Listing",
"data": {
"children": [
{
"kind": "t1",
"data": {
"id": "m1a2b3c",
"name": "t1_m1a2b3c",
"author": "ai_researcher",
"body": "DeepSeek-Coder 33B and Qwen 2.5 Coder are exceptional.",
"score": 142,
"replies": {
"kind": "Listing",
"data": {
"children": [
{
"kind": "t1",
"data": {
"id": "m1a2b3d",
"name": "t1_m1a2b3d",
"author": "junior_coder",
"body": "What quantized format runs best on 16GB VRAM?",
"score": 19,
"replies": "" // <-- EMPTY STRING WHEN EMPTY, OBJECT WHEN NESTED!
}
}
]
}
}
}
},
{
"kind": "more", // <-- MORECOMMENTS STUB: DROPS BODIES AND AUTHORS!
"data": {
"count": 27,
"name": "t1_m1a2b3e",
"id": "m1a2b3e",
"parent_id": "t3_18xyz9",
"children": [
"m1a2b3f",
"m1a2b3g",
"m1a2b3h"
]
}
}
]
}
}
]The 3 Structural Roadblocks
1. Type Mutation
replies is a JSON Listing object if nested comments exist, but an empty string "" if empty.
2. MoreComments Stubs
Deep replies vanish into "kind": "more" stubs. Re-hydrating them requires dozens of extra HTTP POSTs to /api/morechildren.
3. Wrapper Bloat
Every node is buried beneath 4 nested layers: data.children[0].data.body, multiplying memory and token waste.
Getting Comments with SubScraper
SubScraper abstracts away all the low-level headaches. Behind the scenes, SubScraper handles TLS fingerprint bypasses, rotates residential proxies, automatically traverses more comment stubs, normalizes empty values into clean arrays, and reconstructs the complete conversational hierarchy.
Using the official @subscraper/sdk or standard REST API, you can fetch both the post and its recursive comment tree in a single call:
import { SubScraperClient } from '@subscraper/sdk';
const client = new SubScraperClient({
apiKey: process.env.SUBSCRAPER_API_KEY!
});
// Fetch post and complete nested comment tree
const { post, comments } = await client.getPost({
postId: '18xyz9', // or full url: 'https://reddit.com/r/programming/comments/18xyz9/...'
limit: 100
});
console.log(`Post: ${post.title} (${post.score} upvotes)`);
console.log(`Fetched ${comments.length} top-level comment threads`);Or if you prefer standard cURL / HTTP REST:
Notice the contrast in output. Instead of nested listings and type bugs, SubScraper returns an intuitive, production-grade payload:
{
"post": {
"id": "t3_18xyz9",
"title": "Best local LLMs for code generation in 2026?",
"author": "dev_architect",
"subreddit": "programming",
"score": 420,
"numComments": 85,
"createdAt": "2026-03-14T10:15:00Z"
},
"comments": [
{
"id": "t1_m1a2b3c",
"author": "ai_researcher",
"body": "DeepSeek-Coder 33B and Qwen 2.5 Coder are exceptional.",
"score": 142,
"createdAt": "2026-03-14T10:28:40Z",
"replies": [
{
"id": "t1_m1a2b3d",
"author": "junior_coder",
"body": "What quantized format runs best on 16GB VRAM?",
"score": 19,
"createdAt": "2026-03-14T11:02:15Z",
"replies": []
}
]
}
]
}Understanding the Comment Tree Structure
In SubScraper, every comment adheres to a strict, recursive TypeScript contract. Because replies is always an array of Comment objects, you never have to worry about undefined, null, or string type mismatches.
export interface Comment {
id: string;
body: string;
author: string;
score: number;
createdAt?: string;
permalink?: string;
replies: Comment[]; // Recursive nested comment tree
}
export interface PostResponse {
post: {
id: string;
title: string;
body: string;
author: string;
subreddit: string;
score: number;
upvoteRatio: number;
createdAt: string;
numComments: number;
};
comments: Comment[];
}Why this recursive model matters:
- Safe UI rendering: Directly render nested threaded comments in React/Next.js with a simple recursive component (
<CommentItem comment={c} />) without defensive null checking. - Predictable depth traversal: Compute conversation depth, top upvoted branches, or controversial sub-threads using standard tree algorithms (BFS or DFS).
- Direct access to engagement metrics: Track
score,author, and timestamps at every branch level.
Flattening the Comment Tree for LLM & RAG Consumption
While tree hierarchies are ideal for user interfaces, Large Language Models (LLMs) prefer linear text. Ingesting deep JSON trees into Claude, GPT-4o, or Gemini wastes expensive context tokens on brackets, duplicate field keys, and commas.
Here is a production-ready helper function that recursively flattens the tree into a typed array with depth markers and formats it into an LLM-friendly dialogue transcript:
import type { Comment } from '@subscraper/sdk';
export interface FlatComment {
id: string;
author: string;
body: string;
score: number;
depth: number;
parentId: string | null;
}
/**
* Recursively flattens a nested Reddit comment tree into a linear array
* suitable for LLM context windows, embedding generation, or tabular storage.
*/
export function flattenCommentTree(
comments: Comment[],
depth = 0,
parentId: string | null = null
): FlatComment[] {
const flattened: FlatComment[] = [];
for (const comment of comments) {
// Push the current node
flattened.push({
id: comment.id,
author: comment.author,
body: comment.body,
score: comment.score,
depth,
parentId,
});
// Recursively flatten children if present
if (comment.replies && comment.replies.length > 0) {
flattened.push(
...flattenCommentTree(comment.replies, depth + 1, comment.id)
);
}
}
return flattened;
}
/**
* Formats a flattened comment tree into an indented dialogue transcript
* optimized for LLM prompts (RAG, sentiment analysis, debate summarization).
*/
export function formatCommentsForLLM(comments: Comment[]): string {
const flat = flattenCommentTree(comments);
return flat
.map((item) => {
const indentation = ' '.repeat(item.depth);
const prefix = item.depth > 0 ? `${indentation}↳ [Reply by u/${item.author} | +${item.score}]: ` : `[u/${item.author} | +${item.score}]: `;
return `${prefix}${item.body.replace(/\n+/g, ' ')}`;
})
.join('\n\n');
}LLM Prompt Formatting Result:
[u/ai_researcher | +142]: DeepSeek-Coder 33B and Qwen 2.5 Coder are exceptional. ↳ [Reply by u/junior_coder | +19]: What quantized format runs best on 16GB VRAM?
This format reduces token consumption by up to 65% compared to raw JSON while preserving the conversational reply relationship for RAG embeddings.
Fetching User Comment History
Sometimes your application needs comments written by a specific user across all subreddits, rather than comments on a single post. This is essential for:
Use SubScraper's getUserCommentsfunction to retrieve a chronological stream of a user's contributions:
import { SubScraperClient } from '@subscraper/sdk';
const client = new SubScraperClient({
apiKey: process.env.SUBSCRAPER_API_KEY!
});
// Fetch a user's recent comment history across all subreddits
const userHistory = await client.getUserComments({
username: 'shittymorph',
limit: 25,
sort: 'new'
});
console.log(`Retrieved ${userHistory.comments.length} comments from u/shittymorph`);
userHistory.comments.forEach((c) => {
console.log(`[${c.subreddit}] (+${c.score}): ${c.body.slice(0, 80)}...`);
});Getting a Single Comment with Context
In social listening and notification systems, you often receive a permalink to a single comment (e.g. from an alert webhook or mention crawler).
Downloading an entire 5,000-comment thread just to inspect one reply is inefficient. SubScraper provides the getCommentPermalink endpoint, which extracts the exact target comment, its immediate parent post title, and any direct nested replies:
import { SubScraperClient } from '@subscraper/sdk';
const client = new SubScraperClient({
apiKey: process.env.SUBSCRAPER_API_KEY!
});
// Fetch a specific comment, its parent context, and subsequent sub-tree
const threadContext = await client.getCommentPermalink({
commentUrl: 'https://www.reddit.com/r/programming/comments/18xyz9/best_local_llms/m1a2b3c/'
});
console.log('Original Post Title:', threadContext.post.title);
console.log('Target Comment by:', threadContext.comment.author);
console.log('Child Replies count:', threadContext.comment.replies.length);Ready to Extract Clean Reddit Comments as JSON?
Stop fighting rate limits, proxy bans, and empty-string typing bugs. SubScraper gives your applications instant access to typed, structured Reddit comments, posts, and community metrics.
No credit card required • Instant API key provisioning • Full TypeScript SDK & MCP support
⚡Related Tools & API References
Frequently Asked Questions
Common questions about getting Reddit comments in JSON format.
How do I get Reddit comments as JSON without an official API key?
While you can append .json to public Reddit URLs, Reddit aggressively blocks cloud IP addresses (returning HTTP 429 Too Many Requests), enforces custom User-Agent rules, and truncates replies into unexpanded more stubs. For automated software, SubScraper provides an unblocked REST API and TypeScript SDK with automatic proxy rotation and reconstructed comment trees.
Why is Reddit's native comment JSON so difficult to parse in production?
Reddit's native JSON has schema inconsistencies: when a comment has no replies, it returns an empty string "" instead of an empty array [], breaking standard JSON deserializers. Furthermore, deeply nested conversation branches are replaced with kind: "more" objects containing only comment IDs, requiring dozens of secondary API requests to fetch the actual text.
How do I prepare Reddit comments JSON for LLM prompts and vector embeddings?
LLMs digest linear text much more efficiently than deeply nested JSON. The recommended approach is to run a recursive tree flattening function that records each comment's author, text, score, and depth level, then output a formatted conversation transcript (e.g. using ↳ [Reply] indentation). This preserves conversational context while reducing token usage by up to 65%.