Why Reddit is the Ultimate Goldmine for Social Listening
Most social listening tools focus on Twitter/X, LinkedIn, or Instagram. While those platforms have high volume, the discussions on them are heavily performative. On LinkedIn, every post is a sanitized corporate announcement. On Twitter, algorithms amplify hot takes and engagement bait.
Reddit is completely different. Because users post pseudonymously, they express honest, unfiltered critiques about software, pricing hikes, feature gaps, and daily workflow frustrations. Communities like r/SaaS, r/webdev, r/startups, and r/technology are packed with users asking questions like:
“We just got hit with a 400% price hike by [Competitor X]. What are the best self-hosted or modern alternatives with a sensible developer tier?”
— Real discussion pattern seen daily in software subreddits
If your marketing or product team spots this thread 15 minutes after it is posted, you can participate authentically, answer their technical questions, or reach out with a direct solution. If you find it 4 days later, your competitors have already scooped up the lead.
In this hands-on tutorial, we will build an automated reddit social listening botfrom scratch using TypeScript. By the end, your bot will poll Reddit every 15 minutes, filter out noise, match high-intent buying signals, and deliver formatted alerts straight to your team's Slack channel.
Architecture Overview
A reliable social listening bot needs to be modular, lightweight, and resilient to transient network failures. Here is how our TypeScript pipeline operates:
Scheduler
Node-cron every 15 minutes
Query Engine
SubScraper SDK searchPosts
Filter & Dedupe
Score check + ID set
Slack Webhook
Rich Block Kit alert
- ✓Zero Rate Limiting: SubScraper handles residential proxy rotation, captcha solving, and Reddit markup shifts under the hood.
- ✓Low Resource Usage: The entire bot consumes less than 80MB of RAM and can run on a free or $5/month instance.
- ✓Idempotent: A persistent JSON state file ensures that bot restarts never result in spammy duplicate notifications.
Setting Up the Project
Let's create a clean Node.js project configured with modern TypeScript (ESM) and install our required dependencies:
mkdir reddit-listening-bot
cd reddit-listening-bot
npm init -y
# Install runtime dependencies
npm install @subscraper/sdk node-cron dotenv
# Install TypeScript and developer tooling
npm install -D typescript @types/node @types/node-cron tsxConfigure your tsconfig.json for modern Node ESM resolution:
{
"compilerOptions": {
"target": "ES2022",
"module": "NodeNext",
"moduleResolution": "NodeNext",
"esModuleInterop": true,
"strict": true,
"skipLibCheck": true,
"outDir": "dist"
},
"include": ["src/**/*"]
}Create a .env file in your root folder. You will need a SubScraper API key (from your SubScraper Dashboard) and an incoming webhook URL from your Slack workspace:
# .env configuration
SUBSCRAPER_API_KEY=sub_live_your_actual_api_key_here
SLACK_WEBHOOK_URL=https://hooks.slack.com/services/T00000000/B00000000/XXXXXXXXXXXXXXXXXXXXXXXX
CHECK_INTERVAL_CRON="*/15 * * * *"
MIN_POST_SCORE=2Defining Your Keywords and Subreddits
Social listening is only as good as the keywords you monitor. A naive bot that searches for a single generic word will flood your Slack channel with false positives.
Instead, we will partition our configuration into three distinct tiers:
- Brand Mentions: Your brand name, domain, and specific product identifiers.
- Competitor Intel: Names of direct competitors, common misspellings, and migration queries.
- Buyer Intent Triggers: High-intent phrases like “alternative to” or “recommendation for”.
Create src/config.ts:
// src/config.ts
import dotenv from 'dotenv';
dotenv.config();
export interface MonitorConfig {
subscraperApiKey: string;
slackWebhookUrl: string;
cronSchedule: string;
minScore: number;
subreddits: string[];
brandKeywords: string[];
competitorKeywords: string[];
intentTriggers: string[];
}
export const config: MonitorConfig = {
subscraperApiKey: process.env.SUBSCRAPER_API_KEY || '',
slackWebhookUrl: process.env.SLACK_WEBHOOK_URL || '',
cronSchedule: process.env.CHECK_INTERVAL_CRON || '*/15 * * * *',
minScore: parseInt(process.env.MIN_POST_SCORE || '2', 10),
// Targeted subreddits where your buyers and peers hang out
subreddits: [
'saas',
'startups',
'webdev',
'technology',
'entrepreneur',
'softwarearchitecture',
],
// Brand aliases, product names, and company domain
brandKeywords: [
'SubScraper',
'subscraper.dev',
'reddit scraper api',
],
// Competitor solutions you want to monitor for switching intent
competitorKeywords: [
'Apify reddit',
'PRAW scraper',
'BrightData reddit',
'Reddit official API pricing',
],
// High-intent phrasing indicating buyer interest or problem points
intentTriggers: [
'alternative to',
'looking for a tool',
'recommend a',
'frustrated with',
'rate limit error',
'how do you scrape',
],
};
if (!config.subscraperApiKey) {
throw new Error('Missing SUBSCRAPER_API_KEY in environment variables.');
}
if (!config.slackWebhookUrl) {
throw new Error('Missing SLACK_WEBHOOK_URL in environment variables.');
}Querying Reddit with the SubScraper SDK
Now let's build the query service. SubScraper's official SDK (@subscraper/sdk) provides fully-typed access to Reddit search.
We construct a compound boolean query using OR operators, search targeted subreddits, sort by 'new', and apply time: 'day' to capture conversations published in the last 24 hours:
// src/reddit.ts
import { SubScraperClient, PostItem } from '@subscraper/sdk';
import { config } from './config.js';
const client = new SubScraperClient({
apiKey: config.subscraperApiKey,
});
/**
* Builds a search query combining brand, competitors, and intent triggers
*/
export function buildQuery(): string {
const terms = [
...config.brandKeywords.map((k) => `"${k}"`),
...config.competitorKeywords.map((k) => `"${k}"`),
];
return terms.join(' OR ');
}
/**
* Searches Reddit for relevant discussions published within the past 24 hours
*/
export async function fetchRecentPosts(): Promise<PostItem[]> {
const query = buildQuery();
const allResults: PostItem[] = [];
console.log(`[Reddit] Executing search query across target subreddits...`);
for (const subreddit of config.subreddits) {
try {
const response = await client.searchPosts({
query,
subreddit,
sort: 'new', // Fetch newest discussions first
time: 'day', // Focus on discussions from the past 24 hours
limit: 25,
});
const posts = response.items || [];
console.log(`[Reddit] r/${subreddit}: Retrieved ${posts.length} posts.`);
allResults.push(...posts);
} catch (error) {
console.error(`[Reddit] Error querying r/${subreddit}:`, error);
}
}
return allResults;
}Filtering for High-Intent Posts & Deduplication
Raw search results can still contain spam, automated bot submissions, or negative-karma posts. Moreover, when running on a 15-minute cron schedule, posts created 2 hours ago will show up on every single run.
In src/filter.ts, we enforce:
- Stateful Deduplication: We maintain a set of seen post IDs persisted to
seen_post_ids.json. - Karma Threshold: Filter out low-effort or heavily downvoted posts (
score >= minScore). - Category Tagging: Classify each post as Brand Mention, Competitor Intel, or Buyer Intent so your team knows the context instantly in Slack.
// src/filter.ts
import fs from 'node:fs';
import path from 'node:path';
import { PostItem } from '@subscraper/sdk';
import { config } from './config.js';
const CACHE_FILE = path.resolve(process.cwd(), 'seen_post_ids.json');
// Load previously alerted post IDs from local disk cache
function loadSeenPostIds(): Set<string> {
if (fs.existsSync(CACHE_FILE)) {
try {
const raw = fs.readFileSync(CACHE_FILE, 'utf8');
const parsed = JSON.parse(raw);
if (Array.isArray(parsed)) return new Set(parsed);
} catch (e) {
console.warn('[Cache] Could not parse seen_post_ids.json. Starting fresh.');
}
}
return new Set<string>();
}
// Persist post IDs to prevent duplicate alerts across server restarts
function saveSeenPostIds(ids: Set<string>): void {
try {
// Keep only the last 1,000 IDs to avoid unbounded growth
const serialized = JSON.stringify(Array.from(ids).slice(-1000), null, 2);
fs.writeFileSync(CACHE_FILE, serialized, 'utf8');
} catch (err) {
console.error('[Cache] Failed to save seen_post_ids.json:', err);
}
}
const seenPostIds = loadSeenPostIds();
export interface FilteredPost {
post: PostItem;
matchedCategory: 'Brand Mention' | 'Competitor Intel' | 'Buyer Intent';
matchedKeywords: string[];
}
export function filterHighIntentPosts(posts: PostItem[]): FilteredPost[] {
const matched: FilteredPost[] = [];
for (const post of posts) {
const id = post.id || post.fullname;
if (!id || seenPostIds.has(id)) {
continue; // Skip previously processed posts
}
// Filter out zero-karma posts or spam
const score = post.score ?? 0;
if (score < config.minScore) {
continue;
}
const searchableText = `${post.title ?? ''} ${post.previewText ?? ''}`.toLowerCase();
// Check for brand mentions
const brandMatches = config.brandKeywords.filter((k) =>
searchableText.includes(k.toLowerCase())
);
// Check for competitor queries
const competitorMatches = config.competitorKeywords.filter((k) =>
searchableText.includes(k.toLowerCase())
);
// Check for high-intent trigger phrases
const intentMatches = config.intentTriggers.filter((k) =>
searchableText.includes(k.toLowerCase())
);
if (brandMatches.length > 0) {
matched.push({
post,
matchedCategory: 'Brand Mention',
matchedKeywords: brandMatches,
});
seenPostIds.add(id);
} else if (competitorMatches.length > 0) {
matched.push({
post,
matchedCategory: 'Competitor Intel',
matchedKeywords: competitorMatches,
});
seenPostIds.add(id);
} else if (intentMatches.length > 0) {
matched.push({
post,
matchedCategory: 'Buyer Intent',
matchedKeywords: intentMatches,
});
seenPostIds.add(id);
}
}
saveSeenPostIds(seenPostIds);
return matched;
}Sending Slack Alerts with Block Kit
A plain text webhook message is easy to overlook. Using Slack's Block Kit format, we can build an interactive, readable alert that displays:
- Category indicator badge (🚨 Brand Mention, ⚔️ Competitor Intel, 🎯 High-Intent Lead)
- Post title linked directly to the Reddit thread
- Body text snippet preview
- Subreddit, author username, upvote score, and total comment count
- One-click “Open on Reddit” interactive button
Create src/slack.ts:
// src/slack.ts
import { FilteredPost } from './filter.js';
import { config } from './config.js';
export async function sendSlackAlert(item: FilteredPost): Promise<void> {
const { post, matchedCategory, matchedKeywords } = item;
const redditUrl = post.permalink
? `https://reddit.com${post.permalink}`
: post.url || 'https://reddit.com';
const categoryEmoji = {
'Brand Mention': '🚨 *Brand Mention*',
'Competitor Intel': '⚔️ *Competitor Intel*',
'Buyer Intent': '🎯 *High-Intent Lead*',
}[matchedCategory];
const bodySnippet = post.previewText
? (post.previewText.length > 280
? `${post.previewText.slice(0, 280)}...`
: post.previewText)
: '_No preview text provided._';
const payload = {
text: `${categoryEmoji}: ${post.title} (r/${post.subreddit})`,
blocks: [
{
type: 'header',
text: {
type: 'plain_text',
text: `${categoryEmoji}`.replace(/[*_]/g, ''),
emoji: true,
},
},
{
type: 'section',
text: {
type: 'mrkdwn',
text: `*<${redditUrl}|${post.title}>*\n${bodySnippet}`,
},
},
{
type: 'context',
elements: [
{
type: 'mrkdwn',
text: `*Subreddit:* r/${post.subreddit} | *Author:* u/${post.author || 'unknown'}`,
},
{
type: 'mrkdwn',
text: `*Score:* ▲ ${post.score ?? 0} | *Comments:* 💬 ${post.commentCount ?? 0}`,
},
{
type: 'mrkdwn',
text: `*Matched:* ``${matchedKeywords.join(', ')}```,
},
],
},
{
type: 'actions',
elements: [
{
type: 'button',
text: {
type: 'plain_text',
text: 'Open on Reddit ↗',
},
url: redditUrl,
style: 'primary',
},
],
},
{
type: 'divider',
},
],
};
const response = await fetch(config.slackWebhookUrl, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(payload),
});
if (!response.ok) {
const errorBody = await response.text();
throw new Error(`Slack Webhook error (${response.status}): ${errorBody}`);
}
}Scheduling with Node-cron
Now let's tie everything together in src/index.ts.
We run an initial listening cycle immediately when the process starts (so you can verify credentials and receive an instant test alert), and then schedule recurring execution every 15 minutes using node-cron:
// src/index.ts
import cron from 'node-cron';
import { config } from './config.js';
import { fetchRecentPosts } from './reddit.js';
import { filterHighIntentPosts } from './filter.js';
import { sendSlackAlert } from './slack.js';
let isRunning = false;
async function runSocialListeningCycle() {
if (isRunning) {
console.log('[Bot] Previous cycle still executing. Skipping this turn.');
return;
}
isRunning = true;
const startTime = Date.now();
console.log(`\n==============================================`);
console.log(`[Bot] Starting social listening cycle: ${new Date().toISOString()}`);
try {
// 1. Fetch recent discussions from SubScraper
const rawPosts = await fetchRecentPosts();
console.log(`[Bot] Total posts fetched: ${rawPosts.length}`);
// 2. Filter for keywords, score threshold, and deduplication
const highIntentMatches = filterHighIntentPosts(rawPosts);
console.log(`[Bot] High-intent matching posts found: ${highIntentMatches.length}`);
// 3. Dispatch alerts to Slack
for (const match of highIntentMatches) {
console.log(`[Bot] Dispatching Slack alert for: "${match.post.title}"`);
await sendSlackAlert(match);
// Brief pause to respect Slack webhook rate limits
await new Promise((res) => setTimeout(res, 500));
}
const elapsed = ((Date.now() - startTime) / 1000).toFixed(2);
console.log(`[Bot] Cycle finished successfully in ${elapsed}s.`);
} catch (error) {
console.error('[Bot] Unhandled error during listening cycle:', error);
} finally {
isRunning = false;
}
}
// 1. Run an immediate cycle on startup to test connectivity
runSocialListeningCycle();
// 2. Schedule recurring cron (default: every 15 minutes)
console.log(`[Bot] Registering cron schedule: "${config.cronSchedule}"`);
cron.schedule(config.cronSchedule, () => {
runSocialListeningCycle();
});
// Graceful shutdown handling
process.on('SIGINT', () => {
console.log('\n[Bot] Received SIGINT. Shutting down gracefully...');
process.exit(0);
});
process.on('SIGTERM', () => {
console.log('\n[Bot] Received SIGTERM. Shutting down gracefully...');
process.exit(0);
});Testing Locally
You can run the bot directly with tsx without a separate build step:
Deploying to Fly.io or Railway
To keep your social listening bot running 24/7, deploy it to a lightweight cloud runner. A single background worker on Railway or Fly.io costs next to nothing ($0 to $5/month) and takes under 5 minutes to set up.
Option A: Deploying on Railway
- Push your code to a private GitHub repository.
- Log in to Railway, click New Project → Deploy from GitHub repo.
- In your service settings, navigate to Variables and add your environment secrets (
SUBSCRAPER_API_KEYandSLACK_WEBHOOK_URL). - Set the start command to
npm run build && node dist/index.js. Railway will keep the worker running continuously.
Option B: Deploying with Docker on Fly.io
Create a multi-stage Dockerfile to keep the image slim:
# Dockerfile
FROM node:20-alpine AS builder
WORKDIR /app
COPY package*.json tsconfig.json ./
RUN npm ci
COPY src ./src
RUN npm run build
FROM node:20-alpine AS runner
WORKDIR /app
ENV NODE_ENV=production
COPY package*.json ./
RUN npm ci --omit=dev
COPY --from=builder /app/dist ./dist
# Keep persistent volume for seen post IDs
VOLUME ["/app/data"]
CMD ["node", "dist/index.js"]Initialize Fly.io and set your secrets via the CLI:
# Launch Fly app configuration
fly launch --no-deploy
# Set secrets securely
fly secrets set SUBSCRAPER_API_KEY=sub_live_xxx SLACK_WEBHOOK_URL=https://hooks.slack.com/...
# Deploy background worker
fly deploy
Conclusion & Next Steps
You now have a fully functional, automated Reddit social listening bot running in TypeScript. Instead of manually refreshing subreddits or paying thousands of dollars for bloated enterprise monitoring software, you have full control over your keywords, filtering logic, and alerting channels.
Here are a few high-value enhancements you can add next:
- AI Sentiment Classification: Pipe matching posts through Claude 3.5 or GPT-4o-mini to calculate positive/negative sentiment scores before alerting.
- CRM Lead Enrichment: Automatically push users seeking software alternatives into HubSpot or your sales pipeline.
- Comment Thread Monitoring: Use SubScraper's getPost endpoint to inspect full comment trees for viral discussions.
Ready to Build Your Social Listening Pipeline?
Get instant access to real-time Reddit search, subreddit scraping, and user data. No proxy configuration, no captcha headaches, and 30 free requests every day.
⚡Related Tools & API References
Frequently Asked Questions
How often should a Reddit social listening bot poll for new posts?
Polling every 10 to 15 minutes is the ideal interval for Reddit social listening. It ensures you discover customer pain points, PR complaints, and competitor comparisons almost as they happen while staying well within reasonable API quotas without triggering rate limits.
Why use the SubScraper SDK instead of Reddit's official API for social listening?
Reddit's official enterprise API requires prohibitive annual contracts costing upwards of tens of thousands of dollars, along with strict commercial approval barriers and strict rate caps. SubScraper provides clean, typed JSON data through residential proxies with zero rate-limit blocks, simple API key authentication, and affordable prepaid packs starting at $1.99.
How do you prevent duplicate Slack alerts when polling Reddit periodically?
To prevent duplicate notifications, maintain a persistent store of processed post IDs (such as a local JSON cache file, SQLite, or Redis). The bot checks whether the post's unique Reddit ID has already been recorded before dispatching the Slack webhook, and saves newly processed IDs immediately upon alerting.