← All runs / RUN #004Next · RUN #003 →
GeoLumen
Free AI search visibility (GEO + SEO) tools for small businesses and creators: an AI crawler checker against RFC 9309 and an llms.txt generator and validator, built behind a safe fetch that blocks private IPs on every redirect.
- Client
- Own product · AI search visibility (GEO + SEO) for small businesses and creators
- Role
- Design · build · ship
- Timeline
- Ongoing
- Stack
- Next.js 15 · TypeScript · Tailwind v4 · shadcn/ui · Supabase
- Status
- Building
01 — The system
Trigger01Domain submittedowner enters a URL
Tool02Safe fetch runshttp(s) only · ports 80/443 · private IPs blocked, pinned DNS on redirects
Code03Match robots.txt to AI crawlersRFC 9309 rule matching
Code04Generate & validate llms.txtsame safe fetch
Code05Visibility report readycrawler + llms.txt results
02 — Results
2free tools live: an AI crawler checker and an llms.txt generator
RFC 9309the spec the AI crawler checker matches robots.txt against
03 — The problem
Small businesses and creators want to show up in AI search, not just classic search, but rarely have a simple, free way to check whether AI crawlers are allowed to read their site, or to create and check an llms.txt file.
04 — Decisions & tradeoffs
- Match robots.txt the way the spec does. The AI crawler checker parses a site's robots.txt and matches it against real AI crawler user agents using RFC 9309's rule matching, not a hand-rolled approximation.
- An llms.txt generator and validator, so a site can produce the file and check it is well-formed, not just read about the convention.
- A hardened safe fetch everywhere a URL is fetched. Only http(s) on ports 80 and 443; private IPs are blocked with pinned DNS, including on every redirect hop; requests are timed out, capped in size, and rate-limited per IP. Checking someone else's URL is an SSRF surface, so the fetch path treats it as untrusted input.
- A Supabase waitlist, since the tools are still being built out: a site can test its own robots.txt and llms.txt today, and join the waitlist for what's next.
05 — What's next
More GEO checks beyond the AI crawler checker and the llms.txt tools, and opening access past the waitlist.