GitHub Packages · MIT · Node ≥ 18
Check which pages of your built static site search engines are told not to index or follow, via meta robots and X-Robots-Tag, against the rules you actually intend.
Check indexability of your built static site: meta robots and X-Robots-Tag (noindex, nofollow, none, unavailable_after, nosnippet) against your intended indexing rules, site-wide noindex leaks, conflicts and malformed directives. Text/JSON/SARIF/Markdown, allowlist with expiry, GitHub Action. Zero deps.
1 Point the @mrlouiswright scope at GitHub Packages in .npmrc
@mrlouiswright:registry=https://npm.pkg.github.com2 Install
npm i -D @mrlouiswright/index-beaconPublished to GitHub Packages, not npmjs.com. GitHub Packages needs a GitHub token with read:packages to install, even for public packages: add //npm.pkg.github.com/:_authToken=${GITHUB_TOKEN} to .npmrc or run npm login --registry=https://npm.pkg.github.com.
$ npx index-beacon test/fixtures/badindex-beacon — /workspace/index-beacon/test/fixtures/bad11 page(s); 5 noindex HIGH _headers:1 /hdr.html is noindex via X-Robots-Tag "noindex" (_headers) [index-unexpected-noindex] ↳ fix: Remove noindex, or add "/hdr.html" to the "noindex" rules if it is intentional.HIGH bot.html:1 /bot.html is noindex via <meta name="googlebot" content="noindex"> [index-unexpected-noindex] ↳ fix: Remove noindex, or add "/bot.html" to the "noindex" rules if it is intentional.HIGH both.html:1 /both.html is noindex via <meta name="robots" content="noindex"> [index-unexpected-noindex] ↳ fix: Remove noindex, or add "/both.html" to the "noindex" rules if it is intentional.HIGH docs.html:1 <meta name="robots" content="noindex"> on /docs.html is outside <head> [index-meta-in-body] ↳ fix: Move robots meta tags into <head>.HIGH docs.html:1 /docs.html is noindex via <meta name="robots" content="noindex"> [index-unexpected-noindex]
One stray noindex can remove a money page from Google, and a staging “X-Robots-Tag: noindex” that ships to production can remove the whole site. The opposite also happens: search result pages, drafts and thank-you pages get indexed. index-beacon reads every built HTML page and the header rules in Netlify/Cloudflare _headers and vercel.json, works out the effective robots directives for each page, and compares them with your intended indexing rules. Each finding has the file, line, severity and a concrete fix.
CLI, library and GitHub Action. Node.js 18+, zero runtime dependencies. Works on any static build output (Astro, Next.js “output: 'export'”, Eleventy, Hugo, Jekyll, Docusaurus, VitePress, plain HTML) and runs offline: point it at the folder your build writes. By Louis Wright.
declare which pages should be noindex (noindex globs; sensible defaults such as /search, /admin/, /drafts/, /thank-you*, /cart*), which must stay indexable (index globs win), and where nofollow is intended. Everything else is expected to be indexable and followed.
site-wide noindex, from a /* header rule or because every page is noindex (critical).
unexpected noindex/none, unexpected nofollow, indexable pages that should be noindex, passed unavailable_after dates.
robots meta outside <head>, contradictory directives (index + noindex, meta vs header), bot-specific rules (googlebot, bingbot, ...) that disagree with the generic one, unknown or misspelt directives (no-index).
nosnippet / max-snippet:0 on indexable pages (hides them from rich results and AI answers) and noimageindex.
# Example
npx index-beacon dist# Example
npx index-beacon dist --noindex "/internal/**,/landing/test-*" --fail-on medium# Example
npx index-beacon public --index "/search" --nofollow "/user/**"# Example
npx index-beacon out --no-defaults --report-intended --format markdownpermissions:
contents: read
security-events: write # only needed for the SARIF upload
jobs:
index-beacon:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
with: { persist-credentials: false }
- run: npm ci && npm run build # produce the static site first
- uses: MrLouisWright/index-beacon@v1
with:
path: dist
fail-on: high
sarif: index-beacon.sarif
- uses: github/codeql-action/upload-sarif@v3
if: always()
with: { sarif_file: index-beacon.sarif }| Flag | Description |
|---|---|
| Site | |
--base-url <url> | Public URL of the site, e.g. https://example.com (default: inferred from canonical/og:url) |
--include <globs> | Only check these pages (file or URL path globs, comma-separated) |
--exclude <globs> | Skip these pages |
--clean-urls | Pages are served without .html (about.html -> /about) |
| Intended indexing | |
--noindex <globs> | Pages that should be noindex (added to the defaults) |
--index <globs> | Pages that must be indexable (wins over --noindex) |
--nofollow <globs> | Pages where nofollow is intended |
--no-defaults | Do not use the built-in noindex globs (/search, /admin/**, /drafts/**, ...) |
--report-intended | Also list intentionally noindex pages (info) |
| Common | |
-c, --config <file> | JSON config (default: .index-beacon.json or "indexBeacon" in package.json) |
-f, --format <fmt> | text (default) | json | sarif | markdown |
--json <file> | Also write a JSON report |
--sarif <file> | Also write SARIF 2.1.0 |
--markdown <file> | Also write Markdown (e.g. for a job summary) |
--fail-on <sev> | critical | high | medium | low | info | none (default: high) |
--list-rules | Print rules and default severities |
--no-fixes | Hide fix suggestions in text output |
--no-color | Disable colours |
Full reference, config file format and API in the README.
Support this project
I build free, open-source tools that help developers ship safer websites. Every coffee helps keep them maintained, documented and free for everyone.
Buy me a coffee (opens Stripe in a new tab)Secure checkout by Stripe
