Files
root cc7a5f2b7c Make the site crawlable and describe itself properly
public/robots.txt was "Disallow: /", which told every crawler to ignore the
entire site. It now allows crawling, keeps /admin and the API paths out of the
index, and points at the sitemap.

Add a generated sitemap covering the static routes plus every post and
category. The API caps per_page at 50, so it follows meta.last_page instead of
asking for more, and every fetch is wrapped so an unreachable API degrades to
the static entries rather than taking the site down.

Replace the placeholder layout metadata with real Persian titles, description,
keywords, canonical URL, hreflang alternates, OpenGraph and Twitter cards, and
add Person/WebSite JSON-LD.
2026-10-01 16:33:19 -04:00

14 lines
291 B
Plaintext

User-agent: *
Allow: /
# Admin panel: no indexing, no crawling of the panel or its assets.
Disallow: /admin
Disallow: /admin/
Disallow: /admin/*
# Internal helpers that should never appear in search results.
Disallow: /og-image
Disallow: /api/
Sitemap: https://parsaaghayi.ir/sitemap.xml