public/robots.txt was "Disallow: /", which told every crawler to ignore the entire site. It now allows crawling, keeps /admin and the API paths out of the index, and points at the sitemap. Add a generated sitemap covering the static routes plus every post and category. The API caps per_page at 50, so it follows meta.last_page instead of asking for more, and every fetch is wrapped so an unreachable API degrades to the static entries rather than taking the site down. Replace the placeholder layout metadata with real Persian titles, description, keywords, canonical URL, hreflang alternates, OpenGraph and Twitter cards, and add Person/WebSite JSON-LD.
14 lines
291 B
Plaintext
14 lines
291 B
Plaintext
User-agent: *
|
|
Allow: /
|
|
|
|
# Admin panel: no indexing, no crawling of the panel or its assets.
|
|
Disallow: /admin
|
|
Disallow: /admin/
|
|
Disallow: /admin/*
|
|
|
|
# Internal helpers that should never appear in search results.
|
|
Disallow: /og-image
|
|
Disallow: /api/
|
|
|
|
Sitemap: https://parsaaghayi.ir/sitemap.xml
|