Optimize crawling: let’s turn things around! • joost.blog

I wrote about crawl optimization last week, mostly about getting stuff that you don’t want crawled to not be crawled. There’s more to say about that, and I will in follow up posts, but first we need to talk about how to get the stuff that you do want indexed, crawled and indexed by search engines. To that end we’ll have to talk about XML sitemaps and the new kid in town, IndexNow. At the end of this post I’ll propose a radical change to how we approach indexing. One that might not come into existence anytime soon, but that…