As you may have heard, Google has recently scaled back the effectiveness of their "submit to Google" tool to thwart SEOs using it to get low quality pages indexed quickly. Pages that once indexed in under 15 minutes are now waiting days, if not weeks to be picked up.
Like many changes of the past, this update has affected the rate at which other sites are being indexed as well, especially newer sites and local business websites that don't update frequently. Fortunately, there are some simple steps you can take to increase the speed at which your site gets indexed.
Although they deny the impact of social signals on SEO, we are certain that social media popularity can result in faster indexation. Twitter for example is a data partner with Google, you may have even seen tweets appearing directly in your search results. One study conducted by Eric Enge found a loose correlation between Tweets and indexation, however, Google has most certainly advanced their capabilities since 2015.
Simply sharing your content on social media channels could help it to be picked up faster, if you can get retweets, especially from more authoritative profiles, the chances increase noticeably.
We have all heard horror stories about robots.txt file inadvertently being set to no-index, no-follow and tanking a website. As such, checking the file has become a standard part of any good SEO audit. Aside from making sure pages aren't being blocked accidentally, there are a few other steps you should take to maximize the impact of your robots.txt file.
As touched on in Ecommerce SEO: The Mistakes That Stifle Rankings & Sales , one use of the file is to prevent unwanted bots from scraping data from your site. Ryan Stewart from Webris put together an excellent robots.txt guide, if you aren't sure how-to setup the optimal file, it is a worthwhile read.
Google loves authority sites. Since "authority" is a measure of trust, it stands to reason that the more trusted a site is, the more likely it is to have the right answer to the user's query. This concept is what allows authority sites to rank for thousands of keywords and is also the loophole low quality sites are using to outrank low power sites with great content.
The fasted way to gain "authority" is by gaining inbound links from other credible websites. Pair this with points 4 & 5 and you have a recipe for rapid growth.
Specific to indexation, if 20 authority sites are linking to a page on your website, Googlebot is going to see the link and crawl it. The authority and frequent content production of authority sites draws Google to the site more often, meaning your link and website are going to be seen quickly.
To get new pages or posts picked up quickly, you need to create an express lane for Google to find them. Internal linking is that express lane, when used correctly.
Let's say you have 3 pages on your site that are gaining authority links quickly and a 4th page that isn't indexed. Imagine for a minute that each of the 3 pages have 10 authority links pointing to them, giving Googlebot 30 opportunities to crawl a link to your site. Once on your site, you can funnel the crawler to the new page via internal linking. This strategy was originally used to "sculpt" PageRank, but it can now be used to improve the rate of indexation for certain pages.
It has long been known that Google will crawl a site more frequently if it regularly publishes new content. I mean, why would they spend their time crawling a site 3 times daily if the last update was 2 years ago?
Now, before you interpret this as "go and produce a bunch of short blog posts to get Google to my site more often", remember the end goal. You want to get QUALITY pages indexed so that they can rank and draw traffic to your site that converts. If your content won't help in reaching that goal, don't publish it.
Instead, focus on producing useful pages or posts and helping them to be discovered using the methods above. Once you get into the flow, you can ramp up production until you hit the point where quality starts to drop, at which time you should slow your pace and set your posting schedule.
Ground breaking steps? Not really, however, it is easy to forget the basics and miss out of quick wins. As Google gets better at understanding sites, some SEOs get lazy, but doing so can cost you in more ways than one.