Google uses web crawlers to discover and index the web in order to be able to provide relevant results on their search engine. As many websites are constantly being updated this indexing is not a one-time occurrence but rather an ongoing process. How often Google will update its information about your website (crawl rate) will vary depending on a variety of factors, one of which is what changes were recently implemented.
Since a steady stream of fresh and relevant content is key to having a successful SEO strategy, it is important to know how this content is found and indexed by search engines, as well as how changes to the website will impact this indexing.
Infrastructure changes will temporarily lower your website’s index rate
When Google detects that a website has been drastically changed, the rate at which their crawlers gather information about the page temporarily decrease. Infrastructure changes are a “major change” that will have an impact on your index rate. The reason for this is, as put by Google’s John Mueller in a recent Q&A, is to let websites update such things without the search engine causing problems.
A website that is rolling out a major change will likely have multiple changes over the course of the implementation. Continuous indexing during this stage could give Google the wrong information about the site, which would have a negative impact on search engine rankings. Instead, website owners are given time to get their changes up and running before the changes take effect.
What classifies as an infrastructure change?
There are a variety of infrastructure components behind a website. The one discussed in the Q&A was regarding a Content Delivery Network (CDN). A content delivery network is a network of servers positioned around the world to speed up the delivery of content by serving the content from a server that is close to the user. This is deemed a significant change in website infrastructure and will result in a temporary reduction of the index rate.
Below is a list of some infrastructure components likely to be similarly treated by the crawlers:
- Content Management System (CMS)
- Website Hosting
- Web Analytics
- Web Programming
- DNS — Domain Name Servers
- Load Balancer
- Application Server and Databases
Best practices for updating your website
Knowing how Google’s crawler treats major website changes allows web developers to plan their implementations to have minimal impact on SEO performance. Timing is key when making large changes to one’s website, and if several changes can be implemented simultaneously the slowed index rate will be limited to one period. It is likewise important to avoid implementing major website changes in connection with publishing new content, as it could delay ranking on the search engines.
Finding the optimal timing for infrastructure changes within the larger digital marketing strategy is one of the aspects of web administration to consider. If you want to read more about how to preserve SEO factors through large changes, check out our website migration checklist, or contact us using the form below.
Timothy Larsson, Digital Marketing Coordinator
Came to GO MO Group in 2020, and now works with GO MO’s communication and brand. Has a deep interest in digitalization and marketing of industrial companies. MSc in Marketing and Consumption from the Gothenburg University School of Business, Economics and Law.