yandex

How to block Yandex using IPTABLES or APF

Re: Yandex IP range, Yandex subnets, Block Yandex Robots

Across our server range we are finding that Yandex continues to ignore robots.txt files and crawls some sites constantly, so how do you stop such an abuse of your network resources?

If you use IPTABLES or APF (you should!) then you can block all Yandex spiders using the following IP ranges:

77.88.0.0/18 # yandex.ru
77.88.22.0/23 # yandex.ru
77.88.24.0/21 # yandex.ru
77.88.24.0/22 # yandex.ru
77.88.28.0/22 # yandex.ru
77.88.36.0/23 # yandex.ru
77.88.42.0/23 # yandex.ru
77.88.44.0/24 # yandex.ru
77.88.50.0/23 # yandex.ru
87.250.224.0/19 # yandex.ru
87.250.230.0/23 # yandex.ru
87.250.252.0/22 # yandex.ru
93.158.128.0/18 # yandex.ru
93.158.137.0/24 # yandex.ru
93.158.144.0/21 # yandex.ru
93.158.144.0/23 # yandex.ru
93.158.146.0/23 # yandex.ru
93.158.148.0/22 # yandex.ru
95.108.128.0/17 # yandex.ru
95.108.128.0/24 # yandex.ru
95.108.152.0/22 # yandex.ru
95.108.216.0/23 # yandex.ru
95.108.240.0/21 # yandex.ru
95.108.248.0/23  # yandex.ru
178.154.128.0/17 # yandex.ru
178.154.160.0/22 # yandex.ru
178.154.164.0/23 # yandex.ru
199.36.240.0/22 # yandex.ru
213.180.192.0/19 # yandex.ru
213.180.204.0/24 # yandex.ru
213.180.206.0/23 # yandex.ru
213.180.209.0/24 # yandex.ru
213.180.218.0/23 # yandex.ru
213.180.220.0/23 # yandex.ru

Simply restart APF and Yandex will no longer be a problem (until they extend their network!).

How do I block the Yandex.ru bot from crawling my site?

Re: How to stop Yandex, Blocking Yandex.RU

Yandex is the most popular search engine in Russia. Your bandwidth can go through the roof if this bot targets your website.

Unfortunately for many, the robots.txt file is ignored so blocking Yandex using the official method is not an option.

If you have a busy forum or website with hundreds of pages you may find that the Yandex Bot is starting to take up more of your site resources by indexing up to 90 pages every 15 minutes often leaving connections open or failing to close them properly.

You can easily disable the Yandex.ru Bot by placing the following in your .htaccess file:

SetEnvIfNoCase User-Agent "^Yandex*" bad_bot
Order Deny,Allow
Deny from env=bad_bot

Using this method saves you the trouble of having to find blocks of Yandex IP addresses and block them individually which would only work for a limited time.

Learn more about the Yandex bot

What is the Yandex.ru bot?

The Yandex.ru bot, also known as YandexBot, is a web crawler and indexing bot used by Yandex, a leading Russian search engine. It scans websites, indexes web pages, and provides data for Yandex’s search results.

How can I check if the Yandex.ru bot has visited my website?

You can check if the Yandex.ru bot has visited your website by examining your website’s server logs or by using tools like Yandex.Webmaster, which provides detailed information about bot activity.

Is it necessary to optimise my website for the Yandex.ru bot?

Yes, if you want your website to appear in Yandex search results, it’s advisable to optimize it for the Yandex.ru bot. This includes following Yandex’s webmaster guidelines for indexing and ranking.

Are there specific rules or guidelines for Yandex.ru bot optimisation?

Yandex provides webmaster guidelines that include recommendations for optimising your website for the Yandex.ru bot. This includes creating a sitemap, using proper meta tags, and ensuring mobile-friendliness.

How can I block or allow the Yandex.ru bot from accessing my website?

You can control bot access through your website’s robots.txt file. To allow Yandex.ru bot, include “User-agent: Yandex” in your robots.txt. To block it, use “Disallow: /” for all Yandex user-agents.