4 минуты назад, by_ix сказал:

https://wd5.ru/zametki-razrabotchika/blokirovka-seo-botov-semrushbot-linkpadbot-blexbot/

Ну я прописал это в .htaccess

Получилось следующее:

<IfModule mod_rewrite.c>
Options -MultiViews
RewriteEngine On
RewriteBase /
RewriteCond %{REQUEST_FILENAME} !-f
RewriteRule \.(js|css|jpeg|jpg|gif|png|ico|map)(\?|$) /404error.php [L,NC]

RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule . /index.php [L]
RewriteCond %{HTTP_USER_AGENT} AhrefsBot
RewriteRule (.*) - [F,L]
RewriteCond %{HTTP_USER_AGENT} MJ12bot
RewriteRule (.*) - [F,L]
RewriteCond %{HTTP_USER_AGENT} Detectify
RewriteRule (.*) - [F,L]
RewriteCond %{HTTP_USER_AGENT} dotbot
RewriteRule (.*) - [F,L]
RewriteCond %{HTTP_USER_AGENT} Riddler
RewriteRule (.*) - [F,L]
RewriteCond %{HTTP_USER_AGENT} SemrushBot
RewriteRule (.*) - [F,L]
RewriteCond %{HTTP_USER_AGENT} LinkpadBot
RewriteRule (.*) - [F,L]
RewriteCond %{HTTP_USER_AGENT} BLEXBot
RewriteRule (.*) - [F,L]
RewriteCond %{HTTP_USER_AGENT} FlipboardProxy
RewriteRule (.*) - [F,L]
RewriteCond %{HTTP_USER_AGENT} aiHitBot
RewriteRule (.*) - [F,L]
RewriteCond %{HTTP_USER_AGENT} trovitBot
RewriteRule (.*) - [F,L]
RewriteCond %{HTTP_USER_AGENT} BUbiNG
RewriteRule (.*) - [F,L]
RewriteCond %{HTTP_USER_AGENT} MauiBot
RewriteRule (.*) - [F,L]
</IfModule>

Все правильно?

У меня nginx,  добавил в настройки сайта 

if ($http_user_agent ~ (yacybot|SputnikBot|omgili|statdom|semrush|sogou|magpie|DotBot|MegaIndex|ltx71|Crowsnest|PaperLiBot|peerindex|ia_archiver|Slurp|Aport|NING|JS-Kit|rogerbot|BLEXBot|MJ12bot|Twiceler|Baiduspider|Java|CommentReader|Yeti|discobot|BTWebClient|Tagoobot|Ezooms|igdeSpyder|AhrefsBot|Teleport|Offline|DISCo|netvampire|Copier|HTTrack|WebCopier) ) {
      return 403;
}

Давно не пополнял, т.к. сейчас использую iptables+ipset, блокирую целыми подсетями. Боты, в большинстве случаев, живут у хостеров, обычных ползателей там нет. 

Edited 10 апреля, 20222 yr by Desti

6 минут назад, Desti сказал:

if ($http_user_agent ~ (yacybot|SputnikBot|omgili|statdom|semrush|sogou|magpie|DotBot|MegaIndex|ltx71|Crowsnest|PaperLiBot|peerindex|ia_archiver|Slurp|Aport|NING|JS-Kit|rogerbot|BLEXBot|MJ12bot|Twiceler|Baiduspider|Java|CommentReader|Yeti|discobot|BTWebClient|Tagoobot|Ezooms|igdeSpyder|AhrefsBot|Teleport|Offline|DISCo|netvampire|Copier|HTTrack|WebCopier) ) {
      return 403;
}

Данный вариант мне не совсем подходит, так как в данный момент сайт на обычном WEB хостинге, где я как понимаю настроить такое нельзя.

Потому что оператор технической поддержки на просьбу разобраться с ботами (Я уже как то им писал по этому поводу), ответил мне:

Спасибо за советы. После них, строк в таблице с 200 уменьшилось до 20. Респект вам!

38 минут назад, ryancoolround сказал:

правильно?

да. все не обязательно, но на будущее пойдёт. 

Игнорировать их можно, если у вас сервер нормальный.. А какой-нить дохлый vps они могут легко положить.

@ryancoolround why don't you use Cloudflare? It's free.

4 минуты назад, BlackShot сказал:

@ryancoolround why don't you use Cloudflare? It's free.

1. I embed tweets into the site, now the tweets don't work.
2. Some of the users used a picture from a third-party photo hosting, and this photo hosting displayed an iframe instead of a picture, which could lead to blocking the site.
3. In our country, one social network became drastically banned, after which I sat all night looking for posts that mention this social network.
4. The website analysis services that I used were sending threats and calls for rallies to my email

Recently, we are facing more and more troubles that are associated with third-party resources.

So, until our governments say, "make peace, make peace and fight no more," I no longer rely on third-party services, even in small communities like mine.
From now on, we live in a world where relying on third-party services is not safe.

And one more thing... Cloudflare probably has the most moronic captcha in the world, which I hate)

29 minutes ago, ryancoolround said:

1. I embed tweets into the site, now the tweets don't work.
2. Some of the users used a picture from a third-party photo hosting, and this photo hosting displayed an iframe instead of a picture, which could lead to blocking the site.
3. In our country, one social network became drastically banned, after which I sat all night looking for posts that mention this social network.
4. The website analysis services that I used were sending threats and calls for rallies to my email

Recently, we are facing more and more troubles that are associated with third-party resources.

So, until our governments say, "make peace, make peace and fight no more," I no longer rely on third-party services, even in small communities like mine.
From now on, we live in a world where relying on third-party services is not safe.

And one more thing... Cloudflare probably has the most moronic captcha in the world, which I hate)

I'm not sure how the points you mentioned were caused by CloudFlare, but it's very tweakable. You can choose what tools and settings to use. Their anti-bot protection is awesome, you don't even need to use their captcha tool for that. If you want to use their captcha though, they have released a new version called Managed Challenge. It's pretty smart and barely shows any captchas.

В 10.04.2022 в 19:52, ryancoolround сказал:

Мечты мечты. 😁

 

Трафика на форуме то много, только не полезный он.. Боты долбанные..

В 10.04.2022 в 19:17, ryancoolround сказал:

Почему то в последние три дня эта таблица увеличивается в размерах очень быстро, уже вышла на первое место если отсортировать таблицы по размерам.

Как то можно сделать так, что бы core_sessions чистился автоматически?

жа было тако тока 2 года назад