Recent

Author Topic: Anubis activated  (Read 5235 times)

Marc

  • Administrator
  • Hero Member
  • *
  • Posts: 2706
Re: Anubis activated
« Reply #45 on: May 21, 2026, 11:57:50 am »
I have a question about spiders...

Obviously we do want spiders such as the ones from Google, DuckDuckGo, Bing and others to scrape the forums in order to index it and make the information available to anyone who searches for it. 

My question is: do those spiders (the ones that are good netizens) get somehow identified so they can scrape the site without Anubis getting in their way ?

According to the Anubis site they should get passed
//--
{$I stdsig.inc}
//-I still can't read someones mind
//-Bugs reported here will be forgotten. Use the bug tracker

rvk

  • Hero Member
  • *****
  • Posts: 7045
Re: Anubis activated
« Reply #46 on: May 21, 2026, 12:45:50 pm »
According to the Anubis site they should get passed
Don't you have a Google search console for devlopers? So you can see of the forum is indexed correctly by Google?

According to that link, Google doesn't seem to index the forum anymore since Anubis is active.

For example, the recent topic about portable FPC should be at the top if you search it. But it is nowhere to be found on Google.

https://www.google.com/search?q=Portable+verion+of+FPC+and+Lazarus&ie=UTF-8
and
https://www.google.com/search?q=Portable+verion+of+FPC+and+Lazarus+site%3Ahttps%3A%2F%2Fforum.lazarus.freepascal.org

(yes, there was a typo in that topic title)


dbannon

  • Hero Member
  • *****
  • Posts: 3825
    • tomboy-ng, a rewrite of the classic Tomboy
Re: Anubis activated
« Reply #47 on: May 23, 2026, 11:19:50 am »
Don't you have a Google search console for devlopers? So you can see of the forum is indexed correctly by Google?
searching for "search console for devlopers" gets me one hit this topic, on DuckDuckGo, is that a DuckWack ?

Davo
Lazarus 4, Linux (and reluctantly Win10/11, OSX Monterey)
My Project - https://github.com/tomboy-notes/tomboy-ng and my github - https://github.com/davidbannon

n7800

  • Hero Member
  • *****
  • Posts: 709
  • Lazarus IDE contributor
    • GitLab profile
Re: Anubis activated
« Reply #48 on: May 24, 2026, 01:21:22 am »
Don't you have a Google search console for devlopers? So you can see of the forum is indexed correctly by Google?
searching for "search console for devlopers" gets me one hit this topic, on DuckDuckGo, is that a DuckWack ?

https://developers.google.com/search/docs/monitor-debug/search-console-start

 

TinyPortal © 2005-2018