I have a question about spiders...Obviously we do want spiders such as the ones from Google, DuckDuckGo, Bing and others to scrape the forums in order to index it and make the information available to anyone who searches for it. My question is: do those spiders (the ones that are good netizens) get somehow identified so they can scrape the site without Anubis getting in their way ?
According to the Anubis site they should get passed
Don't you have a Google search console for devlopers? So you can see of the forum is indexed correctly by Google?
Quote from: rvk on May 21, 2026, 12:45:50 pmDon't you have a Google search console for devlopers? So you can see of the forum is indexed correctly by Google?searching for "search console for devlopers" gets me one hit this topic, on DuckDuckGo, is that a DuckWack ?