hacker_forum_search_-_surface_web
Differences
This shows you the differences between two versions of the page.
| Both sides previous revision Previous revision | |||
|
hacker_forum_search_-_surface_web [2024/07/02 14:45] kaduuwikiadmin |
hacker_forum_search_-_surface_web [2025/01/11 14:51] (current) kaduuwikiadmin |
||
|---|---|---|---|
| Line 83: | Line 83: | ||
| Hacker forums are not professional websites in terms of stability, accessibility and functionality. So it happens again and again that they are temporarily offline. Sometimes they disappear completely and then reappear under a new domain. All hacker forums that have relevant data have in common that you have to log in with a user, solve a captcha and also very often have to pass an anti-bot check (Cloudflare). These components change frequently (e.g. the type of captcha) and thus automated scripts that are tuned to a forum quickly get problems when such a change takes place. But also changes to the HTML code within the forum can lead to errors if our tools are programmed to search for data in very specific areas of the forums. Another problem is that hacker forums do not tolerate automated bots like ours. So if a forum detects our activity, they will block our account or IP address. In summary, the technology will never be 100% reliable, there will always be changes and errors in individual forums. If there are several hundred forums, these errors will naturally accumulate and in kaduu only the errors are displayed on the very first page, which gives the impression that many forums have errors. But this is not the case and we are working daily on the optimization of the crawlers. | Hacker forums are not professional websites in terms of stability, accessibility and functionality. So it happens again and again that they are temporarily offline. Sometimes they disappear completely and then reappear under a new domain. All hacker forums that have relevant data have in common that you have to log in with a user, solve a captcha and also very often have to pass an anti-bot check (Cloudflare). These components change frequently (e.g. the type of captcha) and thus automated scripts that are tuned to a forum quickly get problems when such a change takes place. But also changes to the HTML code within the forum can lead to errors if our tools are programmed to search for data in very specific areas of the forums. Another problem is that hacker forums do not tolerate automated bots like ours. So if a forum detects our activity, they will block our account or IP address. In summary, the technology will never be 100% reliable, there will always be changes and errors in individual forums. If there are several hundred forums, these errors will naturally accumulate and in kaduu only the errors are displayed on the very first page, which gives the impression that many forums have errors. But this is not the case and we are working daily on the optimization of the crawlers. | ||
| + | |||
| + | ===== Why is the number of live searches restricted? ===== | ||
| + | When performing a live search through Kaduu, we programmatically log in to various forums. This process includes solving captchas, bypassing anti-bot protections such as Cloudflare, and authenticating with our account. Once logged in, we use the forum’s search field to locate your search term. | ||
| + | |||
| + | Throughout this process, we aim to minimize the number of queries we make. Allowing clients to perform multiple simultaneous live searches could trigger bot protection mechanisms or lead forum administrators to block or ban our accounts. Since many forums require paid accounts, account bans can result in significant costs. | ||
| + | |||
| + | Unlike database queries, each live search query incurs substantial costs on our end. For example, we rely on paid residential proxies to bypass anti-bot protections. Additionally, | ||
| + | |||
| + | Due to these constraints, | ||
| + | |||
| + | |||
hacker_forum_search_-_surface_web.txt · Last modified: 2025/01/11 14:51 by kaduuwikiadmin
Page Tools
