Why does an agent stop at a single 403?
Ask an agent to do web research and it often breaks on a perfectly public page, declaring it cannot access the URL after a single 403, WAF, or CAPTCHA. The content is there; the default fetch simply stops trying the moment it meets a blocking signal.
Run automation for real and this access step causes more friction than model quality. Where a person would try a mobile URL, a feed, or a cache by hand, the agent just halts.
Escalation until a route gets through
insane-search does not assume a block is final; it raises the route step by step. It first knocks on public APIs and feeds, then tries light variants like mobile, .json, or /rss, then impersonates a TLS fingerprint, and finally drives a real headless browser, trying each until one returns clean public text.
No API key or proxy setup is needed, and tools such as curl_cffi or yt-dlp install themselves on first use. It generally reads sites that expose a public page or feed, including X, Reddit, YouTube captions, Naver, Coupang, arXiv, and GitHub.
Why the line it refuses to cross matters
What stands out more is what it deliberately refuses to do. It does not push past login walls or paywalls; it reports that authentication is required, and it never stores or transmits credentials. By design it stays a reader of public content rather than a tool to defeat authentication.
When you put AX to work in the field, the real friction is often less about model quality and more about whether you can reach the data. An approach that keeps trying public routes instead of dying at the first block, while drawing a firm line it will not cross, is worth sharing. I did not build it; I am recommending it after using it.