ai-web-research
Deterministic, LLM-independent robots/sitemap-aware crawler producing LLM-ready Markdown with a resumable frontier
Stage 1 AI crawler: deterministic robots/sitemap-aware crawling with SSRF guard, Crawl4AI fetch, SHA-256 dedup
Last updated: 2026-09-13T07:00:09.895753+00:00
Evidence Chain
| Field | Value | Source | Tier | Fetched | Confidence |
|---|---|---|---|---|---|
| archived | No | https://api.github.com/repos/kakon77777-commits/ai-web-research | 1 | 2026-09-13T07:00:09.895753+00:00 | 0.99 |
| description | Stage 1 AI crawler: deterministic robots/sitemap-aware crawling with SSRF guard, Crawl4AI fetch, SHA-256 dedup | https://api.github.com/repos/kakon77777-commits/ai-web-research | 1 | 2026-09-13T07:00:09.895753+00:00 | 0.99 |
| license | NOASSERTION | https://api.github.com/repos/kakon77777-commits/ai-web-research | 1 | 2026-09-13T07:00:09.895753+00:00 | 0.99 |
| license_hint_from_doc | MIT | D:\我的研究\未來計畫區\網路爬蟲_AI爬蟲與Agent自動化搜尋技術整理_2026-07-14.md | 3 | 2026-09-13T07:00:09.895753+00:00 | 0.8 |
| open_issues | 0 | https://api.github.com/repos/kakon77777-commits/ai-web-research | 1 | 2026-09-13T07:00:09.895753+00:00 | 0.99 |
| page_title | GitHub - kakon77777-commits/ai-web-research: Stage 1 AI crawler: deterministic robots/sitemap-aware crawling with SSRF guard, Crawl4AI fetch, SHA-256 dedup | https://github.com/kakon77777-commits/ai-web-research | 2 | 2026-09-13T07:00:09.895753+00:00 | 0.9 |
| pushed_at | 2026-09-07T08:56:28Z | https://api.github.com/repos/kakon77777-commits/ai-web-research | 1 | 2026-09-13T07:00:09.895753+00:00 | 0.99 |
| stars | 0 | https://api.github.com/repos/kakon77777-commits/ai-web-research | 1 | 2026-09-13T07:00:09.895753+00:00 | 0.99 |
| topics | — | https://api.github.com/repos/kakon77777-commits/ai-web-research | 1 | 2026-09-13T07:00:09.895753+00:00 | 0.99 |