| Field | Value |
|---|---|
| Bot name | KonvartBot |
| robots.txt token | KonvartBot |
| Info URL (in the User-Agent) | https://konvart.com/bot |
| Contact | support@konvart.com |
The full default User-Agent string KonvartBot sends is:
Mozilla/5.0 (compatible; KonvartBot/1.0; +https://konvart.com/bot; contact: support@konvart.com)
What is KonvartBot?
KonvartBot is Konvart's web crawler. When a customer runs a site audit or asks Konvart to check a page, KonvartBot visits the site to collect the pages, links, and technical signals behind the report — status codes, indexability, headings, meta, canonicals, images, internal/external links, robots.txt and sitemaps.
What it crawls
- Pages from the site's sitemap, homepage link discovery, or a URL list the customer provides
- Internal links between those pages, plus one-off checks of off-sitemap/external links
- robots.txt (fetched once per audit and cached for that run) and sitemaps
Rendering
Pages are fetched over HTTP first. When a page needs JavaScript (or a captcha/JS check is detected), the audit can re-fetch it in a headless Chrome browser — still using the same KonvartBot User-Agent.
How to recognise KonvartBot
The token site owners will search their logs for is KonvartBot. Alongside the User-Agent above, the audit crawler sends these request headers:
| Header | Value |
|---|---|
| User-Agent | KonvartBot (full string above) |
| Accept-Language | en-US,en;q=0.9 |
| Cache-Control | no-cache |
| Pragma | no-cache |
| Accept | Varies by fetch — HTML for pages, XML for sitemaps, plain text for robots.txt |
How to control KonvartBot
You control KonvartBot through your site's robots.txt file:
Allow KonvartBot (default behaviour)
User-agent: KonvartBot Allow: /
Block KonvartBot
User-agent: KonvartBot Disallow: /
Slow it down
User-agent: KonvartBot Crawl-delay: 10
- KonvartBot respects robots.txt Disallow/Allow rules and Crawl-delay (a KonvartBot group takes precedence over *).
- Default crawl pace is 30 URLs per minute per host for a site audit; customers can change that rate in audit settings.
- If a firewall/WAF blocks it, the fix is to allowlist KonvartBot — add “KonvartBot or https://konvart.com/bot” to your allowlist/robots.txt.
Questions?
Have any questions regarding Konvart, contact us at support@konvart.com.