# Website

> Crawl a public website into an agent's knowledge.

Source: https://docs.runbear.io/agents/knowledge/website

Last updated: 2026-08-26

Crawl and sync content from any public website so your agent can answer questions grounded in that site's content.

## How it works

You provide one or more URLs, and Runbear crawls the pages, extracts the text content, and indexes it for retrieval. The agent searches this content at query time to provide accurate answers.

No authentication is required — Website works with any publicly accessible URL.

## Setup

1. Open your agent in the Runbear dashboard and go to the **Knowledge** tab.
2. Select **Website**.
3. Enter the URLs you want to crawl.
4. Click **Save**. Runbear begins crawling and indexing the pages.

## What gets synced

- Text content from the pages at the URLs you provide
- Runbear periodically re-crawls to pick up content changes