# Website

### What is this?

The **Website** tab under *Agent Knowledge Base* allows you to import public website content and train your AI Agent on it. You can structure the training by categories, set import preferences, and manage sync/visibility settings to keep your agent’s knowledge current and accurate.

***

### Why is it beneficial?

* **Multi-source training**: Ingest content directly from your public website(s)
* **Knowledge grouping**: Organize URLs into categories for better control
* **Real-time updates**: Retrain or sync data anytime to reflect site changes

***

### How to configure it

1. Go to [**BotPenguin**](https://app.botpenguin.com/) and log in to your account.
2. Click on **Agents** from the sidebar and select the agent you want to configure.
3. Navigate to the **Agent Knowledge Base** section and open the **Website** tab (it loads by default).

<figure><img src="/files/PTL2gzrM3Mz808YcPXmI" alt=""><figcaption></figcaption></figure>

4. Use the **Search Bar** to find specific website categories you've already added.
5. Review the table, which includes:
   * **Category**: Label grouping URLs like "chatbot-pricing" or "General"
   * **Web Pages**: Number of web URLs under each category
   * **Last Trained**: Timestamp of the last training session
   * **Visibility Toggle**: Enable/disable knowledge availability for agent responses
   * **Actions**: Edit category name, sync/retrain content, or delete the entire category
6. To add new website data:
   * Click the **Add Website** button at the top.

<figure><img src="/files/RhUY8LXpKVki1f43gWwQ" alt=""><figcaption><p>Click on Add website to add the website as data source</p></figcaption></figure>

* Enter your **Website URL** in the field provided.
* Choose an **Import Preference**:
  * *Scan Full Page*: Includes all linked subpages
  * *Scan Single Page*: Only the direct URL’s content
  * *Scan JSON Only*: Focuses only on structured JSON content
*

```
<figure><img src="/files/VtqgatE5dqPJ2JwW4lnq" alt=""><figcaption></figcaption></figure>
```

* Optionally, click **Show Advanced Options** to toggle **Train on Images as well**. This allows your agent to also learn from images on the page, though it increases time and cost.

4. Once set, click **Save** to trigger training.

***

### Need Help?

If your URLs fail to import, content doesn’t train correctly, or the agent doesn't use the expected website knowledge, reach out to [**support@botpenguin.com**](mailto:support@botpenguin.com). Our team will assist you within 48 business hours.


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://help.botpenguin.com/ai-agents/creating-ai-agents-in-botpenguin/training-your-ai-agent/website.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.