Website

Use this section to feed knowledge into your AI Agent by training it on specific URLs or entire websites. Organize, retrain, and manage data sources with full visibility and control.

What is this?

The Website tab under Agent Knowledge Base allows you to import public website content and train your AI Agent on it. You can structure the training by categories, set import preferences, and manage sync/visibility settings to keep your agent’s knowledge current and accurate.

---

Why is it beneficial?

• Multi-source training: Ingest content directly from your public website(s)

• Knowledge grouping: Organize URLs into categories for better control

• Real-time updates: Retrain or sync data anytime to reflect site changes

---

How to configure it

1. Go to BotPenguin and log in to your account.

2. Click on Agents from the sidebar and select the agent you want to configure.

3. Navigate to the Agent Knowledge Base section and open the Website tab (it loads by default).

1. Use the Search Bar to find specific website categories you've already added.

2. Review the table, which includes:

   • Category: Label grouping URLs like "chatbot-pricing" or "General"

   • Web Pages: Number of web URLs under each category

   • Last Trained: Timestamp of the last training session

   • Visibility Toggle: Enable/disable knowledge availability for agent responses

   • Actions: Edit category name, sync/retrain content, or delete the entire category

3. To add new website data:

   • Click the Add Website button at the top.

Click on Add website to add the website as data source

• Enter your Website URL in the field provided.

• Choose an Import Preference:

  • Scan Full Page: Includes all linked subpages

  • Scan Single Page: Only the direct URL’s content

  • Scan JSON Only: Focuses only on structured JSON content

• 
• Optionally, click Show Advanced Options to toggle Train on Images as well. This allows your agent to also learn from images on the page, though it increases time and cost.

1. Once set, click Save to trigger training.

---

Need Help?

If your URLs fail to import, content doesn’t train correctly, or the agent doesn't use the expected website knowledge, reach out to [email protected]. Our team will assist you within 48 business hours.