Setting up Knowledge Graph web connectors

Add specific webpages and entire domains to create a Knowledge Graph that you can use with Ask Writer and custom chat agents.

Connecting websites to a Knowledge Graph

To begin, navigate to Setup > Knowledge Graph and select the (+) plus icon in the top right corner.

Give your Knowledge Graph a name and a description, so your team knows when and how they should use it. Select Add websites.

In the popup modal, you can add up to 20 websites separated by commas.

Select Include all subpages to automatically include subpages
Select Include single page(s) only to include only the exact URL listed

Select Add to create your web connector.

The web connector will only look at pages under the exact domains or directories listed in the Knowledge Graph. If you intend to search links on a particular page, you will need to extract those URLs and add them individually to the Knowledge Graph.

Limitations

There are a few types of URLs which are not supported:

Websites which are behind a paywall or an auth screen
Websites which are not indexable by Google Search
Dynamic URLs (e.g. URLs which contain search?, query=? or other parameters)
URLs which lead to files like PDFs or non-text files, like JPGs. If you'd like to add files to your Knowledge Graph, please use a data connector or the manual upload option.

Managing your Knowledge Graph web connector

Editing a URL

When you create a web connector for a Knowledge graph, Writer will display a summary of the Knowledge Graph's status, including any errors.

If you'd like to edit or remove a URL from your Knowledge Graph, select the ••• three dot icon on the right side of the URL and select Edit or Remove.

Excluding URLs

If you've added a web domain and you'd like to exclude specific pages underneath (e.g. a specific set of customer stories), select the ••• three dot icon on the right side of the URL and select Exclude a URL.

Enter the URL of the website(s) you'd like to exclude from your Knowledge Graph and select Exclude.

Once you've excluded a URL, you'll see the excluded URL highlighted, with a stricken-out cloud icon next to it. You can select the ••• three dot icon menu to the right of the excluded URL, then select Edit or Remove.

Once you've finished adding, adjusting, and excluding the full set of URLs you'd like this Knowledge Graph to work with, you're all done! You'll now be able to leverage the information on these websites within Ask Writer, or when working with custom agents in AI Studio.

Frequently asked questions

Why isn't the web connector working well for my URL?

There are a few types of URLs which are not supported:

Websites which are behind a paywall or an auth screen
Websites which are not indexable by Google Search
Dynamic URLs (e.g. URLs which contain search?, query=? or other parameters)
URLs which lead to files like PDFs or non-text files, like JPGs. If you'd like to add files to your Knowledge Graph, please use a data connector or the manual upload option.

Finally, the web connector is very literal. Writer will search for the URL exactly as it is written, with no redirects. If you enter http://www.wikipedia.org but you really wanted http://en.wikipedia.org Writer will not follow the redirect to search the second URL. Double-check that you've formatted your URL exactly the way you want it.

What types of websites can be added to a Knowledge Graph?

Knowledge Graph can support any public website which can be indexed by Google Search. Websites behind a login or paywall are not currently supported. Knowledge Graph does not support dynamic URLs, e.g. URLs containing parameters like search? or query?=.

How often are web connectors synced?

Writer will retrieve the latest data from the web pages at the moment a query is submitted.

How can you delete a web connector?

From the Knowledge Graph, locate the web connector you'd like to delete and select the ••• three dot icon on the right and select Remove.

Is there a limit to the number of websites I can add to a Knowledge Graph?

Currently, you can add up to 20 URLs (whether they are included URLs or excluded URLs). A domain only counts as one URL!

Can I add websites that aren't owned by my company?

Yes, you can add any publicly accessible, indexable domain to a Knowledge Graph.

Does customer data get used in LLM training?

No. When you make a request to Writer, we retrieve the relevant pieces of data from Knowledge Graph and send that data to our LLMs to reason and generate an answer. The indexed data in your Knowledge Graph is stored completely separately from the LLM. Customer data is never used in LLM training or fine-tuning. Learn more here.