Introduction

A well-structured, up to date knowledge library in Conveyor forms the backbone of accurate AI-generated responses to security questionnaires and RFPs or ad hoc questions from sales, and provides a valuable resource for your Trust Center. This guide outlines best practices for creating, organizing, and maintaining your knowledge library to ensure it is leveraged effectively to generate accurate answers in complex setups (e.g. across multiple product lines) while remaining manageable.

This guide is for the Conveyor admin, curating and managing the knowledge library, to help you get the most out of Conveyor.

Source Types

Conveyor's knowledge library can include some or all of these components:

1. Knowledge Base Q&A Pairs: Curated questions and answers
   1. These are frequently asked questions that you would like ConveyorAI to use as trusted sources. Ideally, these include 200-400 questions for which you have solid, curated answers.
2. Documents: Supporting documentation that provides additional context
   1. These are your canonical documents such as SOC2 compliance report, corporate policies, or other high quality documents that contain relevant information for incoming questionnaires
   2. You can upload the documents to Conveyor (full control on content by Conveyor admins) or leverage our Google Drive integration or APIs to point Conveyor to where the documents live (and are being maintained)
3. Past answers: Historical responses that enrich the knowledge base automatically
   1. Whenever you make an edit or answer a question ConveyorAI could not answer - ConveyorAI will automatically remember that and reuse that answer if a similar question comes up again. Never answer the same question twice!
   2. You can also upload previously answered to bootstrap Conveyor’s past answers repository
4. Private sources: Internal wikis such as Confluence or Notion, used to share information within the company (e.g. competitive battlecards, HR policies)
   1. Connect ConveyorAI to these sources to avoid replicating information and making sure ConveyorAI uses the freshest information, directly from wherever it is already maintained
5. Public external sources: Websites and pages that are maintained and shared with customers (e.g. product documentation, marketing website)
   1. Connect ConveyorAI to these if they include information that can be useful for answering incoming questions (e.g. information about your company)

Product Lines

Product lines are one of Conveyor’s most powerful ways to generate accurate, context-aware answers. In many organizations, multiple product lines are offered to customers.

When an incoming questionnaire is issued, the submitter can specify which product lines this questionnaire is relevant for, and then ConveyorAI will only use the relevant sources for that product line (or general info).

This is important in cases where the answer may be different for different products. For example: “Where is your data hosted?” would have completely different answers if one of our products was on the Cloud and one was On-Prem.

However, if you have multiple product lines but the answers are the same for all of them - then there’s no need to define separate product lines in Conveyor.

Once product lines are defined, they can be applied to knowledge sources.

A knowledge source may be labeled as relevant to one or more product lines (if it applies for more than one). If you don’t specify any products, Conveyor AI will assume this content is relevant for any product (e.g. “Company address” should not be tagged with a specific product lines).

If a questionnaire is started for product A, it will use the sources labeled as relevant for product A, and also all sources labeled as ‘any product’.

If a questionnaire is started for product A and product B (or “all products”), ConveyorAI will use sources relevant for all specified products (and ‘any product’ sources). If the answer is different between these products, ConveyorAI may generate an answer such as “For product A, the data is hosted on the AWS, and for product B it is hosted on-prem”. If the answer is the same it will simply answer “Data is hosted on AWS”.

Setting up your Knowledge Library

Define product lines

• If all answers are going to be the same regardless of the product reviewed - you can skip this step
• Define product lines for your different products. Note that the product line names you choose must be clear - they will be visible to anyone submitting a new questionnaire (to choose which products are reviewed), anyone submitting one of questions (from Slack, Conveyor app, Chrome Extension or Trust Center). These names may also appear in generated answers (as in the example above if a questionnaire is submitted for multiple product lines).
• Do not define a product line called “General” or “Company-wide” - you can use ‘any product’ on the relevant sources instead. Plus see above point.
• There is no limitation on the number of product lines defined, but only define separate product lines if the answer may be different depending on the product. Keeping the number down will help reduce maintenance overhead and avoid confusion by submitters.

❗️

Do not create a "General" product line!

Any content that is not tagged to a specific product line will always be available for retrieval (so that it can be used to answer questions about your organizational controls, or anything that is not product-specific).

You do not need - nor should you - create a "General" product line.

Add knowledge

• Documents
  • Upload documents containing useful information for answering. These are typically SOC2, CAIQ, internal policies, etc.
    These documents can also be shared with your customers (via the Trust Center). But you don’t have to - you can choose for some documents to be shared with ConveyorAI only.
• Curated Q&As (recommended number - 200-400)
  • If you have an existing list of curated Q&As (e.g. frequently asked questions), that are up to date and of high quality - you can upload them (from excel/csv) or manually add them via the UI
  • If you do not have such a list, or you have one but content quality is questionable, it is recommended not to upload it to Conveyor. Remember - ConveyorAI’s answers are only as good as the content it uses.
    If that is the case, add the other sources (documents, past questionnaires, etc.) and use Conveyor’s recommended 200 questions list (contact support). Then you can run these as a questionnaire, and review the generated answers (based on your other content). You can then answer the ones that were not answered, or make edits to turn answers to some of the common questions to your “gold standard”.
    Then, you can upgrade these answers to curated Q&As.
  • These Q&A pairs can also be shared with your customers via the Trust Center (for self-service).
• Past answers - it is recommended to upload 3-5 filled questionnaires, to bootstrap ConveyorAI’s past answers repository
  • Use recent questionnaires (answers are up to date)
  • Use “representative” questionnaires (e.g. avoid unusual questionnaires - in content or in format)
  • In case you have multiple product lines, try to have good coverage (i.e. avoid all questionnaires are for the same product and none for other products)
• Additional public and private sources
  • If you already have a well-maintained source of truth for some of the relevant knowledge - it is recommended to direct ConveyorAI to read directly from there
  • Avoid “noise” (e.g. external website that contains a blog with topics unrelated to your company or out of date information) - these may confuse ConveyorAI.
    • You can direct ConveyorAI to more specific subpages, vs. the entire website
    • Same goes for Confluence/Notion - select only the relevant pages

More details on specific source recommendations can be found at What content should I add to Conveyor?.

Additional Tips

• More is not necessarily better - make sure to connect to reliable, up to date and relevant sources (and not “everything”).
  • Specifically - if you have a big knowledge base (e.g. exported from a legacy system) - it may be better to start over, leveraging the above approach (i.e. leaning on well-maintained sources). Remember - garbage in, garbage out.
• Product lines - anything that’s not specific to a product line - should be labeled as ‘Any product’
• Freshness - point Conveyor to where sources live or use APIs to automatically update (to avoid knowledge getting stale).

Bonus: Custom Prompt to address out of scope topics

Some questionnaires include questions that you simply prefer not to answer - either unrelated to your team’s expertise (e.g. legal), or questions that company policy prevents answering (e.g. provide personal contact info of company employees).
An alternative to adding to the knowledge library curated Q&As for these, you can leverage ConveyorAI’s Custom prompt. For example, you could add to the prompt:

If the question is about legal or contractual obligations, replace the answer with “Please refer to the MSA”
If the question requires personal contact info, replace the answer with “Our company policy does not allow sharing personal information. Please contact your sales representative for more information”

This can serve as a “catch all” for certain types of questions or topics.