Skip to main content

X-ray

Trust but verify. That's our motto (one of them, anyway). We take great pride in the quality of answers that users receive from Copilot. Of course, most of the magic happens in the training data; we're just the messengers (really really eloquent messengers).

Sometimes, it can be helpful to check Copilot's work. What this means is to peer into Copilot's mind and ask "why did you respond this way". Common situations in which you might want to do this are:

  • You identify an incorrect or suboptimal answer and want to understand why it was produced (for example, maybe Copilot was referencing incorrect source data)
  • You are especially impressed with Copilot's answer and want to understand how it produced such brilliance
  • You're curious how Copilot works under the hood

X-ray is the solution to all these problems.

X-ray tab​

The most common use of X-ray is during the debugging or testing phase, conducted in the dashboard. From the Copilot tab, you know that you can play around with Copilot on the right side of the screen. If you select the X-ray tab of the Copilot section, Copilot will explain why it's responding the way it is, for each response. How cool is that?

X-ray will include

  • Relevant source content. This means passages that were provided to Copilot as part of our answer generation process (which picks out relevant passages from which Copilot can form its answer). These come from source passages (we break down every doc into smaller passages) as well as answers. Any source content marked in green was cited by Copilot directly (implying it was the source content that informed the answer the most).
  • Other considered sources. These are passages that seemed relevant to the user's question but Copilot decided not to reference.
  • Any API endpoints that were used (including the request and response payload)
  • If Copilot was responding using instructions from a Workflow, it will explain that

Along with all of this information, Copilot also provides a little paragraph that explains its reasoning. This often includes a description of how it synthesized information from multiple sources. If a workflow was invoked, it was explain which part of the workflow was followed and why.

Auditing historical chats​

X-ray works the same way for historical chats as it does for live debugging. To audit individual Copilot chats that already happened, head over the Copilot analytics dashboard. From here, select a chat. This will make it appear from the right. At the top of the chat, you'll see a button called X-ray. Click that, and you'll see a panel with one row for each response Copilot gave during the chat.

Clicking a card will reveal the source material that Copilot had available when generating its answers.