How to Improve AI Responses Using AI Feedback Dashboard, AI Sandbox, and Indexing of FAQs

Even the best AI systems occasionally miss the mark. When this happens using Chime V5 connected through Microsoft Teams, it’s important to have a streamlined process to improve the AI's performance for future interactions.

This guide walks you through the full process—from reporting bad responses to creating new indexed FAQ documents and validating improvements.

Step 1: Encounter an AI Message That Needs Improvement

If a user receives an inaccurate or unhelpful response from the AI assistant in Teams, this marks the beginning of the improvement cycle.

Example: A user asks, “How do I submit a PTO request?” and the AI gives an outdated or unrelated answer.

Step 2: Submit AI Negative Feedback

Click the AI Feedback button associated with the response. Mark the feedback as Negative, and (optionally) provide context about why the response was unhelpful.

Step 3: Review Feedback in the Admin Portal

AI feedback is logged in the admin portal and can be reviewed under either:

• AI Feedback Dashboard

• AI Feedback Entries

These tools allow admins to track trends and prioritize which gaps to address first.

Step 4: Create a New FAQ Document

To address the knowledge gap, create a new FAQ entry that properly answers the original query.

Include:

• A clear title

• Related content that answers the query

• Relevant keywords or tags to support AI matching

Step 5: Enable Indexing and Run the Indexer

Once your FAQ is complete:

1. Go to the Indexing tab in the FAQ editor.

2. Enable the document for AI indexing.

3. Save and Publish the document.

4. Navigate to AI Index Overview.

5. Click Run Indexer Now to trigger immediate indexing.

Step 6: Test the Query in the AI Sandbox

After indexing is complete:

1. Open the Test AI Prompts sandbox.

2. Retest the original query.

3. Confirm that the new document is referenced and the AI response is accurate and complete.

Step 7: Retest in Microsoft Teams

Finally, test the same query again directly in Teams (the main user-facing pipeline). This ensures the fix is live and behaving correctly in the production environment.

This full-cycle improvement process helps ensure your AI assistant continues to learn and evolve from user feedback. By capturing failed interactions, crafting high-quality answers, and validating updates through indexing and testing, you can build a more responsive and reliable AI experience across your organization.