Home AI agent Training the AI agent

Training the AI agent

Last updated on Sep 02, 2025

The AI agent needs relevant information about your store and support processes to respond accurately to customers. Two methods accomplish this:

Knowledge is your information library from Shopify (products, site pages) and helpdesk (help articles, training responses, canned responses) that the AI agent accesses. Knowledge is dynamically provided based on how well information matches customer requests.

Guidance consists of custom instructions you write in natural language—like explaining processes to a new teammate. Use guidance to define AI agent behavior and responses for specific shopper scenarios. For example, guidance tells the AI agent what to say and do when someone requests a return.

Knowledge sources available to the AI agent

The AI agent accesses multiple information types to handle customer queries effectively. View all available data by navigating to AI agent → Knowledge sources.

Available knowledge sources:

  • Help articles: Help portal content from your help center

  • Canned responses: Response templates created by human agents

  • AI training responses: Saved responses from successful human-agent interactions. Manually add question-and-answer pairs through the Knowledge portal

  • Shopify products: Product catalog data from your connected Shopify store

  • Shopify website pages: Store website content and page information

AI training responses

Responses you can save from good human-agent interactions. These provide the AI agent with examples of how to handle various customer questions, improving its ability to respond effectively.

How to add an AI training response

You can find an "Add AI training response" button by hovering over any message sent by a human agent. It opens a popup where you can edit the response (automatically polished by AI) and save it to your knowledge library.

In the playground, you'll see a "Revise answer" button when you hover over any message sent by the AI agent. You can give the AI the expected correct response and save it to the training responses.

Guidance

When the AI agent picks up a ticket, it considers guidance before any other knowledge source or action. While shoppers don't see guidance directly, it significantly shapes their experience with your store. Your guidance enables the AI agent to handle shopper inquiries consistently with your brand.

Guidance provides specific instructions and actions for the AI agent when it recognizes customer requests or scenarios. Think of it as teaching the AI agent your support team's best practices—not giving it scripts to copy and paste.

Writing effective guidance

Guidance field

Use specific, customer-focused scenarios that match how customers describe their issues:

  • Good: "Customer wants to return item after 30 days"

  • Good: "Customer received wrong size shoes"

  • Good: "Customer asking about shipping delay"

  • Poor: "Returns" or "Shipping" or "Order issues"

Trigger field

Add context and variations to help the AI agent recognize the scenario. Include alternative phrases customers might use:

Example 1: "Customer received the wrong size, color, or style and wants to exchange or return. May say 'this is too small,' 'wrong color,' 'not what I ordered,' or 'doesn't match the photo.'"

Example 2: "Customer's order is taking longer than expected to arrive. May ask 'where is my order,' 'when will it ship,' 'tracking shows no updates,' or 'it's been X days.'"

Example 3: "Customer wants to cancel their order that hasn't shipped yet. May say 'cancel my order,' 'I don't want this anymore,' 'changed my mind,' or 'ordered by mistake.'"

"Instructions" field

⚠️ Critical: Write instructions for the AI agent, not message templates

The AI agent needs to know WHAT to do, not exactly what to say. It will craft appropriate responses based on your instructions and the specific context of each conversation.

❌ DON'T write templates for the AI to copy:

Send to customer: "We sincerely apologize for the inconvenience. 
Your refund will be processed within 3-5 business days."

✅ DO write clear instructions for the AI to follow:

1. Apologize for the inconvenience
2. Explain that refunds typically process within 3-5 business days
3. Confirm the refund amount
4. Add label: "refund-processing"

Why this matters: When you provide templates, the AI agent sends the exact same message regardless of context. When you provide instructions, the AI agent adapts its response to match the customer's tone, address their specific concerns, and maintain a natural conversation flow.

Structure your instructions with:

1. Conditional logic (IF/THEN statements)

IF order was placed within 30 days, THEN:
- Confirm eligibility for return
- Provide return portal link
- Explain next steps

IF order was placed over 30 days ago, THEN:
- Explain the 30-day policy
- Check if exception applies (damaged item, etc.)
- Escalate to human agent if needed

2. Clear action steps

1. Verify the customer has an order
2. Check order date against return policy
3. Request photos of damaged items (if applicable)
4. Explain available resolution options
5. Process according to policy guidelines
6. Add appropriate labels for tracking

3. Information to gather

- Ask for order number if not provided
- Request photos for damage claims
- Confirm shipping address for replacements
- Get specific item details if multiple items ordered

4. Policy context (not scripts) Instead of providing a full paragraph to copy, give the AI agent the key policy points:

Damage policy key points:
- Report within 7 days for fastest resolution
- Replacements ship in 1-2 business days
- Full refunds available as alternative
- Damaged items don't need to be returned
- We cover return shipping costs

Common guidance mistakes to avoid:

Mistake 1: Writing customer-facing messages

  • Wrong: "Tell them: 'Thank you for contacting us. We appreciate your business...'"

  • Right: "Thank the customer for their patience and express appreciation for their business"

Mistake 2: Over-scripting responses

  • Wrong: "Respond with: 'I understand your frustration...'"

  • Right: "Acknowledge the customer's frustration"

Mistake 3: Missing conditional logic

  • Wrong: "Offer free expedited shipping"

  • Right: "IF order hasn't shipped yet AND customer needs item within 5 days, THEN offer expedited shipping upgrade. IF already shipped, THEN provide tracking and expected delivery date"

Mistake 4: Vague instructions

  • Wrong: "Handle appropriately"

  • Right: "1. Verify warranty status, 2. If under warranty, initiate replacement, 3. If expired, offer repair options"

Testing and troubleshooting guidance

Test guidance using the AI agent playground where you can impersonate a customer and their order history. Ensure guidance is enabled in the Guidance settings.

Signs your guidance needs improvement:

  • AI responses sound robotic or identical across different conversations

  • The AI isn't adapting to customer tone or specific situations

  • Important context from the conversation is being ignored

  • The AI is sending overly formal or scripted-sounding messages

Remember: The AI agent is designed to have natural conversations. Your guidance should empower it to use its conversational abilities while following your business rules—not restrict it to copying and pasting pre-written messages.

Supported actions for AI agent

The AI agent can perform specific actions when instructed in your guidance. You can refer to these actions using natural language—the AI understands variations of how you describe them.

⚠️ Important: Shopify order management actions must be enabled under AI agent → Actions in your settings before the AI can perform them.

Available actions:

Order management (requires enabling in AI agent settings):

  • Get latest order information for customers

  • Add tags to Shopify orders

  • Remove items from orders

  • Change shipping addresses

  • Cancel orders

  • Hold order fulfillment

Conversation management (always available):

  • Mark conversations as resolved

  • Mute conversation notifications

  • Change conversation status (open, resolved, snoozed)

  • Set or clear conversation priority

  • Assign conversations to teams or agents

  • Escalate to human agents

  • Add private notes to conversations

  • Add labels to conversations

How to reference actions in guidance: You don't need exact wording—write naturally and the AI will understand:

  • "Cancel the customer's order" → AI cancels order

  • "Assign this to the returns team" → AI assigns to returns team

  • "Add a note about the damaged packaging" → AI adds private note

  • "Tag the order as 'rush-delivery'" → AI adds tag to Shopify order

  • "Hand this over to a human" → AI escalates to agent

Common action mistakes:

  • Wrong: "Create a discount code" (not a supported action)

  • Wrong: "Process a refund" (not a supported action)

  • Right: "Cancel the order and escalate to human agent for refund processing"

If you instruct the AI to perform an unsupported action, it will either skip that step or escalate to a human agent instead.