Copilot in Microsoft 365: Protecting Sensitive Data with Compliance

Generative AI tools like Microsoft 365 Copilot are transforming workplace productivity such as drafting emails, summarizing meetings and generating reports at lightning speed. However, this power comes with new risks like sensitive data exposure and prompt injection attacks. Microsoft Purview and its Communication Compliance solution offer a powerful set of tools to monitor, detect and remediate Risky interactions between users and Generative AI applications. In this blog, we will learn how Microsoft Purview helps organizations establish security strategies for Copilot, with real-world use cases, technical capabilities and advanced classifiers like Prompt Shields and Protected Material.

Microsoft Purview Communication Compliance: Core Capabilities

Microsoft Purview Communication Compliance policies are designed to monitor and manage internal communications across Microsoft 365 apps, including MS 365 Copilot interactions. It offers enhanced ability to:

• Detect policy violation where it can identify prompts and responses with harassing, discriminatory or threatening language.
• Monitor sensitive data by flagging unauthorized sharing of confidential or proprietary information.
• Detects profanity or inappropriate images.

Detecting Risky AI Content: How Purview’s Classifiers Protect Your Organization

To enhance detection accuracy, Microsoft Purview introduces trainable classifiers- AI models that identify specific types of risky content. Two key classifiers for generative AI offered by Microsoft Purview Communication Compliance are:

Prompt Schield

This classifier is designed to detect and prevent prompt injection attacks in generative AI systems like Microsoft 365 Copilot. It identifies attempts to manipulate AI behavior by embedding harmful or unauthorized instructions in user prompts. By catching risky prompts before they cause issues, it helps organizations maintain data security.  Prompt Shield is integrated into the “Detect Microsoft Copilot Interactions” policy template in Purview and can be customized to meet specific organizational needs.

Protected Material

Protected Material classifier designed to detect and identify AI-generated responses that contain branded, copyrighted, or proprietary content.

Risk Detection Workflow Diagram

This flowchart represents how Microsoft Purview Communication Compliance manages risks in interactions with Microsoft 365 Copilot.

How Purview flags risky Copilot interactions and routes them for review and remediation.

It begins when a user engages with Copilot, and the system tags the message as Copilot-related. Once tagged, a compliance policy is triggered to review the interaction. The next step involves classifier evaluation, where advanced models such as Prompt Shield and Protected Material analyze the content. Prompt Shield identifies prompt injection risks, while Protected Material detects intellectual property violations or sensitive content exposure.  If any risk is found, the system flags it and alerts a compliance reviewer for assessment. Finally, remediation actions are taken, which may include notifying the user, blocking the content or escalating the issue.

Step-by-Step: How to use Prompt Shield and Protected Material classifiers in Policy

Let’s walk through a typical policy integration scenario using these classifiers.

Step 1: Access the Microsoft Purview portal

1. Go to the Microsoft Purview portal
2. Select “Communication Compliance” from the left menu.

Step 2: Create a New Policy

1. Click on “Policies”  → Create policy
2. Choose the template: Detect Microsoft Copilot Interaction.

Step 3: Configure Policy Details

1. Enter policy name, select users/groups and set reviewers.
2. Set how long detected messages should be preserved

Step 4: Select Classifiers

1. In the classifier section, choose:

1. • Prompt Shield (detects risky prompts and prompt injection attacks)
   • Protected Material (detects confidential, branded or proprietary content)

Step 5: Set Scope & Locations

1. Define which users/departments the policy applies to.

Step 6: Review, Create & Validate

1. Review all settings and click “Create policy” → Validate.

Real-World Use Case: Policy Workflow

John Smith is the Compliance Manager at AlphaBank, a financial institution that uses Microsoft 365 Copilot to help employees draft emails and generate client reports. John is concerned that Copilot might accidentally include sensitive customer data or proprietary financial information in its AI-generated content.

To address this, John logs into the Microsoft Purview portal and selects Communication Compliance from the menu. He creates a new policy using the Detect Microsoft Copilot Interaction template. John names the policy “Finance AI Compliance,” selects the Finance department and assigns reviewers from his compliance team. He sets the preservation period for flagged messages to one year.

In the classifier section, John chooses Prompt Shield to detect risky prompts and prompt injection attacks, and Protected Material to identify confidential or branded content. He defines the scope to include all Finance department communications and ensures the policy covers both email and Teams chat channels.

After reviewing all settings, John creates and validates the policy. Now, whenever Copilot generates a message containing risky prompts or sensitive financial data, the system flags it for review, helping AlphaBank prevent data leaks and maintain regulatory compliance.

Learn More and Build Your Skills to Protect Your Copilot Interaction.

Learning more and building your skills to protect Copilot interactions empowers you to safeguard sensitive information and ensure compliance in your organization. By mastering these solutions, you can confidently prevent data leaks, mitigate regulatory risks and enable the safe and responsible use of AI.

Companies such as CloudThat offer hands-on courses and labs designed for security engineers and GenAI practitioners who want to strengthen their information protection strategies and capabilities. Programs such as SC-401: Information Security Administrator Associate & MS-4002: Prepare Security and Compliance to Support  have guided practice in configuring, managing and validating their GenAI security policies.

Secure Copilot Compliance

By leveraging built-in classifiers like Prompt Shield and Protected Material, organizations can confidently deploy Copilot while maintaining compliance, protecting sensitive data and preserving trust. Implementing robust guardrails for generative AI like Microsoft 365 Copilot is essential to protect sensitive data and maintain compliance in today’s digital workplace.

About CloudThat