AI-102 Study Series Exercise 15: Multi-Agent Orchestration with Semantic Kernel

Overview This exercise demonstrates how to orchestrate multiple AI agents using Azure AI Foundry and the Semantic Kernel SDK. The solution involves two agents: Incident Manager Agent – Analyzes service logs and recommends resolution actions. DevOps Assistant Agent – Executes corrective actions and updates logs. Steps & Configuration Details 1. Deploy a Model in Azure AI Foundry Open Azure AI Foundry portal (https://ai.azure.com) and sign in. Search for gpt-4o and select Use this model. Configuration Items: Azure AI Foundry Resource: A valid name. Subscription: Your Azure subscription. Resource Group: Select or create a resource group. Region: Choose any AI Services-supported location. Deployment Name: gpt-4o (default). Tokens per Minute Rate Limit: 40,000 TPM (adjusted in Models and Endpoints). 2. Clone the Repository Open Azure Portal (https://portal.azure.com). Launch Azure Cloud Shell (PowerShell environment). Clone the repository: rm -r ai-agents -f git clone https://github.com/MicrosoftLearning/mslearn-ai-agents ai-agents Navigate to the correct folder: cd ai-agents/Labfiles/05-agent-orchestration/Python Install dependencies: python -m venv labenv ./labenv/bin/Activate.ps1 pip install python-dotenv azure-identity semantic-kernel[azure] Open the configuration file: code .env Update Configuration Values: Project Endpoint (copied from Azure AI Foundry portal). Model Deployment Name (gpt-4o) Save the configuration file. 3. Implement AI Agents Open the agent code file: code agent_chat.py Add references: from azure.identity import DefaultAzureCredential from semantic_kernel.agents import AzureAIAgent, AzureAIAgentSettings, AgentGroupChat from semantic_kernel.functions import kernel_function from typing import Annotated Define the Incident Manager Agent: incident_agent_definition = await client.agents.create_agent( model=ai_agent_settings.model_deployment_name, name="Incident_Manager", instructions="Analyze service logs, identify issues, and recommend resolution actions." ) agent_incident = AzureAIAgent( client=client, definition=incident_agent_definition, plugins=[LogFilePlugin()] ) Define the DevOps Assistant Agent: devops_agent_definition = await client.agents.create_agent( model=ai_agent_settings.model_deployment_name, name="DevOps_Assistant", instructions="Execute corrective actions based on recommendations from the Incident Manager." ) agent_devops = AzureAIAgent( client=client, definition=devops_agent_definition, plugins=[DevopsPlugin()] ) 4. Implement Multi-Agent Strategies Define Selection Strategy (determines which agent responds next): class SelectionStrategy: async def select_agent(self, agents, history): if history[-1].name == "DevOps_Assistant" or history[-1].role == "User": return next(agent for agent in agents if agent.name == "Incident_Manager") return next(agent for agent in agents if agent.name == "DevOps_Assistant") Define Termination Strategy (ends conversation when resolution is complete): class ApprovalTerminationStrategy: async def should_agent_terminate(self, agent, history): return "no action needed" in history[-1].content.lower() 5. Implement Multi-Agent Chat Create a Group Chat: chat = AgentGroupChat( agents=[agent_incident, agent_devops], termination_strategy=ApprovalTerminationStrategy(agents=[agent_incident], maximum_iterations=10, automatic_reset=True), selection_strategy=SelectionStrategy(agents=[agent_incident, agent_devops]) ) Append log file data: await chat.add_chat_message(logfile_msg) Invoke the chat: async for response in chat.invoke(): if response is None or not response.name: continue print(f"{response.content}") 6. Run the AI Agent Sign into Azure: az login Run the application: python agent_chat.py Example output: INCIDENT_MANAGER > /home/.../logs/log1.log | Restart service ServiceX DEVOPS_ASSISTANT > Service ServiceX restarted successfully. INCIDENT_MANAGER > No action needed. 7. Clean Up Delete Azure resources to avoid unnecessary costs: Open Azure Portal (https://portal.azure.com). Navigate to Resource Groups. Select the resource group and click Delete. This summary captures the essential steps while highlighting all configuration items and code references required for orchestrating multiple AI agents in Azure AI Foundry.

June 7, 2025 · 3 min · Taner

AI-102 Study Series Exercise 16: Text Analysis with Azure AI Language

Overview This exercise demonstrates how to analyze text using Azure AI Language, including language detection, sentiment analysis, key phrase extraction, and entity recognition. Steps & Configuration Details 1. Provision an Azure AI Language Resource Open Azure Portal (https://portal.azure.com) and sign in. Select Create a resource → Search for Language Service → Click Create. Configuration Items: Subscription: Your Azure subscription. Resource Group: Select or create a resource group. Region: Choose any available region. Name: Enter a unique name. Pricing Tier: F0 (Free) or S (Standard). Responsible AI Notice: Agree. After provisioning, navigate to Keys and Endpoint in the Resource Management section. ...

June 7, 2025 · 3 min · Taner

AI-102 Study Series Exercise 17: Question Answering with Azure AI Language

Overview This exercise demonstrates how to create a Question Answering solution using Azure AI Language. The solution enables users to query a knowledge base of FAQs using natural language. Steps & Configuration Details 1. Provision an Azure AI Language Resource Open Azure Portal (https://portal.azure.com) and sign in. Select Create a resource → Search for Language Service → Click Create. Configuration Items: Subscription: Your Azure subscription. Resource Group: Select or create a resource group. Region: Choose any available location. Name: Enter a unique name. Pricing Tier: F0 (Free) or S (Standard). Azure Search Region: Same global region as Language resource. Azure Search Pricing Tier: Free (F) or Basic (B). Responsible AI Notice: Agree. After provisioning, navigate to Keys and Endpoint in the Resource Management section. ...

June 7, 2025 · 3 min · Taner

AI-102 Study Series Exercise 18: Conversational Language Understanding

Overview This exercise demonstrates how to create a conversational language understanding model using Azure AI Language Service. The model interprets user input, predicts intent, and identifies relevant entities. Steps & Configuration Details 1. Provision an Azure AI Language Resource Open Azure Portal (https://portal.azure.com) and sign in. Select Create a resource → Search for Language Service → Click Create. Configuration Items: Subscription: Your Azure subscription. Resource Group: Select or create a resource group. Region: Choose from: East US West US 3 North Europe South Central US Name: Enter a unique name. Pricing Tier: F0 (Free) or S (Standard). Responsible AI Notice: Agree. After provisioning, navigate to Keys and Endpoint in the Resource Management section. ...

June 7, 2025 · 3 min · Taner

AI-102 Study Series Exercise 19: Custom Text Classification with Azure AI Language

Overview This exercise demonstrates how to create a custom text classification model using Azure AI Language Service. The model is trained to categorize text into predefined classes. Steps & Configuration Details 1. Provision an Azure AI Language Resource Open Azure Portal (https://portal.azure.com) and sign in. Select Create a resource → Search for Language Service → Click Create. Configuration Items: Subscription: Your Azure subscription. Resource Group: Select or create a resource group. Region: Choose from: Australia East Central India East US North Europe South Central US Switzerland North UK South West Europe West US 2 West US 3 Name: Enter a unique name. Pricing Tier: F0 (Free) or S (Standard). Storage Account: Create a new storage account. Storage Account Type: Standard LRS. Responsible AI Notice: Agree. After provisioning, navigate to Keys and Endpoint in the Resource Management section. ...

June 7, 2025 · 4 min · Taner

AI-102 Study Series Exercise 20: Custom Entity Extraction with Azure AI Language

Overview This exercise demonstrates how to extract custom entities from text using Azure AI Language Service. The solution involves training a model to recognize specific entities in classified ads. Steps & Configuration Details 1. Provision an Azure AI Language Resource Open Azure Portal (https://portal.azure.com) and sign in. Select Create a resource → Search for Language Service → Click Create. Configuration Items: Subscription: Your Azure subscription. Resource Group: Select or create a resource group. Region: Choose from: Australia East Central India East US East US 2 North Europe South Central US Switzerland North UK South West Europe West US 2 West US 3 Name: Enter a unique name. Pricing Tier: F0 (Free) or S (Standard). Storage Account: Create a new storage account. Storage Account Type: Standard LRS. Responsible AI Notice: Agree. After provisioning, navigate to Keys and Endpoint in the Resource Management section. ...

June 7, 2025 · 3 min · Taner

AI-102 Study Series Exercise 21: Text Translation with Azure AI Translator

Overview This exercise demonstrates how to translate text using Azure AI Translator, enabling users to convert input text into different languages. Steps & Configuration Details 1. Provision an Azure AI Translator Resource Open Azure Portal (https://portal.azure.com) and sign in. Search for Azure AI services → Select Create under Translator. Configuration Items: Subscription: Your Azure subscription. Resource Group: Select or create a resource group. Region: Choose any available region. Name: Enter a unique name. Pricing Tier: F0 (Free) or S (Standard). Responsible AI Notice: Agree. After provisioning, navigate to Keys and Endpoint in the Resource Management section. ...

June 7, 2025 · 2 min · Taner

AI-102 Study Series Exercise 22: Speech Recognition and Synthesis with Azure AI Speech

Overview This exercise demonstrates how to recognize and synthesize speech using Azure AI Speech. The solution enables users to convert spoken words into text (speech-to-text) and generate audible speech from text (text-to-speech). Steps & Configuration Details 1. Provision an Azure AI Speech Resource Open Azure Portal (https://portal.azure.com) and sign in. Search for Azure AI services → Select Create under Speech service. Configuration Items: Subscription: Your Azure subscription. Resource Group: Select or create a resource group. Region: Choose any available region. Name: Enter a unique name. Pricing Tier: F0 (Free) or S (Standard). Responsible AI Notice: Agree. After provisioning, navigate to Keys and Endpoint in the Resource Management section. ...

June 7, 2025 · 3 min · Taner

AI-102 Study Series Exercise 23: Speech Translation with Azure AI Speech

Overview This exercise demonstrates how to translate speech using Azure AI Speech, enabling users to convert spoken language into translated text and synthesized speech. Steps & Configuration Details 1. Provision an Azure AI Speech Resource Open Azure Portal (https://portal.azure.com) and sign in. Search for Azure AI services → Select Create under Speech service. Configuration Items: Subscription: Your Azure subscription. Resource Group: Select or create a resource group. Region: Choose any available region. Name: Enter a unique name. Pricing Tier: F0 (Free) or S (Standard). Responsible AI Notice: Agree. After provisioning, navigate to Keys and Endpoint in the Resource Management section. ...

June 7, 2025 · 3 min · Taner

AI-102 Study Series Exercise 24: Audio-Enabled Chat with Phi-4 Multimodal Model

Overview This exercise demonstrates how to develop an audio-enabled chat application using Azure AI Foundry and the Phi-4-multimodal-instruct model. The app provides AI assistance by summarizing voice messages left by customers. Steps & Configuration Details 1. Create an Azure AI Foundry Project Open Azure AI Foundry portal (https://ai.azure.com) and sign in. Select + Create project. Configuration Items: Hub Name: A valid name. Subscription: Your Azure subscription. Resource Group: Select or create a resource group. Location: Choose from: East US East US 2 North Central US South Central US Sweden Central West US West US 3 Connect Azure AI Services: Create a new AI Services resource. Connect Azure AI Search: Skip connecting. 2. Deploy a Multimodal Model Navigate to Models + endpoints → Deploy base model. Search for Phi-4-multimodal-instruct and select it. Configuration Items: Deployment Name: A valid name. Deployment Type: Global Standard. Deployment Details: Use default settings. 3. Configure the Client Application Open Azure Portal (https://portal.azure.com). Launch Azure Cloud Shell (PowerShell environment). Clone the repository: rm -r mslearn-ai-audio -f git clone https://github.com/MicrosoftLearning/mslearn-ai-language mslearn-ai-audio Navigate to the correct folder: Python: cd mslearn-ai-audio/Labfiles/09-audio-chat/Python C#: cd mslearn-ai-audio/Labfiles/09-audio-chat/C-sharp Install dependencies: Python: python -m venv labenv ./labenv/bin/Activate.ps1 pip install -r requirements.txt azure-identity azure-ai-projects azure-ai-inference C#: dotnet add package Azure.Identity dotnet add package Azure.AI.Inference --version 1.0.0-beta.3 dotnet add package Azure.AI.Projects --version 1.0.0-beta.3 Open the configuration file: Python: .env C#: appsettings.json Update Configuration Values: Project Connection String (copied from Azure AI Foundry portal). Model Deployment Name (Phi-4-multimodal-instruct). Save the configuration file. 4. Implement the AI Chat Client Open the code file: Python: audio-chat.py C#: Program.cs Add references: Python: from dotenv import load_dotenv from azure.identity import DefaultAzureCredential from azure.ai.projects import AIProjectClient from azure.ai.inference.models import SystemMessage, UserMessage, TextContentItem C#: using Azure.Identity; using Azure.AI.Projects; using Azure.AI.Inference; Initialize the AI Foundry client: Python: project_client = AIProjectClient.from_connection_string( conn_str=project_connection, credential=DefaultAzureCredential() ) C#: var projectClient = new AIProjectClient(project_connection, new DefaultAzureCredential()); Create a chat client: Python: chat_client = project_client.inference.get_chat_completions_client(model=model_deployment) C#: ChatCompletionsClient chat = projectClient.GetChatCompletionsClient(); 5. Submit an Audio-Based Prompt Python: file_path = "https://github.com/MicrosoftLearning/mslearn-ai-language/raw/refs/heads/main/Labfiles/09-audio-chat/data/avocados.mp3" response = chat_client.complete( messages=[ SystemMessage(system_message), UserMessage([ TextContentItem(text=prompt), {"type": "audio_url", "audio_url": {"url": file_path}} ]) ] ) print(response.choices[0].message.content) C#: string audioUrl = "https://github.com/MicrosoftLearning/mslearn-ai-language/raw/refs/heads/main/Labfiles/09-audio-chat/data/avocados.mp3"; var requestOptions = new ChatCompletionsOptions() { Messages = { new ChatRequestSystemMessage(system_message), new ChatRequestUserMessage( new ChatMessageTextContentItem(prompt), new ChatMessageAudioContentItem(new Uri(audioUrl)) ) }, Model = model_deployment }; var response = chat.Complete(requestOptions); Console.WriteLine(response.Value.Content); 6. Run Your Application Python: python audio-chat.py C#: dotnet run Example prompt: Can you summarize this customer's voice message? The response should display the AI-generated summary. 7. Clean Up Delete Azure resources to avoid unnecessary costs: Open Azure Portal (https://portal.azure.com). Navigate to Resource Groups. Select the resource group and click Delete.

June 7, 2025 · 3 min · Taner