How to orchestrate your Agents with Azure AI Agent Service and Azure API Management
Εισαγωγή
Azure AI Agent Service is a game-changer for developers. This fully managed service empowers you to build, deploy, and scale high-quality, extensible AI agents securely, without the hassle of managing underlying infrastructure. What used to take hundreds of lines of code can now be achieved in just a few lines! So here it is, a web application that streamlines document uploads, summarizes content using AI, and provides seamless access to stored summaries. This article delves into the architecture and implementation of this solution, drawing inspiration from our previous explorations with Azure AI Foundry and secure AI integrations.
Architecture Overview
Our Azure AI Agent Service WebApp integrates several Azure services to create a cohesive and scalable system:
- Azure AI Projects & Azure AI Agent Service: Powers the AI-driven summarization and title generation of uploaded documents.
- Azure Blob Storage: Stores the original and processed documents securely.
- Azure Cosmos DB: Maintains metadata and summaries for quick retrieval and display.
- Azure API Management (APIM): Manages and secures API endpoints, ensuring controlled access to backend services.
This architecture ensures a seamless flow from document upload to AI processing and storage, providing users with immediate access to summarized content.
Azure AI Agent Service – Frontend Implementation
The frontend of the Azure AI Agent Service WebApp is built using Vite και React, offering a responsive and user-friendly interface. Key features include:
- Real-time AI Chat Interface: Users can interact with an AI agent for various queries.
- Document Upload Functionality: Supports uploading documents in various formats, which are then processed by the backend AI services.
- Document Repository: Displays a list of uploaded documents with their summaries and download links.
This is the main UI , ChatApp.jsx. We can interact with Chat Agent for regular chat, while the keyword “upload:” activates the hidden upload menu.
import { useState } from 'react';
import axios from 'axios';
import { Brain } from 'lucide-react';
import { Link } from 'react-router-dom';
//import Documents from './pages/Documents';
const ChatApp = () => {
const [messages, setMessages] = useState([]);
const [input, setInput] = useState('');
const [showUpload, setShowUpload] = useState(false);
const [file, setFile] = useState(null);
const [loading, setLoading] = useState(false);
const sendMessage = async () => {
if (!input.trim()) return;
// If user types "upload:", show file input but do NOT add it to messages
if (input.startsWith('upload:')) {
setShowUpload(true);
setInput(''); // Clear input field
return;
}
// Add user message to chat
const newMessages = [...messages, { role: 'user', content: input }];
setMessages(newMessages);
setInput('');
try {
setLoading(true);
const response = await axios.post(
`${import.meta.env.VITE_API_URL}/chat`,
{ userMessage: input },
{
headers: {
'Ocp-Apim-Subscription-Key': import.meta.env.VITE_APIM_KEY,
'Content-Type': 'application/json',
},
}
);
// Append AI response
setMessages([...newMessages, { role: 'ai', content: response.data.aiResponse || "No response." }]);
} catch (error) {
console.error('Error fetching AI response:', error);
} finally {
setLoading(false);
}
};
const handleFileUpload = async () => {
if (!file) return;
setLoading(true);
try {
const formData = new FormData();
formData.append('file', file);
const response = await axios.post(
`${import.meta.env.VITE_API_URL}/upload`,
formData,
{
headers: {
'Ocp-Apim-Subscription-Key': import.meta.env.VITE_APIM_KEY,
'Content-Type': 'multipart/form-data',
},
}
);
// Ensure response contains the summary
const summary = response.data.summary || "Upload complete, but no summary available.";
setMessages([...messages, { role: 'ai', content: summary }]);
} catch (error) {
console.error('Error uploading file:', error);
setMessages([...messages, { role: 'ai', content: "File upload failed." }]);
} finally {
setLoading(false);
setShowUpload(false);
setFile(null);
}
};
return (
<div className="fixed inset-0 flex items-center justify-center bg-gray-50">
{/* Top-right menu */}
<div className="absolute top-4 right-4 flex gap-2">
<Link
to="/"
className="px-3 py-2 bg-gray-300 hover:bg-gray-400 text-gray-800 rounded"
>
Κεντρική
</Link>
<Link
to="/documents"
className="px-3 py-2 bg-gray-300 hover:bg-gray-400 text-gray-800 rounded"
>
Documents
</Link>
</div>
<div className="container mx-auto flex justify-center px-4">
<div className="w-full max-w-lg bg-white rounded-xl shadow-xl p-6">
<h1 className="text-2xl font-bold text-center mb-6 text-gray-800">
Ask your Azure AI Agent
</h1>
<div className="flex flex-col items-center space-y-4 mb-6">
<img
src="/logo.png"
alt="AI Agent Logo"
className="h-16 w-auto object-contain"
/>
<div className="bg-blue-500 rounded-full p-4 shadow-lg">
<Brain className="w-12 h-12 text-white" />
</div>
</div>
<div className="h-80 overflow-y-auto border border-gray-200 mb-4 p-2 rounded-lg bg-gray-50">
{messages.slice(-5).map((msg, index) => (
<div
key={index}
className={`p-3 my-2 rounded-lg max-w-[80%] ${
msg.role === 'user'
? 'bg-blue-100 ml-auto'
: 'bg-gray-200'
}`}
>
{msg.content}
</div>
))}
</div>
{showUpload && (
<div className="mb-4 flex items-center gap-2">
<input
type="file"
onChange={(e) => setFile(e.target.files[0])}
className="flex-grow p-2 border border-gray-300 rounded-lg"
/>
<button
onClick={handleFileUpload}
className="px-4 py-2 bg-green-500 text-white rounded-lg hover:bg-green-600 transition-colors"
>
Upload
</button>
</div>
)}
<div className="flex items-center gap-2">
<input
type="text"
className="flex-grow p-3 border border-gray-300 rounded-lg focus:outline-none focus:ring-2 focus:ring-blue-500"
value={input}
onChange={(e) => setInput(e.target.value)}
placeholder="Type a message..."
onKeyDown={(e) => e.key === 'Enter' && sendMessage()}
/>
<button
onClick={sendMessage}
className="px-6 py-3 bg-blue-500 text-white rounded-lg hover:bg-blue-600 transition-colors"
disabled={loading}
>
{loading ? 'Loading...' : 'Send'}
</button>
</div>
</div>
</div>
</div>
);
};
export default ChatApp;
The second important page is the Documents.jsx , which allows us to search through saved Document Summaries and download our files.
Azure AI Agent Service – Backend Services
The backend is developed using Express.js, orchestrating various services to handle:
- File Uploads: Accepts documents from the frontend and stores them in Azure Blob Storage.
- AI Processing: Utilizes Azure AI Projects to extract text, generate summaries, and create concise titles.
- Metadata Storage: Saves document metadata and summaries in Azure Cosmos DB for efficient retrieval.
One of the Challenges was to not recreate the Agents each time our backend reloads. So a careful plan is configured, with several files – modules for the Azure AI Agent Service interaction and Agents creation. The initialization for example is taken care by a single file-module:
const { DefaultAzureCredential } = require('@azure/identity');
const { SecretClient } = require('@azure/keyvault-secrets');
const { AIProjectsClient, ToolUtility } = require('@azure/ai-projects');
require('dotenv').config();
// Keep track of global instances
let aiProjectsClient = null;
let agents = {
chatAgent: null,
extractAgent: null,
summarizeAgent: null,
titleAgent: null
};
async function initializeAI(app) {
try {
// Setup Azure Key Vault
const keyVaultName = process.env.KEYVAULT_NAME;
const keyVaultUrl = `https://${keyVaultName}.vault.azure.net`;
const credential = new DefaultAzureCredential();
const secretClient = new SecretClient(keyVaultUrl, credential);
// Get AI connection string
const secret = await secretClient.getSecret('AIConnectionString');
const AI_CONNECTION_STRING = secret.value;
// Initialize AI Projects Client
aiProjectsClient = AIProjectsClient.fromConnectionString(
AI_CONNECTION_STRING,
credential
);
// Create code interpreter tool (shared among agents)
const codeInterpreterTool = ToolUtility.createCodeInterpreterTool();
const tools = [codeInterpreterTool.definition];
const toolResources = codeInterpreterTool.resources;
console.log('🚀 Creating AI Agents...');
// Create chat agent
agents.chatAgent = await aiProjectsClient.agents.createAgent("gpt-4o-mini", {
name: "chat-agent",
instructions: "You are a helpful AI assistant that provides clear and concise responses.",
tools,
toolResources
});
console.log('✅ Chat Agent created');
// Create extraction agent
agents.extractAgent = await aiProjectsClient.agents.createAgent("gpt-4o-mini", {
name: "extract-agent",
instructions: "Process and clean text content while maintaining structure and important information.",
tools,
toolResources
});
console.log('✅ Extract Agent created');
// Create summarization agent
agents.summarizeAgent = await aiProjectsClient.agents.createAgent("gpt-4o-mini", {
name: "summarize-agent",
instructions: "Create concise summaries that capture main points and key details.",
tools,
toolResources
});
console.log('✅ Summarize Agent created');
// Create title agent
agents.titleAgent = await aiProjectsClient.agents.createAgent("gpt-4o-mini", {
name: "title-agent",
instructions: `You are a specialized title generation assistant.
Your task is to create titles for documents following these rules:
1. Generate ONLY the title text, no additional explanations
2. Maximum length of 50 characters
3. Focus on the main topic or theme
4. Use proper capitalization (Title Case)
5. Avoid special characters and quotes
6. Make titles clear and descriptive
7. Respond with nothing but the title itself
Example good responses:
Digital Transformation Strategy 2025
Market Analysis: Premium Chai Tea
Cloud Computing Implementation Guide
Example bad responses:
"Here's a title for your document: Digital Strategy" (no explanations needed)
This document appears to be about digital transformation (just the title needed)
The title is: Market Analysis (no extra text)`,
tools,
toolResources
});
console.log('✅ Title Agent created');
// Store in app.locals
app.locals.aiProjectsClient = aiProjectsClient;
app.locals.agents = agents;
console.log('✅ All AI Agents initialized successfully');
return { aiProjectsClient, agents };
} catch (error) {
console.error('❌ Error initializing AI:', error);
throw error;
}
}
// Export both the initialization function and the shared instances
module.exports = {
initializeAI,
getClient: () => aiProjectsClient,
getAgents: () => agents
};Our backend utilizes 4 agents, creating the Azure AI Agent Service Agents and we will find them in the portal, when the Backend deploys
At the same time, each interaction is stored and managed as thread and that’s how we are interacting with the Azure AI Agent Service.
Deployment and Security of Azure AI Agent Service WebApp
Ensuring secure and efficient deployment is crucial. We’ve employed:
- Azure API Management (APIM): Secures API endpoints, providing controlled access and monitoring capabilities.
- Azure Key Vault: Manages sensitive information such as API keys and connection strings, ensuring data protection.
Every call to the backend service is protected with Azure API Management Basic Tier. We have only the required endpoints pointing to the matching Endpoints of our Azure AI Agent Service WebApp backend.
Also we are storing the AIConnectionString variable in Key Vault and we can move all Variables in Key Vault as well, which i recommend !
Get started with Azure AI Agent Service
To get started with Azure AI Agent Service, you need to create an Azure AI Foundry hub and an Agent project in your Azure subscription.
Start with the quickstart guide if it’s your first time using the service.
- You can create a AI hub and project with the required resources.
- After you create a project, you can deploy a compatible model such as GPT-4o.
- When you have a deployed model, you can also start making API calls to the service using the SDKs.
There are already 2 Quick-starts available to get your Azure AI Agent Service up and running, the Basic and the Standard.
I have chosen the second one the Standard plan, since we have a WebApp, and the whole Architecture comes very handy ! We just added the CosmosDB interaction and the API Management to extend to an enterprise setup !
Our own Azure AI Agent Service deployment, allows us to interact with the Agents, and utilize tools and functions very easy.
Κλείσιμο
By harnessing the power of Azure’s cloud services, we’ve developed a scalable and efficient web application that simplifies document management through AI-driven processing. This solution not only enhances productivity but also ensures secure and organized access to essential information.
Αναφορές:
- Azure AI Agent Service Documentation
- What is Azure AI Agent Service
- Azure AI Agent Service Quick starts
- Azure API Management
- Azure AI Foundry
- Azure AI Foundry Inference Demo
