Intelligent Call Agent
An AI-powered outbound calling agent built with ElevenLabs that contacts prospective customers with personalized offers — retrieving client data in real time, presenting pre-approved credit lines, and activating services live on the call through Azure Functions.
Project Overview
The company wanted to reach prospective customers with personalized outbound calls offering available services and pre-approved credit lines. Traditional call center operations couldn’t scale this process efficiently — agents needed to look up customer information, tailor the pitch, and handle activation requests, all while maintaining a natural conversation.
This project built a digital call agent powered by ElevenLabs’ conversational AI that autonomously conducts outbound calls. At the start of each call, the agent retrieves the customer’s profile — name, available products, and pre-approved credit line — and uses this context to deliver a personalized, human-like conversation. If the customer accepts, the agent can activate the credit line in real time by calling internal services during the live call.
Architecture
High-Level Flow
Component Breakdown
- External Call Provider — Handles the telephony infrastructure for outbound calls. Configured to pass the customer code to ElevenLabs when initiating each call.
- ElevenLabs Conversational Agent — The core of the system. Configured with detailed instructions covering company information, available products, conversation rules, behavioral guidelines, and the dynamic customer context retrieved at call start.
- Azure Functions (Client Data) — Called at the beginning of each conversation to retrieve customer information from the database: name, assigned product, and pre-approved credit-line amount. Protected with token-based authentication.
- Azure Functions (Line Activation) — Called during the live conversation when a customer accepts the offer. Activates the credit line using the customer code and returns confirmation. Also protected with token authentication.
Agent Configuration
The ElevenLabs agent was configured with a comprehensive prompt covering several areas:
Knowledge Base
- Detailed information about the company and its financial products
- Terms, conditions, and eligibility criteria for each service
- Common customer questions and approved responses
Behavioral Rules
- Greeting protocol using the customer’s real name (retrieved at call start)
- Personalized pitch referencing the customer’s specific pre-approved credit line and product
- Escalation paths for objections or questions outside the agent’s scope
- Conversation closure and follow-up procedures
Tool Integration
The agent has access to two serverless functions via function calling:
| Tool | Trigger | Action |
|---|---|---|
| Get Client Info | Call start | Queries the database with the customer code and returns profile data to personalize the conversation |
| Activate Line | Customer accepts offer | Executes the credit-line activation in the internal system and returns confirmation |
Both Azure Functions are secured with token-based authentication to prevent unauthorized access.
Proof of Concept Results
Preliminary testing demonstrated that the agent:
- Successfully conducted natural, personalized conversations using real customer data
- Correctly retrieved and presented pre-approved credit lines specific to each customer
- Executed live credit-line activations during calls when customers accepted
Observed Limitation
A noticeable latency delay was present during conversations — the gap between the customer finishing a sentence and the agent responding was perceptible enough to affect the natural flow of the interaction. This latency is inherent to the round-trip between speech recognition, LLM reasoning, and speech synthesis, and was flagged as the primary area for optimization before production deployment.
Tech Stack
| Layer | Technology |
|---|---|
| Telephony | External Call Provider |
| Conversational AI | ElevenLabs |
| Serverless Tools | Azure Functions (Python) |
| Authentication | Token-based API security |
| Storage | Internal Database |
Conclusion
This proof of concept validated the feasibility of using AI voice agents for personalized outbound customer outreach. The combination of ElevenLabs for natural voice interaction and Azure Functions for real-time service execution created a system capable of not just informing customers but completing transactions during the call. The primary challenge — conversational latency — represents the current frontier of real-time voice AI and is expected to improve as the underlying models and infrastructure mature.
