Recipe: Personal AI Phone Agent
Recipe: Personal AI Phone Agent
Build a personal AI phone agent that can call your phone and hold real conversations — no Twilio account, no ngrok, no server to host. Just a few API calls or natural language in any MCP client.
Inspired by Garry Tan’s voice-to-brain recipe, which requires Twilio credentials, ngrok tunnels, OpenAI Realtime API keys, and a Node.js server. AgentPhone handles all of that for you.
Option 1: Natural language via MCP (fastest)
If you have the AgentPhone MCP server configured in Claude, Cursor, or Windsurf, just say:
“Create a voice agent called Brain that greets callers warmly, asks what’s on their mind, and has a thoughtful conversation. Buy it a phone number in the 415 area code, then call me at +1XXXXXXXXXX.”
That’s it. The AI client will:
- Call
create_agentwith a hosted voice and system prompt - Call
buy_numberand attach it to the agent - Call
make_conversation_callto dial your phone
Your phone rings, and you’re talking to your AI agent.
Option 2: Python SDK (3 calls)
Option 3: cURL (no SDK needed)
Requires jq for JSON parsing. Install with brew install jq (macOS) or apt install jq (Linux).
Receiving inbound calls
Once your agent has a number, anyone can call it. For a hosted agent, inbound calls are handled automatically — the agent picks up and follows its system prompt.
To receive calls on your personal number and have them answered by your agent, forward your phone to the agent’s number (in your phone’s settings or carrier app).
Comparison: AgentPhone vs DIY
What’s next
- Customize the voice: Use
list_voicesto browse options andupdate_agentto change - Add a webhook: Get notified of calls and messages via webhooks
- View transcripts: Every call is transcribed — check call details in the dashboard or API
- SMS too: Your agent’s number can also send and receive texts
- Web calls: Let people talk to your agent from a browser — no phone needed
