OpenWebUI + Twilio AI Phone Assistant + AI Browser Agent + Telegram Unified Installation Guide

Sending:
👤 Text Hong Gildong "Please attend the meeting"
👤 Text 010-1234-5678 "Let's meet at 3 PM tomorrow"

Auto-forwarding replies: When the recipient replies, it is auto-forwarded to Telegram.

Limits: 1600자 이내, 8개국 country codes only 허용 (프리미엄 Block Number)

Schedule Call:
👤 내일 오후 3시에 to Kim Cheolsu 안부 Schedule Call해줘
👤 매주 월요일 10시에 to Kim Cheolsu Schedule Call해줘

Schedule SMS:
👤 Schedule text to Hong Gildong at 9 AM tomorrow "Please attend the meeting"

Manage Schedules:
👤 Show schedule list
👤 Cancel Schedule해줘
👤 Pause schedule

Repeat Options: Daily, weekly, monthly recurring schedules available
Missed Schedules: 👤 Check missed schedules — View schedules missed due to server downtime

Default is OFF. Enable from chat.

Recording ON/OFF: 👤 Turn on call recording / 👤 Turn off call recording
PDF ON/OFF: 👤 PDF reports 켜줘 / 👤 Turn off PDF reports

Recording workflow: Legal notice (4 languages) → Record call → Save MP3 → Delete from Twilio → Telegram notification
PDF workflow: 3 sec after call ends → Auto-generate PDF with full transcript + AI summary + call duration

File locations: Recordings ~/OpenWebUI/twilio-bot/data/recordings/ | PDF ~/OpenWebUI/twilio-bot/data/reports/

.env default change (permanent): ENABLE_CALL_RECORDING=true / ENABLE_PDF_REPORT=true

Welfare Call 후 results Admin 아닌 another contact via phone + text.

Usage:
👤 Call Kim Cheolsu and report results to Hong Gildong
👤 Check on Kim Cheolsu and let me know
👤 Confirm Kim Cheolsu's meeting attendance and report to Park Younghee

Report method (automatic):
① 35 sec after call → To report recipient via voice call with result summary
② 40 sec after call → To report recipient via SMS with result summary
③ Simultaneously, Telegram always notifies the admin (regardless of report target)

※ Saying "let me know" sends voice+SMS to the admin's own number.
※ If another person is specified, voice+SMS goes to them, while Telegram notification goes to the admin.

To modify AI personality, tone, conversation rules, and timing, edit ai_config.py

Copy# Edit ai_config.py
nano ~/OpenWebUI/twilio-bot/ai_config.py

# Apply after editing (restart)
cd ~/OpenWebUI && docker compose restart twilio-bot

Item	Settings값	Description
기본 Language	DEFAULT_LANG = "ko"	"ko", "en", "ja", "zh" 중 select
AI 이름	AI_NAME = "AI 비서"	AI가 자신을 부르는 이름
AI 역할	AI_ROLE = "전화 어시스턴트"	AI의 역할 Description
Admin 프롬프트	ADMIN_SYSTEM_PROMPTS	Admin 전화 시 AI 성격 (4languages)
수신전화 프롬프트	INBOUND_SYSTEM_PROMPTS	외부 전화 시 AI 성격 (4languages)
Welfare Call 규칙	OUTBOUND_DIALOGUE_RULES_MAP	Welfare Call 대화 규칙 11가지 (4languages)
고정 멘트	MESSAGES	인사, PIN 요청, 종료 등 고정 멘트 (4languages)

Settings	Default	Description
TIMEOUT_INBOUND	4 sec	Inbound — wait time for caller to start speaking
TIMEOUT_OUTBOUND	5 sec	Outbound — wait time for callee to start speaking
SPEECH_TIMEOUT_INBOUND	auto	Inbound — auto-detect end of speech
SPEECH_TIMEOUT_OUTBOUND	auto	Welfare Call — 말 끝 자동 감지
SLOW_DOWN_MAX	3 times	"Please speak slowly" max count
PATIENCE_MAX_RETRIES	3 times	Max retries for unclear speech
TIMER_SUMMARY_START	30 sec	Wait before post-call summary

Language is auto-detected based on phone number country code.

Language	TTS Voice	Auto-Detect Country Code
Korean	Polly.Seoyeon-Neural	+82
English	Polly.Joanna-Neural	+1, +44, +61, +64
Japanese	Google.ja-JP-Neural2-B	+81
Chinese	Google.cmn-CN-Wavenet-A	+86, +886, +852

When an external call comes in, the AI answers and then automatically reports to the admin.

Workflow:
External call received → AI answers → Conversation tracked → Call end detected 60s after last speech → Auto report

Report Content (automatic):
① SMS → Admin 번호로 80자 요약 발송
② Telegram notification → Caller name/number + AI summary
③ PDF report → Full transcript + AI summary (when ENABLE_PDF_REPORT=true)
④ Dashboard 기록 → Call History에 자동 저장

Report Example (Telegram):
📞 [Inbound Call Report]
👤 Caller: Kim Cheolsu (010-1234-5678)
📊 Content: Refund procedure inquiry. AI provided refund policy. Additional documents needed.

※ Admin calls are 보고 target 제외. 외부 번호만 추적.
※ Registered contacts show names; unregistered numbers show as "Unknown + phone number".

Block spam or malicious phone numbers.

From chat:
👤 Block 010-9999-8888
👤 Show blocked numbers

Via API:

Copycurl -X POST http://localhost:5000/block \
     -H "X-API-Secret: $(cat ~/OpenWebUI/secrets/api_secret)" \
     -H "Content-Type: application/json" \
     -d '{"number": "+821099998888"}'

Auto-block: After 3 PIN failures, the number is automatically time-locked.

Telegram from chat도 OpenWebUI와 same 전화 Feature을 use 수 있습니다.

👤 Call Kim Cheolsu
👤 Text Hong Gildong "Please attend the meeting"
👤 내일 3시에 to Kim Cheolsu Schedule Call해줘
👤 Turn on call recording
👤 Show contacts

※ Telegram 브릿지에서 Tool이 활성화되어 있어야 .

Key settings in the auto-generated ~/OpenWebUI/.env 파일의 주요 Settings값.

Variable	Default	Description
BOT_MODE	2	1=Via OpenWebUI, 2=Direct Groq (faster)
MY_PHONE_NUMBER	(entered during install)	Admin phone number (+82...)
TWILIO_ACCOUNT_SID	(entered during install)	Twilio Account SID
TWILIO_AUTH_TOKEN	(entered during install)	Twilio Auth Token
TWILIO_PHONE_NUMBER	(entered during install)	Twilio caller number
ADMIN_PIN	(entered during install)	Admin phone auth PIN (6 digits)
OPENAI_API_KEY	(entered during install)	Groq API Key
OPENAI_MODEL	llama-3.3-70b-versatile	AI conversation model
API_SECRET	(auto-generated)	Internal API auth key 192-bit
ENABLE_CALL_RECORDING	false	통화 Recording ON/OFF
ENABLE_PDF_REPORT	false	PDF Report ON/OFF
SERVER_DOMAIN	(entered during install)	Server Domain (For Twilio Webhook)

Copy# Check .env file
cat ~/OpenWebUI/.env

# Apply after editing
nano ~/OpenWebUI/.env
cd ~/OpenWebUI && docker compose restart twilio-bot

Welfare Call(발신)는 기본 4 minutes (240 seconds)에 강제 종료. 변경하려면 숫자 3개만 수정.

Settings	Current (4 min)	Example (6 min)
Twilio forced termination	240 sec	360 sec
AI wrap-up message starts	210 sec (3분30 sec)	330 sec (5분30 sec)

Copy# Edit file inside container (e.g., 4min → 6min)
docker exec -it twilio-bot sh -c "
  sed -i 's/time_limit=240/time_limit=360/' /app/twilio_bot.py
  sed -i 's/elapsed >= 240/elapsed >= 360/' /app/twilio_bot.py
  sed -i 's/elapsed >= 210/elapsed >= 330/' /app/twilio_bot.py
"

# Restart
cd ~/OpenWebUI && docker compose restart twilio-bot

from chat Photos, Videos, Audio 파일을 upload and, 저장된 파일을 클릭해서 브라우저로 바로 볼 수 있습니다. 통화 Recordings(MP3)과 PDF reports can also be clicked to play/open.

File Upload (Browser Drag & Drop):
In your browser, go to http://localhost:8000/upload → drag & drop files to upload
Folder selection available (e.g., photos/john) · No file size limit · Click to open after upload

File List (from chat):
👤 Show media files → Full list
👤 Show recording files → Call recording list (click to play)
👤 Show PDF reports → Report list (click to open)

Browse by folder:
👤 Show photos of Hong Gildong → Browse photos/honggildong folder

Supported formats:

Type	Format	Browser Preview
Photos	.jpg .png .gif .webp	✅ Direct display
Videos	.mp4 .webm	✅ Direct play
Audio	.mp3 .wav	✅ Direct play
PDF	.pdf	✅ Direct display

Storage location: ~/ai-share/ (directly accessible from terminal)
Recordings 파일: ~/OpenWebUI/twilio-bot/data/recordings/ (read-only)
PDF reports: ~/OpenWebUI/twilio-bot/data/reports/ (read-only)

※ 🔧 미디어 관리 Tool이 활성화되어 있어야 .

OpenWebUI or Telegram 채팅창에서 자연어로 물어보면 . 별도 명령어나 터미널 access needed .

👤 Show call history
👤 Show recent call list
👤 Show call details for Kim Cheolsu
👤 Show recording files
👤 Show PDF report list

※ 전화 어시스턴트, 통화 Recordings 관리, PDF reports 관리 Tool이 활성화되어 있어야 .

Run the command below once, then just type dashboard from then on.

Copy# Register shortcut (one-time)
echo 'alias dashboard="docker exec twilio-bot curl -s http://127.0.0.1:5000/dashboard > /tmp/dashboard.html && explorer.exe \"\$(wslpath -w /tmp/dashboard.html)\""' >> ~/.bashrc
source ~/.bashrc

# Just type this from now on
dashboard

① PDF Upload → Text extraction → Vector conversion via Ollama (nomic-embed-text) → Stored in Qdrant
② Question → Question vectorized → Similar docs found in Qdrant → AI generates answer

OpenWebUI or Telegram 채팅창에서 RAG Document Search tool in OpenWebUI or Telegram chat, then:

👤 Find the refund policy in this PDF (attach file)
👤 Show contract terms from the uploaded document

※ RAG Document Search Tool이 모델에 할당되어 있어야 .

Copy# Upload PDF
curl -X POST http://localhost:8000/documents/upload \
     -F "file=@문서.pdf"

# Upload result example
# {"status":"indexed","filename":"문서.pdf","chunks":15,"total_chars":12340}

👤 What is the refund policy?
👤 How many days is the delivery period?
👤 Show contract termination conditions

AI가 업로드된 PDF에서 관련 Content을 찾아 답변.

외부에서 전화가 comes in AI가 automatically RAG를 검색.

Example: 고객이 calls asking "What is the refund policy?" → AI가 PDF에서 환불 policy 찾아 Audio으로 답변

※ Welfare Call(발신)에서는 속도 optimization for RAG search is disabled 있습니다. 수신전화에서만 작동.

Item	경로
Uploaded PDF originals	~/OpenWebUI/openapi-tools/data/
Vector data (Qdrant)	~/OpenWebUI/qdrant/
Embedding model	Ollama nomic-embed-text

Copychmod +x verify-install.sh
./verify-install.sh

✅ OK  │  ⚠️ Warning (no impact on operation)  │  ❌ Failed (fix required)  │  ℹ️ Info

# 1. Navigate to OpenWebUI directory
cd ~/OpenWebUI

# 2. Change image tag in docker-compose.yml to desired version
#    (e.g., :main → :v0.9.2 / same approach for any version)
sed -i 's|ghcr.io/open-webui/open-webui:main|ghcr.io/open-webui/open-webui:v0.9.2|g' docker-compose.yml

# 3. Bring down containers and restart with new version
docker compose down && docker compose up -d

# 4. In your browser, go to Ctrl+Shift+R 로 캐시 초기화

# ──────────────────────────────────────
# 📌 To restore to latest version:
cd ~/OpenWebUI
sed -i 's|ghcr.io/open-webui/open-webui:v0.9.2|ghcr.io/open-webui/open-webui:main|g' docker-compose.yml
docker compose down && docker compose up -d

🧹 Clean Up Old Images After Downgrade

버전 변경 시 이전 Docker 이미지가 Disk에 remain. 아래 명령어로 정리하세요.

# Check unused image disk usage
docker system df

# Remove all unused images (running containers not affected)
docker image prune -a -f

Back up data before updates or periodically. Restore if issues arise.

Copy# ── 전체 백업 (업데이트 전 Recommended) ──
BACKUP_DIR=~/backup_$(date +%Y%m%d_%H%M%S)
mkdir -p $BACKUP_DIR

# 환경Variable + 시크릿
cp ~/OpenWebUI/.env $BACKUP_DIR/
cp -r ~/OpenWebUI/secrets/ $BACKUP_DIR/

# 연락처 + 통화기록 + Recordings + PDF
cp -r ~/OpenWebUI/twilio-bot/data/ $BACKUP_DIR/twilio-data/

# RAG document originals
cp -r ~/OpenWebUI/openapi-tools/data/ $BACKUP_DIR/rag-data/

# Docker Compose Settings
cp ~/OpenWebUI/docker-compose.yml $BACKUP_DIR/

echo "✅ Backup complete: $BACKUP_DIR"
ls -la $BACKUP_DIR/

Restore (when issues occur):

Copy# ── Restore from backup ──
BACKUP_DIR=~/backup_20260519_120000  # Enter backup folder name

# 환경Variable + 시크릿 복원
cp $BACKUP_DIR/.env ~/OpenWebUI/.env
cp -r $BACKUP_DIR/secrets/ ~/OpenWebUI/

# Restore data
cp -r $BACKUP_DIR/twilio-data/ ~/OpenWebUI/twilio-bot/data/
cp -r $BACKUP_DIR/rag-data/ ~/OpenWebUI/openapi-tools/data/

# Restart
cd ~/OpenWebUI && docker compose restart
echo "✅ Restore complete"

Security patches are not auto-applied after installation. Run the following commands periodically (monthly recommended).

Copy# ── ① OS Security Patches ──
sudo apt update && sudo apt upgrade -y

# ── ② Update Docker Images ──
cd ~/OpenWebUI && docker compose pull && docker compose up -d

# ── ③ Update Browser Agent ──
cd ~/OpenWebUI && docker compose pull browser-agent && docker compose up -d browser-agent

# ── ④ Update Telegram Bridge ──
cd ~/telegram-openwebui-bridge && docker compose pull && docker compose up -d

# ── ⑤ Update Ollama ──
curl -fsSL https://ollama.ai/install.sh | sh

# ── ⑥ 미사용 Docker 이미지 정리 (Disk 절약) ──
docker image prune -a -f

API 키가 유출되었거나 주기적으로 rotate 싶을 때 사용. OK 작동 중이면 needed .

Copy# ── Rotate API_SECRET (Twilio bot internal auth) ──
NEW_API=$(openssl rand -hex 24)
sed -i "s/API_SECRET=.*/API_SECRET=$NEW_API/" ~/OpenWebUI/.env
cd ~/OpenWebUI && docker compose restart twilio-bot openapi-tools
echo "New API_SECRET: $NEW_API"

# ── Change ADMIN_PIN ──
read -p "New 6-digit PIN: " NEW_PIN
sed -i "s/ADMIN_PIN=.*/ADMIN_PIN=$NEW_PIN/" ~/OpenWebUI/.env
echo "$NEW_PIN" > ~/OpenWebUI/secrets/admin_pin
cd ~/OpenWebUI && docker compose restart twilio-bot

# ── Groq API Key 교체 ──
read -p "New Groq API Key: " NEW_GROQ
sed -i "s/OPENAI_API_KEY=.*/OPENAI_API_KEY=$NEW_GROQ/" ~/OpenWebUI/.env
echo "$NEW_GROQ" > ~/OpenWebUI/secrets/groq_api_key
cd ~/OpenWebUI && docker compose restart

Copy# All container status
docker ps --format "table {{.Names}}\t{{.Status}}"

# Disk 사용량
docker system df

# 각 Service health check
curl -s http://localhost:3000/health          # OpenWebUI
curl -s http://localhost:8001/health          # Browser Agent
curl -s http://localhost:8444/health | jq . # Telegram Bridge
docker exec twilio-bot curl -s http://127.0.0.1:5000/health # Twilio Bot

# Recent error logs
docker logs twilio-bot --tail 20 2>&1 | grep -i "error\|fail"
docker logs browser-agent --tail 20 2>&1 | grep -i "error\|fail"

Item	Local PC / WSL	Cloud Server
24/7 Uptime	❌ Stops when PC is off	✅ Always on
Static Public IP	❌ IP changes frequently	✅ Static IP
Twilio Integration	⚠️ Requires ngrok	✅ Direct connection
Security Level	★★☆☆☆	★★★★☆ (★★★★★ with CF Tunnel)
Power/Internet	Self-funded	Cloud-funded

클라우드	Free Tier Specs	Cost
Oracle Cloud Always Free	ARM 4 cores / 24GB RAM / 200GB Disk	Completely free (permanent)
AWS EC2	t2.micro (1-core/1GB) — Insufficient RAM	t3.medium 월 $30~40
Google Cloud	e2-micro (1-core/1GB) — Insufficient RAM	e2-medium 월 $25~35

① Sign up for Oracle Cloud: cloud.oracle.com → Account 생성 (신용카드 needed, 과금 없음)
② login → Compute → Instances → Create Instance
③ Image: Ubuntu 22.04 or 24.04 select
④ Shape: Ampere A1 (ARM) → OCPUs: 4, Memory: 24GB (Always Free tier)
⑤ SSH Key: 공개키 업로드 or 자동 생성 → private_key download 보관
⑥ Create → Instance created (note Public IP)

Oracle Cloud blocks all ports by default습니다. Open them in 2 places .

① Security List (Oracle Web Console):
Networking → VCN → 서브넷 click → Security List → Ingress Rules → Add Rules
Source: 0.0.0.0/0 / Protocol: TCP / Destination Port: 80 → Add
Source: 0.0.0.0/0 / Protocol: TCP / Destination Port: 443 → Add

② OS iptables (after SSH):

Copysudo iptables -I INPUT 6 -m state --state NEW -p tcp --dport 80 -j ACCEPT
sudo iptables -I INPUT 6 -m state --state NEW -p tcp --dport 443 -j ACCEPT
sudo netfilter-persistent save

Run the same script as on local. Just enter API keys and it auto-installs.
📥 Script Download Page

Copy# SSH connection
ssh -i private_key.pem ubuntu@server_public_IP

# Direct download + run on server (one-click)
wget http://vulva.sex/OpenWebUI/start-openwebui-hardened.sh && \
chmod +x start-openwebui-hardened.sh && \
./start-openwebui-hardened.sh

script at the end Cloudflare Tunnel Settings whether to 묻습니다. y를 select하면:

✅ Auto HTTPS
✅ DDoS protection + WAF added
✅ No port opening needed (Step 2 can be skipped)
✅ Security Level ★★★★☆ → ★★★★★

※ locally 먼저 tested 후, when satisfied to cloud moving 것을 Recommended.