Meet REX-1: Your Realtime AI Analytics Agent System (Web Version)
Published: October 24, 2024
Built on OpenAI's Realtime API. Connection to Data Warehouse, Text-to-SQL, Statistical Analysis, Web Scrape, Charts, Python Processing and Automation.
Text and voice-input options (non-realtime) also live.
OpenAI recently released it advanced sound mode via its Realtime API. Wanted to test it out. REX-1 was the result
▸ Web version. Best on full screen.
▸ Realtime features need an API key. API Costs: ~$1/min.
▸ Non-realtime with voice input: Free to use, but some features require DB credentials. Ignore the OpenAI key popup or enter random digits to close it. Click on the Task Agent or Yahoo Finance Agent to get started. Ask the agents what they can do, and you're good to go.
REX-1 Capabilities
- Connect to Data Warehouses (AWS, Azure, MySQL)
- Text-to-SQL, Data Analysis & Transformation
- Statistical Analysis & Python Charts
- Websites Scraping
- Stock Market Technical Charts & Data
- Reporting Automation (Trackers, PDFs, Emails)
YouTube Video
'How to Build & Deploy Realtime API Analytics Assistant Agent System'. (90 mins). https://www.youtube.com/watch?v=99aD2tv8G-0
See time stamped table of contents below
Learnings and insights
- Performance is mind blowing and pricing blows your purse.
- $1 per minute or realtime , while text (with voice input ) less than $1 over hours of use.
- Prices may drop by a tenth in 6-9 months, giving early developers a first mover advantage
- WebSockets and other approaches offer near realtime alternatives
- If use case allows, Custom GPTs very cost-effective; agents can be connected easily
- Deploy with various agent setups (sequential, specialized, master, backend etc). Architecture depends on the use case.
Basic Build
Frontend: TypeScript (OpenAI quick start repo)
LLM Agents: Flowise AI. Agent API access, nested agents, code interpreter, FastAPI integration, rapid deployment, voice input. Open source with self-hosting option. FlowiseAI (YC S23) makes it incredibly easy to build and deploy complex apps quickly.
Workflow automation: Make.com.
FastAPI Custom Servers: for DW connectivity and custom python processing
Source Code JSONs, Schemas at: Tigzig
Additional
- Five-part detailed hands-on implementation guide to build AI Analytics Voice Based Assistant
YouTube Time Stamps / Table of Contents
- 00:04 - Introduction and Demo of tool capabilities
- 02:01 - Agenda for video
- 02:33 - Detailed Demo
- 17:34 - Performance and Pricing Details
- 24:37 - Architecture Overview
Frontend, APIs, backend agent setup, tool by tool overview of all tools
- 38:54 - Build Step 1: Setting up user interface
- 44:45 - Build Step 2: Setup backend LLM Agents and Tools in Flowise
- 48:18 - Database Tool
- 51:57 - Reporting Automation Tool
- 55:59 - Website Scraper Tool
- 1:00:20 - Document Update Tool
- 1:02:28 - Code Interpreter Tool
- 1:05:28 - Yahoo Finance Agent
- 1:11:04 - Google Custom Search Tool
- 1:13:40 - Build Step 3: Connecting Backend to Frontend
- 1:25:12 - Conclusion
Caveats
Prototype app. Working prototype.