The Open Source A/B Testing Arena for LLMs.
View Live Demo »
Report Bug
·
Request Feature
Table of Contents
"Stop guessing which prompt is better. Duel them and let the data decide."
When developing AI Agents, small semantic changes in a prompt can lead to drastically different outputs. Tracking this in spreadsheets is messy. You need a way to blind test these outputs against each other to get clean, unbiased data.
PromptDuel is a lightweight tool designed for AI Engineers and Prompt Designers. It solves the "vibe check" problem by providing a structured, data-driven environment to evaluate LLM outputs side-by-side.
- Shadcn/UI
- Recharts
- @dnd-kit
To run this code locally and start your own duel arena, follow these steps.
- Node.js (v18+)
- npm or pnpm
-
Clone the repo
git clone https://github.com/SAGAR-TAMANG/prompt-duel-main.git
-
Install dependencies
npm install- Environment Setup
Create a
.env.localfile in the root directory and add your Supabase credentials:
NEXT_PUBLIC_SUPABASE_URL=your_supabase_url
NEXT_PUBLIC_SUPABASE_ANON_KEY=your_supabase_anon_key
- Database Migration
Run the SQL scripts provided in
supabase/migrations(or the SQL editor) to set up theduelsandvotestables. - Run the development server
npm run dev
- ⚖️ Side-by-Side Arena: A clean, split-screen interface for comparing two text outputs (supports Markdown).
- 🫣 Blind Testing Mode: Model names are hidden from voters to ensure unbiased feedback.
- 🔗 Instant Sharing: Generate public, read-only links for clients or team members to cast votes.
- 📊 Analytics Dashboard: Track vote velocity and win rates visually.
- 🔐 Secure: Row Level Security (RLS) via Supabase ensures data integrity.
- Design & Develop the Arena UI
- Implement Drag & Drop Duel Management
- Integrate Recharts for Vote Velocity
- Supabase Integration for Real-time Voting
- Elo Rating System: Calculate Elo scores for prompts across multiple duels.
- API Integration: Auto-fetch outputs from OpenAI/Anthropic directly.
- Export Data: Download vote results as CSV/JSON.
See the open issues for a full list...
Contributions are what make the open-source community such an amazing place to learn, inspire, and create. Any contributions you make are greatly appreciated.
- Fork the Project
- Create your Feature Branch (
git checkout -b feature/AmazingFeature) - Commit your Changes (
git commit -m 'Add some AmazingFeature') - Push to the Branch (
git push origin feature/AmazingFeature) - Open a Pull Request
Distributed under the MIT License. See LICENSE for more information.
Sagar Tamang - LinkedIn
Project Link: https://github.com/SAGAR-TAMANG/prompt-duel-main