# Vibes Benchmark v0.1

A tool for benchmarking different AI models by comparing their responses to custom questions.

## Prerequisites

- Python 3.8 or higher
- An OpenRouter API key ([Get one here](https://openrouter.ai/))

## Setup

1. Clone the repository:
   ```bash
   git clone [repository-url]
   cd vibes-benchmark
   ```

2. Install dependencies:
   ```bash
   pip install -r requirements.txt
   ```

3. Configure environment variables:
   ```bash
   cp .env.example .env
   ```
   Then edit `.env` and add your OpenRouter API key

## Usage

1. Prepare a text file with your questions (one per line)
2. Run the application:
   ```bash
   python app.py
   ```
3. Upload your questions file through the web interface
4. Click "Run Benchmark" to start comparing model responses

## Features

- Compare responses from different AI models side by side
- Supports up to 10 questions per benchmark
- Randomly selects different models for comparison
- Real-time response generation

## Supported Models

- Claude 3 Opus
- Claude 3 Sonnet
- Gemini Pro
- Mistral Medium
- Claude 2.1
- GPT-4 Turbo
- GPT-3.5 Turbo

## License

[Your chosen license]

Run it with 
`python app.py`