A.C.E. Router (AI Cost Efficiency Router)

A smart, fast AI server that automatically routes your prompts to the best and cheapest AI model. By quickly checking how hard a question is, remembering past answers, and searching the web for real-time facts, this tool saves 30%+ on AI costs without losing any answer quality.

---

---

Key Features

• Smart Question Checker: Instantly checks if a question is "simple" or "complex" in under 5 milliseconds before sending it to an AI.
• Memory Cache (ChromaDB): Remembers past questions. If someone asks the exact same thing again, it gives the saved answer instantly for $0 cost.
• Live Web Search: Searches the internet for real-time facts so the AI doesn't guess or make things up.
• Hide Private Info: Automatically hides sensitive personal information (like names or emails) before the question goes to the AI.
• Background Quality Check: Checks the quality of the AI's answer in the background, so the user doesn't have to wait for it to finish.
• AI Grader: An expert AI grades the final answers out of 100 to make sure the quality stays high.
• Money-Saving Dashboard: A clear screen with charts showing how many questions were asked, how fast they were answered, and how much money you saved.
• Supports Top AI Models: Connects easily to super-fast AI (Groq) and highly intelligent AI (Mistral).

---

How It Works (Diagrams)

1. Overall System Setup

This shows the main parts of the app and how they talk to each other.

flowchart TD
    A[Dashboard UI] -->|Show Charts| B([User Request])
    B -->|Send Question| C[Main App Server]

    subgraph GW ["Step-by-Step Server Checks"]
        C --> C1[1. Hide Private Info]
        C1 --> C2[2. Check Memory for Past Answers]
        C2 --> C3[3. Search Web for Facts]
    end

    C3 -->|Quick Check| D[Difficulty Checker]
    C3 -->|Run in Background| E[Background Quality Check]
    C3 -->|Send to AI| F[AI Models: Groq / Mistral]

    E -->|Score the Answer| F
    E -->|Save Details| G[(Database)]
    F -->|Save Details| G
    G -.->|Send Data to Charts| A

Loading

2. Request Flow Lifecycle

This shows exactly what happens the moment a user asks a question.

sequenceDiagram
    autonumber
    actor User as App User
    participant Server as Main Server
    participant Memory as Memory Cache
    participant Web as Web Search
    participant Checker as Difficulty Checker
    participant AI as AI Models (Groq / Mistral)
    participant Worker as Background Task
    participant DB as Database

    User->>Server: Ask a question
    
    Note over Server: Hide Private Info

    Server->>Memory: Check for past saved answers
    
    alt Found in Memory
        Memory-->>Server: Saved Answer
        Server-->>User: Give answer instantly (Free)
    else Not Found
        Server->>Web: Search internet for facts
        
        Server->>Checker: Check how hard the question is
        Checker-->>Server: Result (Simple or Hard)
        
        Note over Server: Pick the right AI Model
        
        Server->>AI: Send question and web facts
        AI-->>Server: AI Answer + Word Count
        
        Server-->>User: Give answer to user
        
        rect rgb(255, 243, 205)
            Note over Server, DB: Start saving details in background
        end
        
        Server-)Worker: Start Quality Check
        Worker->>AI: Ask Judge AI to grade answer
        AI-->>Worker: Grade (0-100)
        Worker->>DB: Save all details to Database
    end

Loading

3. How Data gets Saved

This shows how we count the words used and save the information to draw the charts.

    flowchart TD
    A[Inbound Prompt] --> B[Hide Private Info]
    B --> C[Check Memory Cache]
    
    C --> D{Found in Memory?}

    %% Fast Path
    D -- Yes --> E[Use Saved Answer: Fast & Free]
    E --> J

    %% Miss Path
    D -- No --> F[Live Web Search]
    F --> G[Check Question Difficulty]
    G --> H[Send to AI: Groq or Mistral]
    H --> I[Get Answer & Count Tokens]

    %% Logging & Evaluation
    I --> J[Save Details]
    I -.-> K[Score Answer Quality]
    K --> J

    J --> L[(Database)]
    L --> M[Dashboard Charts]

Loading

---

Tools Used (Tech Stack)

• Main Server: FastAPI (To run the app quickly)
• Smart Brain & Memory: Scikit-learn (To check question difficulty) and ChromaDB (To remember past answers)
• Web Search: duckduckgo-search (To find real-time facts on the internet)
• AI Connections: Groq (For super-fast answers) and Mistral (For hard questions and grading answers)
• Dashboard: Streamlit (To build the charts and website)
• Database: SQLite3 (To save the history locally on your computer)

---

Project Structure

ACE-Router/
├── .streamlit/             
│   └── config.toml             # Settings for the dashboard
├── .venv/                      # Python environment
├── app/                        # Main Application Code
│   ├── core/               
│   │   ├── __init__.py
│   │   └── config.py           # Settings and keys
│   ├── db/                 
│   │   ├── __init__.py
│   │   ├── autopilot.db        # The saved database file
│   │   ├── database.py         # Code to connect to the database
│   │   └── models.py           # Database tables
│   └── schemas/            
│       ├── __init__.py
│       └── router_schema.py    # Rules for data shapes
├── data/                   
│   └── seed_prompts.json       # Example questions for testing
├── notebooks/              
│   └── train_classifier.ipynb  # Workspace for training the AI checker
├── tests/                  
│   ├── test_api.py             # Code tests for the server
│   ├── test_classifier.py      # Code tests for the checker
│   ├── test_evaluator.py       # Code tests for the grader
│   └── test_provider.py        # Code tests for the AI connection
├── .env                        # Your secret API keys
├── .env.example                # A blank template for API keys
├── .gitignore                  # Files to hide from GitHub
├── app_hf.py                   # Script for uploading to Hugging Face
├── entrypoint.py               # Main startup script
├── Makefile                    # Shortcuts for running commands
├── pyproject.toml              # Project setup
├── pyrightconfig.json          # Type checker rules
├── README.md                   # This instruction file
├── requirements.txt            # List of required Python packages

---

Installation

Prerequisites

• Python 3.10+
• uv package manager

How to Install and Run

1. Download the Code

Open your terminal and run:

git clone https://github.com/aryan-code14/ACE-Router.git
cd ACE-Router

2. Setup Python

Create a virtual environment and install the required tools:

python -m venv .venv
source .venv/bin/activate  # If you are on Windows, type: .venv\Scripts\activate
pip install -r requirements.txt

3. Add Your Secret Keys

Create a file named .env in the main folder. Open it and paste your API keys inside:

GROQ_API_KEY=gsk_your_groq_key_here
MISTRAL_API_KEY=your_mistral_key_here

---

Running the App Locally

You need to open two terminal windows.

Terminal 1: Start the Main Server

Run this command to start the backend API:

uvicorn app.main:app --reload --port 8000

You can view the API at http://localhost:8000/docs

Terminal 2: Start the Dashboard

Run this command to see the charts and chat screen:

streamlit run dashboard/app.py --server.port 8501

This automatically opens a website in your browser at http://localhost:8501

---

How It Works

First, the tool hides any private information in your question to keep you safe. Next, it checks its memory to see if it already knows the answer, which saves time and money. If it's a new question, it searches the internet for real-time facts so the AI doesn't guess or make things up. Then, it quickly decides if the question is simple or hard, and sends it to the best AI for the job. Finally, a second AI secretly grades the answer in the background to make sure you always get high-quality replies.

---

Future Improvements

• Auto-Learning: Make the tool automatically learn from the AI grader's scores so it gets even smarter at picking the best AI over time.
• Backup AI Models: Add a safety net so if one AI goes down or stops working, the tool instantly switches to a backup AI without breaking.
• Cloud Database: Move the saved charts and memory to a large cloud database so it can handle thousands of users asking questions at the exact same time.

---

Contact

Aryan Marodia - aryan-code14

Project Link: https://github.com/aryan-code14/ACE-Router