diff --git a/README.md b/README.md index 02a12ec..472545e 100644 --- a/README.md +++ b/README.md @@ -6,7 +6,7 @@ English | [繁體中文](README.zh-TW.md) ![The top bar links to the How it works page, the switch at the top right changes the language, and the sidebar picks what runs the loop (the built-in harness or pi), the model and what runs it; on the left is the chat, with the sample files to ask about above the input box, on the right the loop diagram and the tokens of step 1. The pointer rests on the special token <|tool_call>, whose popover says what it is for and shows its probability, its rank and the top 5 candidates.](docs/screenshot-en.png) -This project is for learning: it shows how an agent loop works underneath, in a way more intuitive and visual than reading about it. Behind an agent's chat box runs a loop: step after step, the model proposes an action and the harness carries it out, until it stops. A chat app shows only the question and the answer; this page draws every step as it happens, down to each token the model writes, so the whole mechanism is laid out in front of you. Each part of the page matches a section of the Unit 01 reading of [Agent Atlas](https://cyyeh.github.io/agent-atlas-learning/) (in Traditional Chinese), the course this project belongs to, and [Experiments to try](#experiments-to-try) turns those ideas into things you can try yourself. +This project is for learning: it shows how an agent loop works underneath, in a way more intuitive and visual than reading about it. Behind an agent's chat box runs a loop: step after step, the model proposes an action and the harness carries it out, until it stops. A chat app shows only the question and the answer; this page draws every step as it happens, down to each token the model writes, so the whole mechanism is laid out in front of you. Each part of the page matches a section of the Unit 01 reading of [Agent Atlas](https://github.com/Agentic-System-Lab/agent-atlas-learning) (in Traditional Chinese), the course this project belongs to, and [Experiments to try](#experiments-to-try) turns those ideas into things you can try yourself. - Models: pick [google/gemma-4-E2B-it](https://huggingface.co/google/gemma-4-E2B-it) or [Qwen/Qwen3.5-0.8B](https://huggingface.co/Qwen/Qwen3.5-0.8B) in the sidebar; they run on your machine with Hugging Face `transformers`, or on [vLLM](https://docs.vllm.ai/): pick vLLM (Docker) under What runs the model in the sidebar and the page starts a vLLM server in Docker for you, on the GPU if there is an NVIDIA one and on the CPU otherwise, or set `VLLM_URL` to use a server of your own, on this machine or on a Linux machine with a GPU ([Running the model on a vLLM server](#running-the-model-on-a-vllm-server)) - The harness: the loop is this project's own (`agent/harnesses/simple_harness.py`), or, picked under What runs the loop in the sidebar, that of [pi](https://pi.dev), a coding agent. pi asks either model above, on transformers or vLLM, which the page serves it on 127.0.0.1, and then every token shows as with the built-in harness; or Claude, GPT or another model pi has credentials for, and then the page draws pi's steps from the events pi reports, without tokens ([pi as the harness](#pi-as-the-harness)) @@ -18,7 +18,7 @@ This project is for learning: it shows how an agent loop works underneath, in a On the left is a chat like ChatGPT's; on the right the agent loop is drawn step by step: the model reads the context and proposes an action, the harness carries it out, and the result goes back into the context, until it stops. The model runs on your own computer, or on a vLLM server that sends back each token's log probabilities, so every token's probability, the candidate tokens, the special tokens, the prefill and the generation speed are all in view (on vLLM, as much of it as the server's flags let it send). The models run the same harness, so switching models changes only their text formats, which makes them easy to compare. -[How it works](https://cyyeh.github.io/agent-loop-visualizer/en/) maps each step of one turn, from sending a question to the reply, to the code that does it, with file names, line numbers and excerpts, each a click away from those lines on GitHub. Where the two models, transformers and vLLM, or the built-in harness and pi do a step differently, the step tells each in a tab of its own, and picking one switches every tab of that kind on the page. It lives on GitHub Pages; How it works at the top of the app opens it in a new tab, in the language the app is in. +[How it works](https://agentic-system-lab.github.io/agent-loop-visualizer/en/) maps each step of one turn, from sending a question to the reply, to the code that does it, with file names, line numbers and excerpts, each a click away from those lines on GitHub. Where the two models, transformers and vLLM, or the built-in harness and pi do a step differently, the step tells each in a tab of its own, and picking one switches every tab of that kind on the page. It lives on GitHub Pages; How it works at the top of the app opens it in a new tab, in the language the app is in. ## What the page shows @@ -242,7 +242,7 @@ What the page leaves out it does not make up (`backend.TOKEN_DETAILS`). In detai - **The Model tab** names the server and its version; the specs come from `config.json` and the context limit from the server. The device, the precision and the parameter count are the server's and are not shown, and the folder card lists the weights as on the server only. - **Not identical to `generate()`**: vLLM batches requests and has kernels of its own, so greedy output need not match `transformers`' token for token, as `simple_backend.py`'s does. -[How it works](https://cyyeh.github.io/agent-loop-visualizer/en/) tells the two backends apart in tabs at every step where they differ, from what each reads in `models/` to MTP, and has a section on the `Backend` interface they share, all with their code. +[How it works](https://agentic-system-lab.github.io/agent-loop-visualizer/en/) tells the two backends apart in tabs at every step where they differ, from what each reads in `models/` to MTP, and has a section on the `Backend` interface they share, all with their code. ### vLLM in Docker, started by the page @@ -300,7 +300,7 @@ To add a model: 1. Another checkpoint of a family that is already here (a bigger Gemma 4 or Qwen3.5, say): add a line to `MODELS` in `agent/model_families/__init__.py`, and one to `REVISIONS` with the commit to download it at. Other Gemma 4 checkpoints have MTP drafters too (`google/gemma-4-*-it-assistant`); add a line to `DRAFTERS` (and the drafter's commit to `REVISIONS`) and the sidebar's MTP switch works for them. 2. A new family: write a module like `agent/model_families/qwen3_5.py` (a parser for the output plus notes on the special tokens, in both languages) and add its `Family` to `FAMILIES`. Loading a model finds it automatically by the `model_type` in the checkpoint's `config.json`. `tests/test_chat_template.py` checks a new family against the real tokenizer and chat template: every special token has a note, tool calls the template writes into the history parse back, and what the step card marks as new is exactly the tool result. -Or ask for it: on a vLLM server, picking a model of a family the page does not know offers a GitHub issue already filled in, and an issue for any other model you would like to watch is welcome too, at https://github.com/cyyeh/agent-loop-visualizer/issues. +Or ask for it: on a vLLM server, picking a model of a family the page does not know offers a GitHub issue already filled in, and an issue for any other model you would like to watch is welcome too, at https://github.com/Agentic-System-Lab/agent-loop-visualizer/issues. ### The two models' formats diff --git a/README.zh-TW.md b/README.zh-TW.md index 8c03873..938087b 100644 --- a/README.zh-TW.md +++ b/README.zh-TW.md @@ -6,7 +6,7 @@ ![上方可以切換到「運作說明」頁，右上角可以切換語言，側欄可以選用什麼跑迴圈（內建 harness 或 pi）、模型和用什麼跑它；左邊是對話，輸入框上方是可以拿來問的範例檔案，右邊是迴圈圖與第 1 步的 token。滑鼠停在 <|tool_call> 這個特殊 token 上，浮窗說明它的用途，並顯示它的機率、排名與前 5 名候選](docs/screenshot-zh.png) -這個專案是學習用的：用更直覺、更視覺化的方式，讓你看懂 agent 迴圈在底層怎麼運作。Agent 的聊天框背後跑著一個迴圈：一步接著一步，模型提出動作、harness 去執行，直到停止。一般的聊天 app 只讓你看到問題和答案；這個網頁在每一步發生的當下把它畫出來，細到模型寫下的每一個 token，整個機制攤開在你眼前。畫面上的每個部分都對應 [Agent Atlas 技術與實戰交流](https://cyyeh.github.io/agent-atlas-learning/) unit 01 導讀的一節，本專案就是它的專案之一；[可以試試看的實驗](#可以試試看的實驗)則把這些概念變成可以自己動手試的實驗。 +這個專案是學習用的：用更直覺、更視覺化的方式，讓你看懂 agent 迴圈在底層怎麼運作。Agent 的聊天框背後跑著一個迴圈：一步接著一步，模型提出動作、harness 去執行，直到停止。一般的聊天 app 只讓你看到問題和答案；這個網頁在每一步發生的當下把它畫出來，細到模型寫下的每一個 token，整個機制攤開在你眼前。畫面上的每個部分都對應 [Agent Atlas 技術與實戰交流](https://github.com/Agentic-System-Lab/agent-atlas-learning) unit 01 導讀的一節，本專案就是它的專案之一；[可以試試看的實驗](#可以試試看的實驗)則把這些概念變成可以自己動手試的實驗。 - 模型：側欄可以選 [google/gemma-4-E2B-it](https://huggingface.co/google/gemma-4-E2B-it) 或 [Qwen/Qwen3.5-0.8B](https://huggingface.co/Qwen/Qwen3.5-0.8B)，在本機用 Hugging Face `transformers` 推論，也可以用 [vLLM](https://docs.vllm.ai/) 跑：在側欄的「用什麼跑模型」選 vLLM (Docker)，網頁會替你在 Docker 裡啟動一台 vLLM 伺服器，有 NVIDIA GPU 就用 GPU，否則用 CPU；或是設定 `VLLM_URL`，連到你自己的伺服器，在這台電腦上或有 GPU 的 Linux 機器上都行（見[在 vLLM 伺服器上跑模型](#在-vllm-伺服器上跑模型)） - Harness：迴圈是這個專案自己寫的（`agent/harnesses/simple_harness.py`），也可以在側欄的「用什麼跑迴圈」選 [pi](https://pi.dev) 這個 coding agent。pi 可以問上面兩個模型，跑在 transformers 或 vLLM 上，由網頁在 127.0.0.1 上提供給它，這時每個 token 都和內建 harness 一樣看得到；也可以問 Claude、GPT 或其他它登入過的模型，這時網頁照 pi 回報的事件畫出它的每一步，看不到 token（「[pi 當 harness](#pi-當-harness)」） @@ -18,7 +18,7 @@ 左邊是像 ChatGPT 的聊天介面，右邊一步一步畫出 agent 迴圈：模型讀進脈絡、提出動作、harness 去執行、結果接回脈絡，直到停止。模型跑在你自己的電腦上，或是跑在一台會把每個 token 的 log 機率傳回來的 vLLM 伺服器上，所以每個 token 的機率、候選 token、特殊 token、prefill 和生成速度都看得到（用 vLLM 時，看得到多少要看伺服器開了哪些參數）。不同模型跑的是同一個 harness，換模型時只有它們的文字格式不同，正好可以對照。 -[運作說明](https://cyyeh.github.io/agent-loop-visualizer/zh/)把一輪對話從送出到回覆的每一步對應到程式碼，附上檔名、行號和程式片段，點一下就能在 GitHub 上看到那幾行。兩個模型、transformers 和 vLLM，或內建 harness 和 pi 做法不一樣的步驟，分成頁籤各自說明，選了一個，整頁同一類的頁籤都跟著換。它放在 GitHub Pages 上，app 上方的「運作說明」會在新分頁打開它，語言和網頁當下的語言一樣。 +[運作說明](https://agentic-system-lab.github.io/agent-loop-visualizer/zh/)把一輪對話從送出到回覆的每一步對應到程式碼，附上檔名、行號和程式片段，點一下就能在 GitHub 上看到那幾行。兩個模型、transformers 和 vLLM，或內建 harness 和 pi 做法不一樣的步驟，分成頁籤各自說明，選了一個，整頁同一類的頁籤都跟著換。它放在 GitHub Pages 上，app 上方的「運作說明」會在新分頁打開它，語言和網頁當下的語言一樣。 ## 畫面上看得到什麼 @@ -242,7 +242,7 @@ VLLM_URL=http://localhost:8000 uv run streamlit run streamlit_app.py - **模型分頁**寫出伺服器和它的版本；規格來自 `config.json`，脈絡上限來自伺服器。裝置、精度和參數量是伺服器的，這裡不顯示；模型資料夾卡片把權重列成「只在伺服器上」。 - **和 `generate()` 不一定一樣**：vLLM 會把請求合成一批跑，也有自己的 kernel，所以 greedy 的輸出不一定和 `transformers` 逐 token 相同；`simple_backend.py` 的會。 -[運作說明](https://cyyeh.github.io/agent-loop-visualizer/zh/)在兩個 backend 做法不一樣的每一步，從它們各自讀 `models/` 裡的哪些檔案到 MTP，都用頁籤分開說明，另有一段講它們共用的 `Backend` 介面，都附上程式碼。 +[運作說明](https://agentic-system-lab.github.io/agent-loop-visualizer/zh/)在兩個 backend 做法不一樣的每一步，從它們各自讀 `models/` 裡的哪些檔案到 MTP，都用頁籤分開說明，另有一段講它們共用的 `Backend` 介面，都附上程式碼。 ### 由網頁在 Docker 裡啟動 vLLM @@ -300,7 +300,7 @@ harness（`harnesses/simple_harness.py`）、解碼迴圈（`simple_backend.py` 1. 同一個家族的另一個 checkpoint（例如更大的 Gemma 4 或 Qwen3.5）：在 `agent/model_families/__init__.py` 的 `MODELS` 加一行，`REVISIONS` 也加一行，寫上要下載的 commit。Gemma 4 的其他 checkpoint 也有對應的 MTP 草稿模型（`google/gemma-4-*-it-assistant`），在 `DRAFTERS` 加一行（草稿模型的 commit 也加進 `REVISIONS`），側欄的 MTP 開關就能用。 2. 新的家族：照 `agent/model_families/qwen3_5.py` 寫一個模組（輸出的 parser 加上特殊 token 的說明，中英文都要），把它的 `Family` 加進 `FAMILIES`。載入模型時會用 checkpoint `config.json` 裡的 `model_type` 自動找到它。`tests/test_chat_template.py` 會拿真的 tokenizer 和 chat template 檢查新家族：每個特殊 token 都有說明、模板寫進歷史的工具呼叫讀得回來、step 卡片標成「新加入」的剛好是工具結果。 -也可以提出需求：在 vLLM 伺服器上選了網頁不認得的家族的模型，網頁會附上一個已經填好的 GitHub issue；其他想看看的模型，也歡迎到 https://github.com/cyyeh/agent-loop-visualizer/issues 提。 +也可以提出需求：在 vLLM 伺服器上選了網頁不認得的家族的模型，網頁會附上一個已經填好的 GitHub issue；其他想看看的模型，也歡迎到 https://github.com/Agentic-System-Lab/agent-loop-visualizer/issues 提。 ### 兩個模型的格式 diff --git a/agent/tools.py b/agent/tools.py index 29841f6..8fa312e 100644 --- a/agent/tools.py +++ b/agent/tools.py @@ -36,7 +36,7 @@ from agent.i18n import pick -WIKI_HEADERS = {"User-Agent": f"agent-loop-visualizer/0.1 (https://github.com/cyyeh/agent-atlas-learning) httpx/{httpx.__version__}"} +WIKI_HEADERS = {"User-Agent": f"agent-loop-visualizer/0.1 (https://github.com/Agentic-System-Lab/agent-atlas-learning) httpx/{httpx.__version__}"} SANDBOX_IMAGE = "agent-loop-sandbox" # docker build -t agent-loop-sandbox agent/sandbox SANDBOX_VERSION = "2" # the agent-loop-sandbox.version label in sandbox/Dockerfile; for an older image the page asks for a rebuild DOCKER_INSTALL = "https://docs.docker.com/get-started/get-docker/" diff --git a/app/guide.py b/app/guide.py index 7337d08..3573344 100644 --- a/app/guide.py +++ b/app/guide.py @@ -46,7 +46,7 @@ ROOT = Path(__file__).resolve().parent.parent # the project's folder, which the files the guide quotes are named from ASSETS = Path(__file__).parent / "assets" -REPO = "https://github.com/cyyeh/agent-loop-visualizer" +REPO = "https://github.com/Agentic-System-Lab/agent-loop-visualizer" FONTS = "https://fonts.googleapis.com/css2?family=Source+Sans+3:wght@400;600;700&display=swap" HTML_LANG = {"zh": "zh-Hant", "en": "en"} STAGES = { diff --git a/app/page.py b/app/page.py index b70f683..5dc6335 100644 --- a/app/page.py +++ b/app/page.py @@ -71,8 +71,8 @@ # the one every Docker setup (Docker Desktop, OrbStack, Colima) shares with its containers. UPLOADS = Path(os.environ.get("AGENT_LOOP_UPLOADS", Path.home() / ".cache" / "agent-loop-visualizer" / "uploads")) MAX_UPLOAD_MB = 50 -GUIDE_URL = "https://cyyeh.github.io/agent-loop-visualizer/" # built from app/guide.py; zh/ and en/ under it are the two languages -ISSUES_URL = "https://github.com/cyyeh/agent-loop-visualizer/issues" # where a model the page does not support yet can be asked for +GUIDE_URL = "https://agentic-system-lab.github.io/agent-loop-visualizer/" # built from app/guide.py; zh/ and en/ under it are the two languages +ISSUES_URL = "https://github.com/Agentic-System-Lab/agent-loop-visualizer/issues" # where a model the page does not support yet can be asked for LANG_COOKIE = "agent_loop_lang" # the language last picked on the switch POLL_SECONDS = 0.5 BOX_HEIGHT = 620 # the chat and each tab scroll in a box this tall; side by side, style.css fits them to the window diff --git a/tests/test_app.py b/tests/test_app.py index 61ad536..15c8532 100644 --- a/tests/test_app.py +++ b/tests/test_app.py @@ -525,7 +525,7 @@ def test_a_model_the_page_does_not_support_says_why_and_offers_a_github_issue_fi [button] = at.link_button url = urlsplit(button.proto.url) - assert f"{url.scheme}://{url.netloc}{url.path}" == "https://github.com/cyyeh/agent-loop-visualizer/issues/new" + assert f"{url.scheme}://{url.netloc}{url.path}" == "https://github.com/Agentic-System-Lab/agent-loop-visualizer/issues/new" query = parse_qs(url.query) [body] = query["body"] assert query["title"] == [title] and asks in body @@ -839,7 +839,7 @@ def test_guide_is_a_link_to_github_pages_in_the_page_language(at, monkeypatch, l lang_switch(at).set_value(lang).run() assert not at.exception, at.exception guide = next(p for p in pages[-2:] if p.title == title) - assert guide.external_url == f"https://cyyeh.github.io/agent-loop-visualizer/{lang}/" + assert guide.external_url == f"https://agentic-system-lab.github.io/agent-loop-visualizer/{lang}/" # ---------- the human in the loop ----------