One script, every device. No per-seat licenses. No vendor lock-in.
Live demo • Features • Get started • Documentation • Community • Release notes
VirtualPyTest captures what a device shows (HDMI, camera, screen mirroring, browser) and drives it back (IR, Bluetooth remote, ADB, Appium, Playwright). On top of that it gives you a navigation graph of your app, a test runner, 24/7 monitoring, and Grafana analytics.
It runs on Linux, Raspberry Pi, Docker, or the cloud, and targets set-top boxes, Android TV, mobile phones, web apps, and anything else you can point a capture card at. It is built to replace commercial device-testing suites that cost $50k+ per year.
#1
 #26 |
 #25 |
 #24 |
All videos (26)
Device
- Device view showing from 1 to 48 devices (#24): Grid, side by side, 2×2 or one stream: the layout that fits
- Fly a drone from the Device page, with live video (#26): Live view, remote, take-off and landing of a Tello from the Device page
- Group control: one remote drives two phones at once (#25): One remote drives several devices at once
- Now compatible: Nintendo Switch and Fire TV Vega OS (#23): Now compatible: a Switch game console and Fire TV on Vega OS
AI
- AI Test Prompting: natural language to a test, replay without AI (#16): Natural language to a test, save, replay without AI
- Ask Atlas to generate a Grafana dashboard (#18): One sentence to Atlas, a Grafana dashboard filtered to your device
- Ask the AI, it acts: driving real devices from a chat with VirtualPyTest (#4): Answers from the docs, then runs a script on a device from the chat
- Ask your test data in plain words: AI charts (#13): Plain questions to Atlas, real charts from your test runs
- Claude Code drives real devices over MCP (#19): Claude Code drives a real device through VirtualPyTest's MCP server
- Real devices driven by Agentic AI: 2048 on mobile and web at once (#10): 2048 played by the same AI on a phone and in a browser, at the same time
Test
- Automatic Subtitle Analysis: compare displayed subtitles against the source (#21): Subtitles a device displays, checked against the source
- Automatic zap detection: zap with the remote, no code to write (#7): Zap with the remote: the backend detects and times it, no code to write
- Every navigation timed, automatically: KPI measurement in VirtualPyTest (#5): A KPI on every run, measured from the frames, with its report
- Localize your screen with Fingerprint: no more image or text references (#22): Which screen is it on? One click, no references to maintain
- Map an Android app automatically on a real phone, then test every path (#15): Every screen of an Android app found and tested on a real phone
- Map your app once, test it everywhere: navigation trees in VirtualPyTest (#2): Screens as nodes, actions as edges, one goto that drives a real browser
- Proof, not a checkmark: what every VirtualPyTest run leaves behind (#3): Every step with its screenshot, verification and evidence
- VirtualPyTest in 60 seconds: automated testing for TV, STB, mobile and web (#1): Dashboard, devices, heatmap, a test run, its report and the KPI it measured.
- What's new in VirtualPyTest: new look, 3 languages, shareable reports, analytics (#20): A new look, EN/FR/DE, shareable reports, analytics
- Why did it fail? Reference, screen and diff, side by side (#6): Reference, screen and pixel diff, side by side, from a failed step
Scripts
- Create and run a test campaign in seconds, no scripting (#14): Pick the tests, save, run: one report for the whole campaign
- QuickTest Builder: no code, just steps. Press CH+, check the picture moves (#9): No code, just steps: press CH+, check the picture moves, run
- Write a test script in the browser and run it on a real device (#17): Paste a test, save it, run it on a real device
Stream
- 24 video streams monitored 24/7 on one small host (#12): HLS, RTSP or file streams monitored 24/7 before they reach a device
- Audio Video Quality, 24/7, no scripts (#11): A/V quality metrics collected for you 24/7, no scripts
- Automatic incident detection + 24 h review buffer on every device (#8): Freezes caught on their own, and a 24 h review buffer on every device, no script
All videos, with new ones as they come, are on the Videos page.
 Map your app once, every test reuses it. Every screen and path becomes a reusable graph. Change a screen once and every test follows. |
 No-code for the team, Python for the engineers. Visual drag-and-drop builder for everyone. Full Python the moment you need real logic. |
 One script, every platform. The navigation graph keeps the UI out of the script. Named variants override only what differs per model. |
 It proves what is actually on screen. Image matching, OCR text, and AI detectors. Every check shows source, reference, and pixel diff. |
 Black screen, freeze, audio loss, caught in real time. Every device watched around the clock. Incidents alerted the moment they happen and tracked to resolution. |
 Talk to your lab. The AI drives the device. A real agent on a real model, with a built-in MCP server so any AI client can inspect and control devices. |
Show 10 more features (references, A/V quality, reports, Grafana, heatmap and rewind, fleet, device info, permissions, web UI, health)
 Reference images and text, easy to maintain. One library for every reference. Auto OCR and focus detection, one-click recapture when the UI changes. |
 Audio and video quality, measured every minute. Per-minute A/V quality KPIs per device, with trends per platform and software version. |
 Every run, a report you can read. Pass/fail summary with initial and final state, video, and KPI and zapping times captured automatically. |
 Every result, in native Grafana dashboards. Pass rates, duration, and volume per script. Per-step KPI radars across versions. Standard Grafana, yours to extend. |
 A fleet heatmap and 24h of screen to scrub back. One glance shows where the fleet hurts. Every device's screen is recorded, so you can see what happened at 3 a.m. |
 Your whole fleet, status and live screen. Hosts, devices, and system metrics on one dashboard. Live screen of every device, restart services from the browser. |
 Device details, read straight off the screen. Model, firmware, and version auto-extracted from the device's own About screen, in any language. |
 Permissions, down to the last action. Scoped per workspace, team, and user. Grant or deny every single action. |
 Runs in any browser, desktop to phone. Nothing to install. Fully responsive, so you can operate the whole lab from anywhere. |
 Every device accounted for, every host healthy. Device occupancy (busy vs idle, manual vs script) and host CPU, memory, disk, and temperature, 24/7. |
Full feature tour with larger screenshots: virtualpytest.angelstreet.io/docs/features
git clone https://github.com/AngelStreetCorp/virtualpytest.git
cd virtualpytest
./setup/docker/install_docker.sh # only if Docker is not installed yet
./setup/docker/launch.sh # full stack: database, server, host, web UI, Grafanalaunch.sh generates every secret, builds the images and prints the URLs when the API answers —
the web UI is on http://<this-machine>:5073. Every install path (Docker, one VM, Proxmox
fleet, developer setup) starts from the Get Started guide.
For Docker there are two supported shapes: use the command above for the complete Compose stack,
or run ./setup/docker/launch.sh --host-only when this machine only owns devices and should join
a server elsewhere. The registry publishes separate virtualpytest-server, virtualpytest-host,
and virtualpytest-frontend images; Compose pulls them together rather than providing one
monolithic image. See Install with Docker and
Add a host.
| VirtualPyTest | Commercial tools | |
|---|---|---|
| Cost | Free, open source | $50k+ / year |
| Runs on | Linux, Raspberry Pi, Docker, cloud | Often Windows only |
| Source code | Yours to read and modify | Vendor locked |
| Monitoring, analytics, AI | Included | Paid add-ons |
Stb-tester gives you Python. Witbe gives you no-code. VirtualPyTest gives you both, and AI can drive all of it.
| VirtualPyTest | Stb-tester | Witbe | |
|---|---|---|---|
| Free | ✅ | ❌ | ❌ |
| Open source | ✅ | ✅ | ❌ |
| Python scripting | ✅ | ✅ | ❌ |
| No-code builder | ✅ | ❌ | ✅ |
| AI drives the whole platform through MCP | ✅ | ❌ | ❌ |
| Hosts on any hardware, Raspberry Pi included | ✅ | ❌ | ❌ |
| Integration: Grafana, TestRail, Postman, Langfuse,slack | ✅ | ❌ | ❌ |
| Farms SauceLabs/BrowserStack and emulators | ✅ | ❌ | ❌ |
| **Fine-grained permissions and Workspace ** | ✅ | ❌ | ❌ |
| Heatmap across devices | ✅ | ❌ | ❌ |
| Automatic KPI measurement | ✅ | ❌ | ❌ |
| Automatic incident detection | ✅ | ❌ | ❌ |
| Requirements and test coverage | ✅ | ❌ | ❌ |
| Ask AI assistant | ✅ | ❌ | ❌ |
| 24h review buffer on every device | ✅ | ❌ | ❌ |
| Add your own controllers for new devices and test systems | ✅ | ❌ |
The five questions worth asking any testing vendor, answered for VirtualPyTest. More in the full FAQ.
Can you test our production app on multiple platforms without changing our code?
Yes. Nothing is installed in your app: no SDK, no instrumentation, no test build. The screen is captured from the outside (HDMI, camera, screen mirroring, browser) and the device is driven the way a user drives it (IR or Bluetooth remote, ADB, Appium, Playwright). The same script runs on STB, Android TV, mobile and web; the navigation tree holds what differs per platform. See Unified controller.
Can you show a live event monitored on multiple platforms, with a recording of what viewers saw?
Yes. Every device's screen is visible at once, with a fleet heatmap flagging devices in trouble. Each device's HDMI output is recorded continuously (video, audio, transcript) on a rolling 24-hour buffer, and incidents keep their start and end frames as evidence. See Visual capture.
Do you measure picture and sound quality, or only whether a screen appeared?
Both, on every captured frame, 24/7: black screen, freeze, blur, blockiness; audio loss, silence, loudness (LKFS), saturation; subtitles (OCR + language detection), zapping, banners; plus an approximate MOS (1–5) per minute in Grafana. The MOS is an indicator, not a certified lab measurement (no VMAF). See AV quality.
What happens to our tests when we redesign the app or switch language?
The scripts don't change. Scripts name destinations like navigate_to("settings"), never pixels or selectors, so a redesign only touches the navigation tree: recapture reference images in one click or let AI exploration rebuild nodes. Screens are recognised by a layout fingerprint that tolerates translated text, and text checks use OCR with language detection and fuzzy matching. A full redesign still needs someone to review the updated tree.
What will 24/7 monitoring on all our devices cost per month, with no surprises?
Software: €0 a month. Open source (AGPL v3), self-hosted, no licence, per-device or usage fees. Hardware is a one-time cost: ~€420 for 1 device (Raspberry Pi 5 + HDMI capture card), ~€480 for up to 4 devices on one host, ~€7 per extra capture card. Running costs are electricity, plus storage beyond the default 24 h of recordings (~7.5 GB per device per day). See the hardware guide.
- 🐛 Issues: report bugs or request features on GitHub Issues.
- 💬 Discussions: ask questions and share ideas in GitHub Discussions.
- 🤝 Contributing: see CONTRIBUTING.md.
- 💖 Sponsor: support the project on GitHub Sponsors.
VirtualPyTest is open source under the GNU AGPL v3. Free to use, study, modify, and self-host, including for commercial internal use. If you distribute a modified version or offer it to others as a service, you must publish your modifications under the same license. For commercial licensing outside these terms, contact the author.