IT-DEV/lab
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
IT-DEV/lab
15 commits 1 branch 0 tags 123 KiB
  • Shell 47.7%
  • JavaScript 31.1%
  • Dockerfile 14.3%
  • HTML 6.9%
Find a file
Exact
Joseph fe4d31ee24 docs(lab): backfill int-saas-m365-tool#18 gate-test result + Carbonyl revisit note 
- §5 Playwright MCP: record PASSED verdict on the Copilot gate test
  via m365-tool#18, note copilot-query skill ship (commit c230992),
  and link the headless server-side follow-up at m365-tool#19.
- §3 Carbonyl: soften the "dead since 2023" DROP with a deferred
  re-eval pointer to int-lab#10 (community signal boost flagged).
2026-04-11 16:50:39 -05:00
carbonyl chore(carbonyl): addendum — tmux/tui-use agent browser concept 2026-04-10 12:52:51 -05:00
cf-crawl test(lab): phase 1 smoke test results for all 5 tools 2026-04-03 12:01:50 -05:00
dev-browser test(lab): phase 1 smoke test results for all 5 tools 2026-04-03 12:01:50 -05:00
docs docs(lab): backfill int-saas-m365-tool#18 gate-test result + Carbonyl revisit note 2026-04-11 16:50:39 -05:00
lightpanda test(lab): phase 1 smoke test results for all 5 tools 2026-04-03 12:01:50 -05:00
page-agent test(lab): phase 1 smoke test results for all 5 tools 2026-04-03 12:01:50 -05:00
.gitignore chore(lab): add .gitignore for .worktrees 2026-04-11 14:25:26 -05:00
docker-compose.yml feat(lab): add docker-compose scaffold and run-all script 2026-04-03 11:46:17 -05:00
README.md docs(lab): update README for int-lab#9 research expansion 2026-04-11 15:54:47 -05:00
run-all.sh feat(lab): add docker-compose scaffold and run-all script 2026-04-03 11:46:17 -05:00

README.md

Lab — AI Browser Tooling Experiments

Phase 1 Smoke Test Results (2026-04-03)

Agent Browser Head-to-Head

dev-browser Lightpanda Carbonyl
Gate 1 (install) PASS PASS PASS
Gate 2 (runs) PASS PASS PASS
Gate 3 (output) PASS PASS PASS
Speed 4/5 (946ms) 5/5 (61ms avg) 3/5 (~5s)
Memory 3/5 (470MB img) 5/5 (64MB img) 2/5 (112MB img)
Output Quality 5/5 4/5 2/5
Agent Integration 4/5 5/5 1/5
Maintenance 3/5 4/5 1/5
TOTAL 19/25 23/25 9/25
VERDICT PROMOTE PROMOTE DROP

Winner: Lightpanda (23/25) — fastest, smallest, built-in MCP server, CDP mode. Runner-up: dev-browser (19/25) — best for visual output (screenshots), genuine security sandbox.

Portfolio: page-agent.js

  • Gates: 2/3 passed (Gate 3 failed — model compatibility, not tool fault)
  • Scorecard: 15/25
  • PCT Use Case: In-page DOM automation for web forms, data extraction
  • Worth a LXC slot? NO (not yet)
  • Reasoning: The tool itself works — it observed the DOM, indexed elements, and successfully clicked a button. But it requires an LLM that correctly follows its tool-calling schema, and current local models (Qwen3-Coder-30B) emit unsupported actions. Revisit when local model tool-calling improves, or test with Claude API backend.

Portfolio: Cloudflare /crawl

  • Gates: 1/3 passed (SKIP — no credentials provided)
  • Scorecard: Not scored
  • PCT Use Case: JS-rendered site crawling, markdown extraction for agent context
  • Worth a LXC slot? N/A — API service, no LXC needed
  • Reasoning: Cannot evaluate without Cloudflare Workers Paid credentials. The API returns markdown/HTML/JSON from a single POST call — very promising for JS-heavy SPAs where Lightpanda's Zig engine may fall short. PCT already has a CF account; worth testing once Workers Paid is confirmed active.

Verdict Key

  • PROMOTE — gets a PVE LXC, Phase 2 integration prototype
  • BENCH — works but no immediate use case or blocked, revisit later
  • DROP — failed gates or scored too low to justify

Key Findings

  • Lightpanda is the standout. 61ms fetches, 64MB image, built-in MCP server, CDP compatibility. It replaces headless Chrome for text/markdown extraction at a fraction of the cost.
  • dev-browser fills the visual gap. When you need screenshots (QA, visual verification), dev-browser's sandboxed Chromium is the right tool. The QuickJS sandbox prevents agents from accessing the filesystem.
  • Carbonyl is dead. Abandoned since Feb 2023, TTY-only, no API. Interesting concept, zero practical value.
  • page-agent.js has potential but is blocked on local LLM tool-calling fidelity.
  • Lightpanda's MCP server (lightpanda mcp) is the most exciting discovery — it can be wired directly into Claude Code as an MCP tool with zero glue code.

See each tool's result.md for full details.

Phase 2 (in progress)

Winners from Phase 1 get promoted to PVE LXC with integration prototypes:

  1. Lightpanda — MCP tool for stack-dashboard chat, markdown extraction pipeline — int-saas-stack-dashboard#51
  2. dev-browser — Agent browsing sandbox, screenshot-based QA automation — watchlist (no concrete target yet)

Research expansion (2026-04-11)

int-lab#9 expanded the scope from the original 5 candidates to 18, adding MCP servers, agent libraries, research benchmarks, open CUA models, and stealth browsers. Full research: docs/2026-04-11-browser-agent-stack-research.md.

Tier 1 additions from that research:

  1. Playwright MCP — M365 Copilot browser-driven fallback primary. Per-workspace persistent profile survives Entra MFA. — int-saas-m365-tool#18
  2. browser-use — conditional fallback-to-the-fallback, only stood up if #18 hits a wall.

Surprise fourth PROMOTE: jo-inc/camofox-browser — Firefox-fork with engine-level fingerprint randomization. Watchlist today, break-glass option if any target fingerprint-walls Playwright.

See the research doc §3 for 18-candidate full comparison, §7 for the tier list, §8 for prototype next-steps.