[OneDev #3] research: develop testing + elaborate Meridian implementation ideas

joseph commented

2026-05-24 04:46:41 +00:00

Owner

Imported from OneDev issue #3 (id 35179)
Original project: internal-joseph
Original state: Closed
Original submitterId: 5
Original submitDate: 2026-04-08T14:37:10.530+00:00
Original lastActivity: {'date': '2026-04-08T15:04:27.549+00:00', 'description': "changed state to 'Closed'", 'userId': 5}

Original fields:

Type: New Feature
Priority: Normal
Assignees: None

"## Idea\nMeridian = a new project for LLM routing — linking capacity (available inference backends: localinf Lemonade, Gemma 4 server, Ollama, Claude API, future GPU pools) to demand (chat clients, agents, batch jobs, dashboards) with intelligent matchmaking based on cost, latency, model fit, and current load.\n\nTrack this issue covers two parallel workstreams:\n1. Implementation design — capture the current architecture thinking into docs/meridian-design.md (repo TBD — likely new repo under PCT-INTERNAL/saas or PCT-INTERNAL/infrastructure/ai)\n2. Testing strategy — define what "working" looks like before code: routing decision tests, capacity probe tests, fallback chain tests, latency budget tests\n\n## Scope\n- Single design doc: routing model, capacity discovery, demand classification, decision algorithm, fallback chains\n- Identify testable surfaces: pure routing function (no I/O), capacity probes (mockable), decision log (auditable)\n- Decide test harness shape before writing code (table-driven? scenario fixtures?)\n- Outline phased milestones — likely Phase 0 = static routing rules, Phase 1 = live capacity probes, Phase 2 = learned routing\n\n## Open questions\n- New repo, or live inside an existing one (e.g. infra-ai, stack-dashboard)?\n- Does Meridian proxy requests (sit in the data path) or advise clients (sidecar query)? Big architectural fork.\n- Relationship to stack-dashboard chat routing (which already does text→Lemonade / image→Gemma) — does Meridian absorb that logic, or does stack-dashboard call out to Meridian?\n\n## Why\nRight now LLM routing is hard-coded in each client (stack-dashboard chat tool, RC-AT, m365-tool, etc.). As more inference backends come online (Lemonade Coder-30B, Gemma 4, future Strix Halo cluster, Claude API for premium tasks) and more clients want to consume them, hand-wired routing becomes the bottleneck. Meridian is the dispatcher that lets capacity and demand evolve independently.\n\n## Next session handoff\n- Pick this up on a Research day (per internal-joseph#1)\n- First deliverable: design doc, not code"

Imported from OneDev issue #3 (id 35179) Original project: internal-joseph Original state: Closed Original submitterId: 5 Original submitDate: 2026-04-08T14:37:10.530+00:00 Original lastActivity: {'date': '2026-04-08T15:04:27.549+00:00', 'description': "changed state to 'Closed'", 'userId': 5} Original fields: - Type: New Feature - Priority: Normal - Assignees: None "## Idea\n**Meridian** = a new project for LLM routing — linking *capacity* (available inference backends: localinf Lemonade, Gemma 4 server, Ollama, Claude API, future GPU pools) to *demand* (chat clients, agents, batch jobs, dashboards) with intelligent matchmaking based on cost, latency, model fit, and current load.\n\nTrack this issue covers two parallel workstreams:\n1. **Implementation design** — capture the current architecture thinking into `docs/meridian-design.md` (repo TBD — likely new repo under `PCT-INTERNAL/saas` or `PCT-INTERNAL/infrastructure/ai`)\n2. **Testing strategy** — define what \"working\" looks like before code: routing decision tests, capacity probe tests, fallback chain tests, latency budget tests\n\n## Scope\n- Single design doc: routing model, capacity discovery, demand classification, decision algorithm, fallback chains\n- Identify testable surfaces: pure routing function (no I/O), capacity probes (mockable), decision log (auditable)\n- Decide test harness shape before writing code (table-driven? scenario fixtures?)\n- Outline phased milestones — likely Phase 0 = static routing rules, Phase 1 = live capacity probes, Phase 2 = learned routing\n\n## Open questions\n- New repo, or live inside an existing one (e.g. `infra-ai`, `stack-dashboard`)?\n- Does Meridian *proxy* requests (sit in the data path) or *advise* clients (sidecar query)? Big architectural fork.\n- Relationship to stack-dashboard chat routing (which already does text→Lemonade / image→Gemma) — does Meridian absorb that logic, or does stack-dashboard call out to Meridian?\n\n## Why\nRight now LLM routing is hard-coded in each client (stack-dashboard chat tool, RC-AT, m365-tool, etc.). As more inference backends come online (Lemonade Coder-30B, Gemma 4, future Strix Halo cluster, Claude API for premium tasks) and more clients want to consume them, hand-wired routing becomes the bottleneck. Meridian is the dispatcher that lets capacity and demand evolve independently.\n\n## Next session handoff\n- Pick this up on a **Research day** (per internal-joseph#1)\n- First deliverable: design doc, not code"

joseph closed this issue

2026-05-24 04:46:41 +00:00

joseph commented

2026-05-24 04:46:49 +00:00

Author

Owner

Imported from OneDev comment id 159 on issue #3
Original userId: 5
Original date: 2026-04-08T15:04:26.549+00:00

Replaced by clean version — description corrupted by od_set_issue_description double-encoding bug. See replacement issue.

Imported from OneDev comment id 159 on issue #3 Original userId: 5 Original date: 2026-04-08T15:04:26.549+00:00 Replaced by clean version — description corrupted by `od_set_issue_description` double-encoding bug. See replacement issue.

Rows
Columns

[OneDev #3] research: develop testing + elaborate Meridian implementation ideas #36