[OneDev #7] track: Manifold gaps — deployment, Meridian auth, intelligent routing #35
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Imported from OneDev issue #7 (id 35184)
Original project: internal-joseph
Original state: Open
Original submitterId: 5
Original submitDate: 2026-04-08T15:05:07.007+00:00
Original lastActivity: {'date': '2026-04-08T15:05:07.007+00:00', 'description': 'opened', 'userId': 5}
Original fields:
"(Replaces internal-joseph#3. Updated 2026-04-09 to track Manifold gaps instead of original research scope.)\n\n## Status\n\nThe "Meridian" concept described in the original issue has been realized as Manifold \u00e2\u20ac\u201d a two-layer LLM inference gateway (Go service + LiteLLM proxy). Phases 1\u00e2\u20ac\u201c9 are code-complete with tests. This issue now tracks remaining gaps.\n\n> Naming note: "Meridian" means two things in the Manifold codebase: (1) the original concept name for this router, and (2)
@rynfar/meridian, the third-party Claude API proxy that is one of Manifold's four backends. This issue uses "Meridian" in sense (1) only.\n\n## Original open questions \u00e2\u20ac\u201d resolved\n\n| Question | Resolution |\n|----------|-----------|\n| New repo or existing? | Newmanifoldrepo (C:/Users/Joseph/git/manifold) |\n| Proxy or sidecar? | Proxy \u00e2\u20ac\u201d sits in the data path (OpenAI-compatible API surface) |\n| Absorb stack-dashboard routing? | stack-dashboard becomes a Manifold consumer |\n\n## Category A \u00e2\u20ac\u201d Unfinished Manifold phases\n\nThese are designed (indocs/implementation-plan.md) but not yet implemented:\n\n- [ ] Phase 10 \u00e2\u20ac\u201d Deployment: systemd units (manifold.service,litellm.service,meridian.service), Caddy config,deploy/setup.sh, deployment docs, Dex client registration\n- [ ] Phase 11 \u00e2\u20ac\u201d Meridian auth lifecycle:claude loginsession setup, crash recovery, health detection of expired sessions, end-to-end chain validation (manifold \u00e2\u2020\u2019 LiteLLM \u00e2\u2020\u2019 Meridian \u00e2\u2020\u2019 Claude API)\n\n## Category B \u00e2\u20ac\u201d Beyond Manifold v1 ("intelligent matchmaking")\n\nThe original issue described routing based on cost, latency, model fit, and current load. Manifold v1 routes by static config (capability aliases + priority tiers). These are the gaps:\n\n- [ ] Load-aware routing \u00e2\u20ac\u201d Manifold checks backend health (up/down) but not load metrics (queue depth, GPU utilization, concurrent requests)\n- [ ] Cost-aware routing \u00e2\u20ac\u201d No cost model; doesn't factor $/token into routing decisions\n- [ ] Latency-aware routing \u00e2\u20ac\u201d Routes by priority tier, not by measured latency history; no latency budgets\n- [ ] Learned routing (original "Phase 2") \u00e2\u20ac\u201d No adaptive/ML component; all routing rules are static YAML config\n- [ ] Dynamic backend registration \u00e2\u20ac\u201d Backends are static inconfig.yaml; no runtime API to add/remove/discover backends\n- [ ] Multi-node Lemonade \u00e2\u20ac\u201d Explicitly deferred as non-goal for v1; needed when more inference hardware arrives (Strix Halo cluster)\n- [ ] E2E testing against real backends \u00e2\u20ac\u201d All tests use mocks (httptest.Server); no integration tests against actual Lemonade/Gemma/Meridian/OpenAI\n\n## Architecture reference\n\n\nConsumer \u00e2\u2020\u2019 manifold (:3020) \u00e2\u2020\u2019 LiteLLM (:4000) \u00e2\u2020\u2019 Backend\n \u00e2\u201d\u0153\u00e2\u201d\u20ac\u00e2\u201d\u20ac Lemonade (:8000)\n \u00e2\u201d\u0153\u00e2\u201d\u20ac\u00e2\u201d\u20ac Gemma 4 (:8081)\n \u00e2\u201d\u0153\u00e2\u201d\u20ac\u00e2\u201d\u20ac Meridian (:3456) \u00e2\u2020\u2019 Claude API\n \u00e2\u201d\u201d\u00e2\u201d\u20ac\u00e2\u201d\u20ac OpenAI (cloud)\n\n\nKey docs:manifold/docs/manifold-design.md,manifold/docs/implementation-plan.md,manifold/docs/test-plan.md\n\n## Why this still matters\n\nManifold v1 solves the "single API surface" and "fallback chains" problems, but the hard-coded capability aliases are still static routing. As backend count grows (Strix Halo, more cloud providers, specialized fine-tunes), static config won't scale \u00e2\u20ac\u201d the intelligent matchmaking layer is the next frontier."