diff --git a/research/INDEX.md b/research/INDEX.md index 8bf2d15..4712836 100644 --- a/research/INDEX.md +++ b/research/INDEX.md @@ -6,11 +6,11 @@ | Metric | Value | |---|---| -| Subjects | 7 | -| Total source documents | 527 | -| — Tier 1 (Source Dossier, independently analyzed) | 154 | +| Subjects | 8 | +| Total source documents | 531 | +| — Tier 1 (Source Dossier, independently analyzed) | 158 | | — Tier 2 (Source Citation Card, listed only) | 373 | -| Total conclusion documents | 15 | +| Total conclusion documents | 16 | | Archive opened | 2026-10-09 | | Last updated | 2026-10-12 | @@ -25,6 +25,7 @@ | [`tui-framework-design`](tui-framework-design/) | 94 (49 / 45) | 5 | 2026-10-10 | 2026-10-11 | Designing tai's in-house TUI framework (retained DOM, cuts/chops compositor, SGR mouse, bubbling events, VT sub-terminals); extended 2026-10-10 with synchronized-output verification (DEC 2026); implemented in working-tree tai.py and verified the same session; same-day follow-up fixed the 19 failures natively, deleted tai_pkg/, suite 320/320 green, committed; extended again for kitty keyboard protocol (double-Shift prompt dialog); extended 2026-10-11 with messaging-layout comparison (telecli, send-rs, za-tui, bubbletea examples) and palette-role theming survey (Textual, lipgloss, bubbletint, ratatui-themes, ratatui-kit, pytermgui) plus 2 conclusions, nothing built yet; extended again 2026-10-11 with Telegram look-and-feel survey (tdesktop day/night themes, lang strings, Android dialog/message cells, FAQ, blog, Bot API/features, WebApps ThemeParams, mini-apps SDK docs, iOS teardown — 16 Tier 1) plus 2 conclusions, nothing built yet | | [`agentic-tool-loop-design`](agentic-tool-loop-design/) | 74 (7 / 67) | 2 | 2026-10-11 | 2026-10-11 | Fixing stop-after-narration on prompt-JSON CLI backends without intent regexes; decided structural loop control plus exactly-once JSON-contract repair plus prompt contract, implemented in tai.py the same session (suite 357/357 + 16/16 green); approval tiers judged already industry-standard, no change | | [`muse-responses-openai-transpiler`](muse-responses-openai-transpiler/) | 66 (3 / 63) | 2 | 2026-10-11 | 2026-10-11 | Documenting Meta muse Responses wire format vs classic Chat Completions; decided bidirectional transpile is feasible with a fixed loss table and system guidance has three channels; scratch transpiler tested, nothing merged | +| [`free-model-discovery`](free-model-discovery/) | 4 (4 / 0) | 1 | 2026-10-12 | 2026-10-12 | Bringing tai's free model lists up to date from live endpoints (zen, openrouter, devplace, pollinations); live discovery replaces hard-coded lists | ## Recent activity log @@ -53,3 +54,4 @@ - Before starting new research, search this table for an existing subject before opening a new one (see [README.md](README.md#2-directory-layout) §2, "one subject, one home"). - Every subject folder is self-contained: its sources live flat inside it, its conclusions live in its `conclusions/` subfolder. - Tier 1 vs Tier 2 is a statement about how deeply a source was actually read, not about its importance — see [README.md](README.md#3-the-three-document-types) §3 for the exact distinction. +| 2026-10-12 | `free-model-discovery` | New subject: 4 Tier 1 dossiers (zen, openrouter, devplace, pollinations model listings, all fetched live) + 1 conclusion (live discovery over hard-coded lists). Implemented and `--update-models` run the same session. | diff --git a/research/free-model-discovery/conclusions/discover-free-models-live.md b/research/free-model-discovery/conclusions/discover-free-models-live.md new file mode 100644 index 0000000..4b9dce5 --- /dev/null +++ b/research/free-model-discovery/conclusions/discover-free-models-live.md @@ -0,0 +1,48 @@ +# Conclusion: Discover free models live, keep static lists only as fallback + +## Metadata + +| Field | Value | +|---|---| +| Institute | Molodetz Institute — Research & Retrieval Division | +| Subject | free-model-discovery | +| Document type | Conclusion | +| Tier | N/A | +| Date decided | 2026-10-12 | +| Decided by | retoor, advised by tai (agent, Claude Sonnet 5) | +| Status | Active | +| Confidence | Medium | + +## Question / problem statement + +Which free models do the providers offer today, and why had tai's candidate lists drifted from that? + +## Conclusion + +Every provider list tai uses (zen, pollinations, openrouter, devplace, opencode) is read from the provider's own endpoint, cached, and refreshed by `tai.py --update-models`. The hard-coded tuples (ZEN_FREE_MODELS, POLLINATIONS_MODELS, DEVPLACE_FREE_ROUTES) are only the fallback when an endpoint is unreachable, and were refreshed to the 2026-10-12 listings. + +## Supporting sources + +- [OpenCode Zen listing](../opencode-zen-v1-models.md): current free zen ids and the header requirement. +- [OpenRouter listing](../openrouter-api-v1-models.md): current `:free` ids, `openrouter/free`, keyless access. +- [DevPlace listing](../devplace-openai-v1-models.md): current ids, why the old filter dropped molodetz. +- [Pollinations list](../pollinations-text-models.md): anonymous tier, correct discovery URL. + +## Alternatives considered and rejected + +- Only refresh the static tuples by hand: rejected, they drifted within weeks (four zen ids already gone). +- Decide freeness by price fields: rejected, only openrouter exposes one. + +## Risks / open questions + +Freeness is inferred from the `-free` suffix, the `:free` suffix, or the pollinations anonymous tier. `big-pickle` (zen, devplace) and `molodetz-pro/-g/-q` are unverified and excluded. + +## Action taken + +Implemented in tai.py (ChatClient.zen_models, pollinations_models, fetch_pollinations_free, is_free_openrouter, updated filters and fallbacks); `--update-models` run live against all reachable sources. + +## Changelog + +| Date | Change | +|---|---| +| 2026-10-12 | Initial conclusion recorded. | diff --git a/research/free-model-discovery/devplace-openai-v1-models.md b/research/free-model-discovery/devplace-openai-v1-models.md new file mode 100644 index 0000000..17cbc75 --- /dev/null +++ b/research/free-model-discovery/devplace-openai-v1-models.md @@ -0,0 +1,47 @@ +# DevPlace /openai/v1/models listing + +## Metadata + +| Field | Value | +|---|---| +| Institute | Molodetz Institute — Research & Retrieval Division | +| Subject | free-model-discovery | +| Document type | Source Dossier | +| Tier | 1 | +| Source title | DevPlace /openai/v1/models listing | +| Source URL | https://devplace.net/openai/v1/models | +| Source publisher / author | Molodetz (devplace.net) | +| Source type | vendor docs | +| Source date (published) | Unknown — not stated | +| Date researched | 2026-10-12 | +| Researched by | tai (agent), model: Claude Sonnet 5, on behalf of retoor | +| Research method | direct page fetch | +| Confidence | Medium | +| Status | Active | + +## Summary + +Authenticated (Bearer DEVPLACE_API_KEY; 401 without) OpenAI-style list of 46 ids, owner `molodetz`. + +## Key findings + +- Includes the router ids `free`, `devii`, `molodetz`, `molodetz-pro`, `molodetz-g`, `molodetz-q`, `autof`, and `-free` models: deepseek-v4-flash-free, laguna-s-2.1-free, ling-3.0-flash-fin-free, mimo-v2.5-free, nemotron-3-ultra-free, nemotron-3.5-lightning-free. +- `muse-spark-1.3-contributor` is listed without `-free`; tai's old filter required both muse and `-free`, so it dropped real free-tier entries and `molodetz`. + +## Pros / strengths + +Live, machine-readable, and the same endpoint tai calls at runtime, so what it reports is what a request will actually see. + +## Cons / limitations / caveats + +No price field, so which of molodetz-pro/-g/-q cost anything is unknown; they were left out of the free set. + +## Relevance to tai + +Old filter kept only `free` and `devii` plus muse-and-free ids. New filter: the route ids (now incl. `molodetz`) plus anything ending in `-free`. + +## Changelog + +| Date | Change | +|---|---| +| 2026-10-12 | Initial dossier created. | diff --git a/research/free-model-discovery/opencode-zen-v1-models.md b/research/free-model-discovery/opencode-zen-v1-models.md new file mode 100644 index 0000000..53e0246 --- /dev/null +++ b/research/free-model-discovery/opencode-zen-v1-models.md @@ -0,0 +1,48 @@ +# OpenCode Zen /v1/models listing + +## Metadata + +| Field | Value | +|---|---| +| Institute | Molodetz Institute — Research & Retrieval Division | +| Subject | free-model-discovery | +| Document type | Source Dossier | +| Tier | 1 | +| Source title | OpenCode Zen /v1/models listing | +| Source URL | https://opencode.ai/zen/v1/models | +| Source publisher / author | OpenCode (opencode.ai) | +| Source type | vendor docs | +| Source date (published) | Unknown — not stated | +| Date researched | 2026-10-12 | +| Researched by | tai (agent), model: Claude Sonnet 5, on behalf of retoor | +| Research method | direct page fetch | +| Confidence | Medium | +| Status | Active | + +## Summary + +Unauthenticated GET returns an OpenAI-style model list. Plain urllib is refused with HTTP 403; the request only succeeds with opencode client headers (User-Agent and x-opencode-*), as curl's default agent was accepted but Python's was not. + +## Key findings + +- 2026-10-12 listing held 80+ ids, of which these end in `-free`: ling-3.1-flash-free, mimo-v2.6-flash-free, muse-spark-1.2-contributor-free, muse-spark-1.3-contributor-free, nemotron-3.5-lightning-free, jev-1.13-free, step-5-preview-free, longcat-2.5-preview-free, space-bunny-free. +- `big-pickle` is listed without a `-free` suffix; its price was not verified. +- tai's static list still named nemotron-3-ultra-free, exo-free, ling-3.0-flash-fin-free and fledge-alpha-free, none of which the endpoint listed. + +## Pros / strengths + +Live, machine-readable, and the same endpoint tai calls at runtime, so what it reports is what a request will actually see. + +## Cons / limitations / caveats + +The listing carries no price field, so freeness is inferred from the `-free` suffix only. Anti-bot behaviour (403 for default Python agent) may change. + +## Relevance to tai + +Zen candidates used a hard-coded list that had drifted. Discovery now reads this endpoint (with opencode headers) and caches it. + +## Changelog + +| Date | Change | +|---|---| +| 2026-10-12 | Initial dossier created. | diff --git a/research/free-model-discovery/openrouter-api-v1-models.md b/research/free-model-discovery/openrouter-api-v1-models.md new file mode 100644 index 0000000..ac32605 --- /dev/null +++ b/research/free-model-discovery/openrouter-api-v1-models.md @@ -0,0 +1,48 @@ +# OpenRouter /api/v1/models listing + +## Metadata + +| Field | Value | +|---|---| +| Institute | Molodetz Institute — Research & Retrieval Division | +| Subject | free-model-discovery | +| Document type | Source Dossier | +| Tier | 1 | +| Source title | OpenRouter /api/v1/models listing | +| Source URL | https://openrouter.ai/api/v1/models | +| Source publisher / author | OpenRouter | +| Source type | vendor docs | +| Source date (published) | Unknown — not stated | +| Date researched | 2026-10-12 | +| Researched by | tai (agent), model: Claude Sonnet 5, on behalf of retoor | +| Research method | direct page fetch | +| Confidence | Medium | +| Status | Active | + +## Summary + +Public model list including a pricing object per model; readable without an API key. + +## Key findings + +- 2026-10-12 free entries (`:free` suffix): apodex-1.1-mini, dots-3-note-preview, lfm-2.5-2.6b, nemotron-3.5-lightning, inkling-small, laguna-s-2.1, inkling, laguna-xs-2.1, north-mini-code, nemotron-3.5-content-safety, nemotron-3-ultra-550b-a55b, nemotron-3-nano-omni-30b-a3b-reasoning, gemma-4-26b-a4b-it, gemma-4-31b-it, nemotron-3-super-120b-a12b. +- `openrouter/free` is a free auto-router id without the `:free` suffix. +- Some zero-priced ids (lyria music models, ling-3.1-flash) are not `:free` and not chat models or not tagged free; tai ignores them. + +## Pros / strengths + +Live, machine-readable, and the same endpoint tai calls at runtime, so what it reports is what a request will actually see. + +## Cons / limitations / caveats + +Zero price on a non-`:free` id is not a guarantee of chat usability. Safety-classifier models (nemotron-content-safety) appear in the free set and are poor chat candidates; health ranking demotes them after failures. + +## Relevance to tai + +tai's discovery already filtered on `:free`; it now also accepts `openrouter/free` and runs without an OpenRouter key. + +## Changelog + +| Date | Change | +|---|---| +| 2026-10-12 | Initial dossier created. | diff --git a/research/free-model-discovery/pollinations-text-models.md b/research/free-model-discovery/pollinations-text-models.md new file mode 100644 index 0000000..6040461 --- /dev/null +++ b/research/free-model-discovery/pollinations-text-models.md @@ -0,0 +1,48 @@ +# Pollinations text model list + +## Metadata + +| Field | Value | +|---|---| +| Institute | Molodetz Institute — Research & Retrieval Division | +| Subject | free-model-discovery | +| Document type | Source Dossier | +| Tier | 1 | +| Source title | Pollinations text model list | +| Source URL | https://text.pollinations.ai/models | +| Source publisher / author | Pollinations | +| Source type | vendor docs | +| Source date (published) | Unknown — not stated | +| Date researched | 2026-10-12 | +| Researched by | tai (agent), model: Claude Sonnet 5, on behalf of retoor | +| Research method | direct page fetch | +| Confidence | Medium | +| Status | Active | + +## Summary + +JSON array with a `tier` per model. The OpenAI-style path `/openai/v1/models` returned an unrelated chat-style text answer, not a list, so it cannot be used for discovery. + +## Key findings + +- 2026-10-12 only one model had tier `anonymous`: `openai-fast` (GPT-OSS 20B reasoning), aliases include `openai`. +- A chat completion against `/openai/v1/chat/completions` with model `openai-fast` succeeded without a key. +- gen.pollinations.ai/v1/models lists many more models with pollen pricing, i.e. not anonymous-free. + +## Pros / strengths + +Live, machine-readable, and the same endpoint tai calls at runtime, so what it reports is what a request will actually see. + +## Cons / limitations / caveats + +Tier semantics are Pollinations' own and can change; only one anonymous model exists today, so discovery returns a list of one. + +## Relevance to tai + +The static default `openai` was an alias; discovery now returns the real anonymous-tier names with `openai-fast` as fallback. + +## Changelog + +| Date | Change | +|---|---| +| 2026-10-12 | Initial dossier created. | diff --git a/tai.py b/tai.py index 21259bf..4f0c327 100755 --- a/tai.py +++ b/tai.py @@ -141,14 +141,15 @@ OPENCODE_AGENT = "build" OPENCODE_PARSE_MAX_SCANS = 200 DEVPLACE_LABEL = "devplace" DEVPLACE_BASE = "https://devplace.net/openai/v1" -DEVPLACE_FREE_ROUTES = ("free", "devii") +DEVPLACE_FREE_ROUTES = ("free", "devii", "molodetz") PREFERRED_MODEL = "molodetz" ZEN_LABEL = "zen" ZEN_BASE = "https://opencode.ai/zen/v1" -ZEN_FREE_MODELS = ("mimo-v2.6-flash-free", "nemotron-3.5-lightning-free", "nemotron-3-ultra-free", "muse-spark-1.3-contributor-free", "muse-spark-1.2-contributor-free", "jev-1.13-free", "exo-free", "space-bunny-free", "longcat-2.5-preview-free", "ling-3.1-flash-free", "ling-3.0-flash-fin-free", "fledge-alpha-free") +ZEN_FREE_MODELS = ("mimo-v2.6-flash-free", "nemotron-3.5-lightning-free", "muse-spark-1.3-contributor-free", "muse-spark-1.2-contributor-free", "jev-1.13-free", "step-5-preview-free", "space-bunny-free", "longcat-2.5-preview-free", "ling-3.1-flash-free") POLLINATIONS_LABEL = "pollinations" POLLINATIONS_BASE = "https://text.pollinations.ai/openai/v1" -POLLINATIONS_MODELS = ("openai",) +POLLINATIONS_MODELS = ("openai-fast",) +POLLINATIONS_LIST_URL = "https://text.pollinations.ai/models" PROVIDER_LABELS = (MUSE_LABEL, GROK_LABEL, CLAUDE_LABEL, CODEX_LABEL, GEMINI_LABEL, OPENCODE_LABEL, OLLAMA_LABEL, OPENROUTER_LABEL, POLLINATIONS_LABEL, ZEN_LABEL, DEVPLACE_LABEL) MENTION_AUTO = "auto" MENTION_ROUTES = {"molodetz": (DEVPLACE_LABEL, PREFERRED_MODEL)} @@ -367,7 +368,7 @@ def known_model_ids(config, health=None): if getattr(config, "explicit_model", ""): ids.add(config.explicit_model) if health is not None: - for name in ("openrouter:free", "opencode:free", "ollama:local", "devplace:free", "opencode:box:free"): + for name in ("openrouter:free", "opencode:free", "ollama:local", "devplace:free", "opencode:box:free", "zen:free", "pollinations:free"): try: cached = health.cached_list(name) except Exception: @@ -3006,9 +3007,9 @@ class Config: self.devplace_models = split_models(os.environ.get("TAI_DEVPLACE_MODELS")) self.openrouter_enabled = (os.environ.get("TAI_OPENROUTER") or "1").strip().lower() not in ("0", "false", "no", "off") self.pollinations_enabled = (os.environ.get("TAI_POLLINATIONS") or "1").strip().lower() not in ("0", "false", "no", "off") - self.pollinations_models = split_models(os.environ.get("TAI_POLLINATIONS_MODELS")) or list(POLLINATIONS_MODELS) + self.pollinations_models = split_models(os.environ.get("TAI_POLLINATIONS_MODELS")) self.zen_enabled = (os.environ.get("TAI_ZEN") or "").strip().lower() in ("1", "true", "yes", "on") - self.zen_models = split_models(os.environ.get("TAI_ZEN_MODELS")) or list(ZEN_FREE_MODELS) + self.zen_models = split_models(os.environ.get("TAI_ZEN_MODELS")) self.voice = os.environ.get("TAI_VOICE") or DEFAULT_VOICE self.auto_approve = bool(args.yes) self.profile = args.profile or "default" @@ -5074,6 +5075,23 @@ def fetch_model_ids(models_url, key="", extra_headers=None, timeout=MODEL_LIST_T return ids or None +def fetch_pollinations_free(): + """Anonymous-tier model names from pollinations; None when unreachable.""" + try: + with urllib.request.urlopen(urllib.request.Request(POLLINATIONS_LIST_URL, method="GET"), timeout=MODEL_LIST_TIMEOUT) as response: + rows = json.loads(response.read().decode("utf-8", "replace")) + except (OSError, ValueError): + return None + if not isinstance(rows, list): + return None + names = [str(row.get("name")).strip() for row in rows if isinstance(row, dict) and row.get("tier") == "anonymous" and row.get("name") and (row.get("output_modalities") or ["text"]) == ["text"]] + return names or None + + +def is_free_openrouter(model): + return model.endswith(":free") or model == "openrouter/free" + + def embedded_failure_message(data): if not isinstance(data, dict): return "" @@ -5170,7 +5188,7 @@ class ChatClient: free = self.health.cached_list("openrouter:free") if free is None: fetched = fetch_model_ids(OPENROUTER_BASE + "/models", self.config.openrouter_key) - free = [item for item in fetched if item.endswith(":free")] if fetched else [] + free = [item for item in fetched if is_free_openrouter(item)] if fetched else [] if free: self.health.store_list("openrouter:free", free) free = prefer_muse(free or []) @@ -5182,18 +5200,22 @@ class ChatClient: def update_models(self): """Rediscover every model list from its source. A source that cannot be reached keeps its old list (reported, never emptied).""" sources = [("opencode:free", OPENCODE_LABEL, lambda: opencode_free_models() if opencode_binary() else None)] - if self.config.openrouter_key: + if getattr(self.config, "openrouter_enabled", True): def openrouter(): fetched = fetch_model_ids(OPENROUTER_BASE + "/models", self.config.openrouter_key) - return [item for item in fetched if item.endswith(":free")] if fetched else None + return [item for item in fetched if is_free_openrouter(item)] if fetched else None sources.append(("openrouter:free", OPENROUTER_LABEL, openrouter)) if self.config.devplace_key: def devplace(): fetched = fetch_model_ids(self.config.devplace_base + "/models", self.config.devplace_key) if not fetched: return None - return [item for item in fetched if item in DEVPLACE_FREE_ROUTES or (MUSE_LABEL in item.lower() and is_free_model(item))] + return [item for item in fetched if item in DEVPLACE_FREE_ROUTES or is_free_model(item)] sources.append(("devplace:free", DEVPLACE_LABEL, devplace)) + if self.config.zen_enabled: + sources.append(("zen:free", ZEN_LABEL, lambda: [item for item in fetch_model_ids(ZEN_BASE + "/models", extra_headers=opencode_headers()) or [] if is_free_model(item)] or None)) + if self.config.pollinations_enabled: + sources.append(("pollinations:free", POLLINATIONS_LABEL, fetch_pollinations_free)) if getattr(self.config, "ollama_enabled", True) is not False: sources.append(("ollama:local", OLLAMA_LABEL, lambda: fetch_model_ids(OLLAMA_BASE + "/models", "", timeout=3))) report = [] @@ -5219,6 +5241,30 @@ class ChatClient: self.health.store_list("opencode:free", models) return prefer_muse(models) + def zen_models(self): + models = list(getattr(self.config, "zen_models", []) or []) + if models: + return prefer_muse(models) + cached = self.health.cached_list("zen:free") + if cached is None: + fetched = [item for item in fetch_model_ids(ZEN_BASE + "/models", extra_headers=opencode_headers()) or [] if is_free_model(item)] + cached = fetched or list(ZEN_FREE_MODELS) + if fetched: + self.health.store_list("zen:free", fetched) + return prefer_muse(cached) + + def pollinations_models(self): + models = list(getattr(self.config, "pollinations_models", []) or []) + if models: + return models + cached = self.health.cached_list("pollinations:free") + if cached is None: + fetched = fetch_pollinations_free() + cached = fetched or list(POLLINATIONS_MODELS) + if fetched: + self.health.store_list("pollinations:free", fetched) + return cached + def ollama_models(self): if getattr(self.config, "ollama_enabled", True) is False: return [] @@ -5240,7 +5286,7 @@ class ChatClient: fetched = fetch_model_ids(self.config.devplace_base + "/models", self.config.devplace_key) if fetched is None: return list(DEVPLACE_FREE_ROUTES) - free = [item for item in fetched if item in DEVPLACE_FREE_ROUTES or (MUSE_LABEL in item.lower() and is_free_model(item))] + free = [item for item in fetched if item in DEVPLACE_FREE_ROUTES or is_free_model(item)] if free: self.health.store_list("devplace:free", free) return prefer_muse(free or []) @@ -5262,10 +5308,10 @@ class ChatClient: for model in self.openrouter_models(): found.append({"provider": OPENROUTER_LABEL, "chat_url": OPENROUTER_BASE + "/chat/completions", "models_url": OPENROUTER_BASE + "/models", "key": self.config.openrouter_key, "model": model, "profile": ""}) if getattr(self.config, "pollinations_enabled", True): - for model in getattr(self.config, "pollinations_models", []) or list(POLLINATIONS_MODELS): + for model in self.pollinations_models(): found.append({"provider": POLLINATIONS_LABEL, "chat_url": POLLINATIONS_BASE + "/chat/completions", "models_url": POLLINATIONS_BASE + "/models", "key": "", "model": model, "profile": ""}) if getattr(self.config, "zen_enabled", False): - for model in getattr(self.config, "zen_models", []) or list(ZEN_FREE_MODELS): + for model in self.zen_models(): found.append({"provider": ZEN_LABEL, "chat_url": ZEN_BASE + "/chat/completions", "models_url": ZEN_BASE + "/models", "key": "", "model": model, "profile": "opencode"}) found.append({"provider": MUSE_LABEL, "chat_url": MUSE_CHAT_URL, "models_url": MUSE_MODELS_URL, "key": "", "model": self.config.model, "profile": ""}) if grok_binary():