> ## Documentation Index > Fetch the complete documentation index at: https://docs.monolex.ai/llms.txt > Use this file to discover all available pages before exploring further. # MonoFetch > Fetch static web content, crawl documentation manifests, and index converted Markdown for Monomento. MonoFetch is the static-web ingestion edge of the Monomento stack. It handles one HTML, Markdown, or text response; `llms.txt` and `llms-full.txt`; and ordinary `sitemap.xml` files. Converted pages are stored in [Monomento](/ai-clis/monomento) as `doc_type=web`. *Published contract: `monofetch 0.3.15`, OpenCLIs build `2026-05-27-231822`. The installed-state row, immutable payload, public CDN pointer, and CDN release metadata agree on this identity. The exact installed payload was also code-signature verified on macOS.* ## Quick Start ```bash theme={null} openclis install monofetch monofetch --version monofetch fetch https://example.com/docs/getting-started --no-index monofetch fetch https://example.com/docs/getting-started monofetch search "installation" ``` Use `--no-index` for first inspection. Use [MonoSurf](/monosurf/index) instead when content depends on JavaScript, a browser login, or challenge handling. ## Published Command Surface The current public build has **5 operational root commands** plus help and version aliases. | Command | Reads | Writes | Contract | | ----------------------------- | ---------------------------------------------------------------------------------- | -------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------- | | `fetch ` | One HTTP response | A `web` document by default | Bare domains are coerced to HTTPS. `--no-index` prints converted content; `--raw` prints decoded response text without HTML conversion. | | `crawl ` | An `llms.txt`, `llms-full.txt`, sitemap, or auto-detected manifest and its targets | Every successfully imported page | Concurrent fetches, serialized index writes, partial-success result counters. | | `search [-n ]` | Shared Monomento index | Nothing | Best-effort web-only results with a substring fallback. Quote multiword queries. | | `toc ` | Network and/or shared index | **May write** | A URL is fetched and indexed before headings are printed. A query normally reads the index, but its fallback may re-fetch a stored URL. | | `list` | Shared Monomento database | Nothing | Lists only rows whose `doc_type` is `web`. | | `help`, `--help`, `-h` | Embedded root guide | Nothing | Prints the same root guide as bare `monofetch`. | | `version`, `--version`, `-v` | Package identity | Nothing | Prints `monofetch 0.3.15`. | `toc` is not uniformly read-only. `monofetch toc https://…` calls the normal indexing fetch path. A short/CJK query can also fall back to a stored HTTP URL and recursively invoke that write path. ## Parser and Help Boundary MonoFetch uses a hand-written dispatcher. There is no command-specific help: ```bash theme={null} monofetch help fetch # prints the root guide monofetch fetch --help # treats --help as a URL-like input and errors ``` `search` consumes only its first positional token as the query. Quote phrases: ```bash theme={null} monofetch search "release verification" -n 20 ``` An absent or invalid `-n` value falls back to 10. Unknown extra flags are not systematically rejected, so use only the documented forms. Repository HEAD contains a `docs [topic]` dispatcher for a future build. The immutable `0.3.15 / 2026-05-27-231822` payload rejects `docs` with a non-zero exit, so it is not part of the current public contract. ## Fetch Output Modes Mode selection is ordered: `--raw` wins over `--no-index`, and `--pretty` has an effect only inside the raw branch. ```text theme={null} fetch URL ├── --raw → charset-decoded response body text on stdout │ └── --pretty → parse as JSON and pretty-print when parsing succeeds ├── --no-index → converted Markdown/text on stdout └── default → converted Markdown + persistent Monomento index ``` Despite the wording in the current embedded help, `--raw` is **not a byte-for-byte archive mode**. The implementation decodes response bytes using the HTTP charset, then an HTML `` hint, then UTF-8; it prints the result as text and adds a line ending. It preserves the unconverted response body semantically, but it cannot serve as cryptographic byte evidence. Binary content types such as images, audio, PDF, ZIP, gzip, tar, WASM, and `application/octet-stream` are rejected. Redirects are followed up to 10 hops; the final normalized URL, without a fragment, becomes the index key. ## Crawl Strategies | Input | Behavior | | -------------------------------------- | --------------------------------------------------------------------------------------------------------- | | `llms.txt` or another `.txt` link list | Resolves Markdown links relative to the manifest and fetches each target. | | `llms-full.txt` | Fetches and indexes the entire response as one document. | | Ordinary `sitemap.xml` | Extracts page `` values and fetches each page. | | `sitemapindex` | Stops with a non-zero error and prints up to eight child sitemap URLs; it does not recurse automatically. | | Other URL | Fetches once, then auto-detects sitemap-like XML versus an llms-style list. | HTTP fetches use `MONOFETCH_CONCURRENCY` with a default of 8 and a minimum of 1. Index writes remain serialized inside one transaction. Individual fetch or index errors are counted while successful pages are committed, so a crawl can partially succeed. There is no robots.txt enforcement, host-aware rate limiter, or retry/backoff loop. Inspect unfamiliar manifests and choose considerate concurrency. ## Search and Deduplication `search` asks the shared Monomento engine for an over-fetched candidate window, filters that window to `doc_type=web`, and takes the requested limit. If no trigram results remain, it runs a web-scoped SQL substring fallback over names and content. This makes short and CJK queries practical, but it is not a guarantee that every lower-ranked web result survives the first candidate window. Index replacement is keyed by the final normalized URL and compares stored content. An unchanged page is skipped; changed content replaces the row. Fragments do not create separate entries. ## Static-Web Boundary MonoFetch does not execute JavaScript. When a page contains scripts but yields very little extracted text, the no-index path warns and recommends MonoSurf. There is no browser cookie jar or interactive authentication surface. Charset resolution is HTTP header, then an HTML meta declaration in the first 1 KB, then UTF-8. Unsupported labels fall back to UTF-8, so malformed or misdeclared pages can still produce mojibake. ## Storage MonoFetch uses Monomento's shared global database resolution: `$MONOMENTO_HOME/monomento.db` when explicitly set, then the current `~/.monolex/monomento.db`, then the legacy Monolex application-support path. Use `monofetch search` for the web-scoped view and [Monomento](/ai-clis/monomento) for cross-source search and section reads. ## Published Platforms | Platform | Binary | | ------------------- | --------------- | | macOS Apple Silicon | `monofetch` | | macOS Intel | `monofetch` | | Linux ARM64 | `monofetch` | | Linux x86-64 | `monofetch` | | Windows x86-64 | `monofetch.exe` | The public `0.3.15` metadata does **not** contain a Windows ARM64 artifact. ## Relationships | Source or need | Preferred tool | | -------------------------------------------------- | ------------------------------- | | Static page, Markdown endpoint, or docs manifest | **MonoFetch** | | JavaScript, authenticated page, or browser session | [MonoSurf](/monosurf/index) | | OpenAPI/Swagger JSON | [MonoAPI](/ai-clis/monoapi) | | Local document or media file | [MonoDoc](/ai-clis/monodoc) | | RSS, Atom, or feed entry import | [MonoRSS](/ai-clis/monorss) | | Wikipedia article | [MonoWiki](/ai-clis/monowiki) | | Cross-source search and reading | [Monomento](/ai-clis/monomento) |