hgstatic: README.md

# hgweb-static
 
Serve a directory of Mercurial repositories with `hg clone` support but a
fully static web rendering.
 
This became necessary for me due to the volume of requests my hgweb UI was
receiving from AI scrapers. The static HTML is much faster to serve, though
doesn't have all the features of hgweb. Currently it generates a repo index
with dates and descriptions, a rendered README for each project, and source
viewer for the latest (tip) revision, with line numbering and syntax
highlighting.
 
## How it works
 
While `hg` technically supports pulling a plain static repository served over
HTTP, this is less efficient than the Mercurial wire protocol, which e.g.
allows a hg client to discover and fetch only changesets that it doesn't
already have locally.
 
The core "trick" used in this project is that a normal `hg clone`/`hg pull`
over HTTP always request to repo root with a `?cmd=` query parameter (i.e. the
"smart" protocol). The web server can be configured to route those requests to
a minimal hgweb instance, but serves everything else as static files
regenerated by a push hook.
 
## Components
 
- `hgstatic.py`: the static site generator. Requires only `hg`; uses
  `pygments` (highlighting), `markdown-it-py` or `markdown`, and `docutils`
  (README rendering) when installed (falling back to `<pre>` without them).
  Settings are read from the command line or from a `hgstatic.ini` (see below).
- `examples/hgweb_wsgi.py` + `examples/hgweb.config`: wire-protocol-only
  hgweb, run under gunicorn (or any WSGI server). The config path comes
  from `$HGWEB_CONFIG`. `examples/hgweb@.service` is a per-domain systemd
  template for it.
- `examples/nginx.conf` / `examples/apache.conf`: the routing: `?cmd=` →
  hgweb, everything else → static files.
 
## Setup
 
Assumes repos in `/srv/hg` (nested directories are fine), static output in
`/var/www/hg`, site at `https://code.example.org`.
 
1. Install the generator and its optional deps:
 
   ```sh
   install -m 755 hgstatic.py /usr/local/bin/hgstatic
   apt install python3-pygments python3-markdown-it python3-docutils
   ```
 
2. Initial full generation (rerun any time to rebuild everything):
 
   ```sh
   hgstatic --root /srv/hg --out /var/www/hg \
       --clone-base https://code.example.org --title "my repositories"
   ```
 
3. Add a hook to regenerate on push. Hooks run with the repo as cwd, so one
   global snippet covers all repos, e.g. in `/etc/mercurial/hgrc.d/hgstatic.rc`
   (or per-repo `.hg/hgrc`):
 
   ```ini
   [hooks]
   changegroup.hgstatic = hgstatic --only "$PWD"
   ```
 
   `--only` regenerates just the pushed repo plus the top-level index.
   Given only `--only`, hgstatic locates an `hgstatic.ini` by walking up
   from the repo (also checking `config/` subdirectories), and exits
   quietly if there is none. The ini holds the settings from step 2:
 
   ```ini
   [hgstatic]
   root = /srv/hg
   out = /var/www/hg
   clone-base = https://code.example.org
   title = my repositories
   ```
 
   With the ini in place, step 2's full run is just
   `hgstatic -c /path/to/hgstatic.ini`. Command-line options can be used to
   override ini values if necessary.
 
4. hgweb for the wire protocol: copy `examples/hgweb.config` to
   `/etc/hgweb.config` and `examples/hgweb_wsgi.py` to `/opt/hgweb/`, then
   run it as the user owning the repos:
 
   ```ini
   # /etc/systemd/system/hgweb.service
   [Unit]
   Description=hgweb (Mercurial wire protocol)
   After=network.target
 
   [Service]
   User=hg
   RuntimeDirectory=hgweb
   ExecStart=/usr/bin/gunicorn --workers 2 \
       --bind unix:/run/hgweb/hgweb.sock --chdir /opt/hgweb \
       hgweb_wsgi:application
 
   [Install]
   WantedBy=multi-user.target
   ```
 
   The app reads its config path from `$HGWEB_CONFIG` (default
   `/etc/hgweb.config`). For several independent instances on one
   machine, use the `examples/hgweb@.service` template instead.
 
5. Web server, either:
   - nginx: adapt `examples/nginx.conf` (server name, TLS, paths).
   - Apache: adapt `examples/apache.conf`; needs
     `a2enmod rewrite proxy proxy_http`, and gunicorn bound to
     `127.0.0.1:8420` instead of the unix socket.
 
### Sympl
 
For a complete from-scratch walk-through on a [Sympl](https://sympl.io/)
host, including multiple domains on one machine, see
[docs/sympl.md](docs/sympl.md).
 
Pushes happen over ssh. HTTP push is disabled in `hgweb.config`.
 
## Notes
 
- Repos are listed newest-first; descriptions come from `web.description`
  in each repo's `.hg/hgrc`.
- A `README`/`README.md`/`README.rst`/`README.txt` at the repo root is
  rendered on the repo page.
- Binary files are listed but not rendered; text files over 512 KiB get a
  placeholder page.
- Each repo is generated into a temp directory and swapped in with a
  rename, so the live site never serves a half-written repo.

1	# hgweb-static
2
3	Serve a directory of Mercurial repositories with `hg clone` support but a
4	fully static web rendering.
5
6	This became necessary for me due to the volume of requests my hgweb UI was
7	receiving from AI scrapers. The static HTML is much faster to serve, though
8	doesn't have all the features of hgweb. Currently it generates a repo index
9	with dates and descriptions, a rendered README for each project, and source
10	viewer for the latest (tip) revision, with line numbering and syntax
11	highlighting.
12
13	## How it works
14
15	While `hg` technically supports pulling a plain static repository served over
16	HTTP, this is less efficient than the Mercurial wire protocol, which e.g.
17	allows a hg client to discover and fetch only changesets that it doesn't
18	already have locally.
19
20	The core "trick" used in this project is that a normal `hg clone`/`hg pull`
21	over HTTP always request to repo root with a `?cmd=` query parameter (i.e. the
22	"smart" protocol). The web server can be configured to route those requests to
23	a minimal hgweb instance, but serves everything else as static files
24	regenerated by a push hook.
25
26	## Components
27
28	- `hgstatic.py`: the static site generator. Requires only `hg`; uses
29	`pygments` (highlighting), `markdown-it-py` or `markdown`, and `docutils`
30	(README rendering) when installed (falling back to `<pre>` without them).
31	Settings are read from the command line or from a `hgstatic.ini` (see below).
32	- `examples/hgweb_wsgi.py` + `examples/hgweb.config`: wire-protocol-only
33	hgweb, run under gunicorn (or any WSGI server). The config path comes
34	from `$HGWEB_CONFIG`. `examples/hgweb@.service` is a per-domain systemd
35	template for it.
36	- `examples/nginx.conf` / `examples/apache.conf`: the routing: `?cmd=` →
37	hgweb, everything else → static files.
38
39	## Setup
40
41	Assumes repos in `/srv/hg` (nested directories are fine), static output in
42	`/var/www/hg`, site at `https://code.example.org`.
43
44	1. Install the generator and its optional deps:
45
46	```sh
47	install -m 755 hgstatic.py /usr/local/bin/hgstatic
48	apt install python3-pygments python3-markdown-it python3-docutils
49	```
50
51	2. Initial full generation (rerun any time to rebuild everything):
52
53	```sh
54	hgstatic --root /srv/hg --out /var/www/hg \
55	--clone-base https://code.example.org --title "my repositories"
56	```
57
58	3. Add a hook to regenerate on push. Hooks run with the repo as cwd, so one
59	global snippet covers all repos, e.g. in `/etc/mercurial/hgrc.d/hgstatic.rc`
60	(or per-repo `.hg/hgrc`):
61
62	```ini
63	[hooks]
64	changegroup.hgstatic = hgstatic --only "$PWD"
65	```
66
67	`--only` regenerates just the pushed repo plus the top-level index.
68	Given only `--only`, hgstatic locates an `hgstatic.ini` by walking up
69	from the repo (also checking `config/` subdirectories), and exits
70	quietly if there is none. The ini holds the settings from step 2:
71
72	```ini
73	[hgstatic]
74	root = /srv/hg
75	out = /var/www/hg
76	clone-base = https://code.example.org
77	title = my repositories
78	```
79
80	With the ini in place, step 2's full run is just
81	`hgstatic -c /path/to/hgstatic.ini`. Command-line options can be used to
82	override ini values if necessary.
83
84	4. hgweb for the wire protocol: copy `examples/hgweb.config` to
85	`/etc/hgweb.config` and `examples/hgweb_wsgi.py` to `/opt/hgweb/`, then
86	run it as the user owning the repos:
87
88	```ini
89	# /etc/systemd/system/hgweb.service
90	[Unit]
91	Description=hgweb (Mercurial wire protocol)
92	After=network.target
93
94	[Service]
95	User=hg
96	RuntimeDirectory=hgweb
97	ExecStart=/usr/bin/gunicorn --workers 2 \
98	--bind unix:/run/hgweb/hgweb.sock --chdir /opt/hgweb \
99	hgweb_wsgi:application
100
101	[Install]
102	WantedBy=multi-user.target
103	```
104
105	The app reads its config path from `$HGWEB_CONFIG` (default
106	`/etc/hgweb.config`). For several independent instances on one
107	machine, use the `examples/hgweb@.service` template instead.
108
109	5. Web server, either:
110	- nginx: adapt `examples/nginx.conf` (server name, TLS, paths).
111	- Apache: adapt `examples/apache.conf`; needs
112	`a2enmod rewrite proxy proxy_http`, and gunicorn bound to
113	`127.0.0.1:8420` instead of the unix socket.
114
115	### Sympl
116
117	For a complete from-scratch walk-through on a [Sympl](https://sympl.io/)
118	host, including multiple domains on one machine, see
119	[docs/sympl.md](docs/sympl.md).
120
121	Pushes happen over ssh. HTTP push is disabled in `hgweb.config`.
122
123	## Notes
124
125	- Repos are listed newest-first; descriptions come from `web.description`
126	in each repo's `.hg/hgrc`.
127	- A `README`/`README.md`/`README.rst`/`README.txt` at the repo root is
128	rendered on the repo page.
129	- Binary files are listed but not rendered; text files over 512 KiB get a
130	placeholder page.
131	- Each repo is generated into a temp directory and swapped in with a
132	rename, so the live site never serves a half-written repo.