Development¶
The repository¶
packages/sfmkit/ the library and CLI, in four layers
packages/viewer/ sfmview, in ports and adapters
packages/contracts/ sfmcontracts: what the two agree on, and all they share
projects/<name>/ one project and all it owns
data/ the photographs; never written to
configs/ one YAML per experiment
runs/<config>/ outputs, not in version control
runs/reference-*/ one frozen run of each kind, tracked and copied into the images
docs/ the long-form notes: the old photograph, optimisations, the viewer
website/ this site
animation-tools/ the scripts that draw the figures and the films. Not installed
Each package is installable on its own, with its own pyproject.toml,
requirements, Dockerfile, src/ and tests/. Commands are run from the
package's directory; the build context of every image is the repository root.
The checks¶
Before any commit, from the root:
and for the viewer, in its own environment:
lint-imports is the layering, enforced rather than described:
| Contract | |
|---|---|
| Layered architecture | apps → render → data → core, one direction only |
| The core does no I/O and draws nothing | no matplotlib, torch, yaml or rich in core |
| Only the render layer imports matplotlib | |
| Only the data layer imports torch |
The viewer's own contracts keep the API away from the adapters, and both
packages test against sfmcontracts/step.schema.json, so a change to the live
message format breaks a test on both sides rather than a run.
The regression check¶
The tests need no photographs. The pipeline's behaviour, though, is checked by re-running the stages on the project's frozen run, whose matches and COLMAP model come from a CPU:
cd projects/valencia
cp -r runs/reference-cpu runs/cpu # cpu.yaml writes `cpu`; start from the frozen one
for s in verify reconstruct localize evaluate; do
sfmkit $s --config configs/cpu.yaml
done
# 14 cameras, 2850 points, mean rotation error 0.338°, the old photo 1.32°
Because its COLMAP model is the frozen run's, the check scores against a fixed
reference: what moves is our code, not COLMAP's randomness. The frozen runs are
called reference-* for a reason — no config writes a run by that name, so
none of them can be overwritten by an ordinary run.
Conventions¶
- The backlog is
TODO.md. Every discovery goes there — a bug, a limitation, an idea, a question left open — with a line of context so it still makes sense a month later. - Documentation is prose. The long arguments live in
docs/:old-photo.mdfor the photograph the project is about,optimizations.mdfor what was measured and what it bought,viewer.mdfor the contract. - A result records its commit. Every stage writes a manifest with the commit, the versions and the whole config, so a figure can be traced to the code that made it.
- Figures and films are code.
animation-tools/draws every one of them from a run, so none of them can drift from the results they illustrate.
Continuous integration¶
.github/workflows/ carries four:
| Workflow | When | What |
|---|---|---|
Checks (ci.yml) |
main, pull requests | pip check on the installed pins, ruff, the import contracts and both test suites, with a Redis service so the broker's own tests run; then the saved example put through verify…evaluate again, and its numbers checked. A pull request's run is cancelled when it is pushed again; main's never is |
Documentation (docs.yml) |
main, pull requests touching website/ |
builds this site with --strict, and publishes it to GitHub Pages from main |
Docker images (images.yml) |
main, published releases, or by hand | builds the CPU and viewer images on every change to them; publishes them to the registry when a release is published, tagged with the version, cpu/latest and the commit. The GPU image is built where there is a GPU: make push DEVICE=gpu |
Notify (notify.yml) |
any of the three finishing | posts a failure to a Discord channel, and nothing when they pass |
Everything they install comes from the same pinned requirements files as the conda environments, so a green CI means the documented installation works.
Because that installation is --no-deps, pip is never asked whether a pinned
version suits the package that needs it, and a wrong pin only shows up as an
import error in the middle of the tests. So each job runs pip check straight
after installing: a disagreement is then one line naming both packages. The one
disagreement allowed is lightglue asking for opencv-python where
opencv-python-headless is installed, which is the same library without the
GUI.
Where a failure is heard¶
GitHub's own route is email, and it goes to the same inbox as everything else.
So the email is turned off — https://github.com/settings/notifications,
Actions, uncheck Email (or leave it on and set Notify for failed
workflows only) — and notify.yml posts failures to a Discord channel
instead, where they can be muted, read late, or left to the phone.
It needs one repository secret:
# Discord: the channel's Edit Channel -> Integrations -> Webhooks -> New
# Webhook -> Copy Webhook URL. Then, in a clone of this repository:
gh secret set DISCORD_WEBHOOK
Without the secret the job says so and passes, so a fork is never failed by a
secret it cannot have. It posts on failure only: a green run is not news.