Local AI & HardwareStats for Nerds
100% bare-metal, local-first AI. Everything that screens, classifies, tracks and writes about the backyard animals runs on our own machines inside our house. There is no corporate cloud camera service in the loop, and the privacy checks happen on our hardware before anything can reach this site. This page is the look under the hood.
Public Access • Specs as last checked 28 October 2026 • ← Back to the Neighbor Note & Disclaimer
*The one exception is the Peanut Heist, our overnight camera story: its edited cut is sent to a cloud AI service to be described, and its narration voice also comes from a cloud service. Nothing else on this page leaves the house to be analyzed.
The Privacy & Wildlife Vision Pipeline
The journey of a camera event, from sensor to site. A clip that fails any gate, or that a gate cannot check, is held back rather than published.
- Hardware animal triggerThe UniFi cameras tag motion in hardware, and we only pull events they tag as an animal; person and vehicle events are ignored from the start. The moving camera in the enclosed back yard is steered by local Frigate object detection, held inside strict angle limits. If a person walks in, it stops, turns to a parked view and waits until the yard has been clear for a few minutes.
- People & vehicle screen Gate 1On every camera that can see the street or the front of the house, a local vision-language model looks at each candidate frame and answers one question: is a person or a vehicle in it? A yes, or any doubt, or a screen that cannot run, quarantines the clip. Nothing from the doorbell or security-only cameras is ever published.
- License plate redaction Gate 2On front-facing views where parked cars can appear, a vehicle detector scans the clip in tiles so distant cars are not missed, and the area where plates live is blurred before publication. If the blur cannot be verified, the clip is withheld.
- Audio stripping Gate 3FFmpeg removes the sound track from every wildlife clip before it is published.
- Species & recurring visitorsBioCLIP 2 names the species on PyTorch and ROCm, a local vision model describes the scene, and embedding models help tell recurring visitors, the neighborhood cats especially, apart.
- Field Dispatch & storytellingValidated sightings are indexed into a local Vespa vector lake using
nomic-embed-textembeddings. Local language models write the daily field reports and character dossiers, and a separate fact-check pass reads each draft against the sightings before it is accepted.
The GPU Cluster & How Calls Are Routed
Three GPU machines, one CPU machine and a rulebook. Calls go through one local gateway and are routed by the size of the job, not by a fixed pecking order.
Catalog & vision heavyweight
Intel Core i7-9700K, 46 GB RAM, NVMe storage. Leads large-model prose, vision classification and BioCLIP, renders images on demand, and hosts the Vespa vector lake. It rests during a nightly quiet window (21:00 to 01:00 Central by default) so its fan stays quiet, and the other nodes cover.
Fast routine & reasoning lead
One model resident at a time, kept warm for ten minutes between calls. Leads quick routine queries and long-context work, and answers while the heavyweight is resting.
High-throughput fallback
Intel Core i9-10900F, 32 GB RAM. The secondary GPU, and also a gaming PC: when a 3D game launches it yields automatically and traffic moves to the other nodes.
Always-on CPU & camera host
Intel UHD 770 with Quick Sync, 24.6 TB of storage. Runs Frigate for the yard camera, with OpenVINO detection on the integrated GPU. It can run a small model such as gemma:2b on the CPU for jobs that ask for it, but it is not part of the normal route.
Deterministic rulebook
The last stop. Every narrative generator has a rule-based fallback that never invents facts, so if every GPU is busy or offline the site publishes plainer, honest copy instead of nothing.
| Kind of job | Tries first | Then |
|---|---|---|
| Routine and quick | Fast routine node | Fallback node, then the heavyweight |
| Large models, code review, vision | Heavyweight | Fast routine node, then the fallback node |
| Every model unavailable | Rulebook | No model involved |
Vision, Re-ID & Image Models
Not everything is a chat model. These run on the same hardware and do the specialist jobs.
| Model | What it does here |
|---|---|
| MegaDetector | Finds animals, people and vehicles in a frame without naming species. Used in our reviews of animals the cameras missed and in the featured-clip checks. |
| BioCLIP 2 (ViT-L/14) | Fine-grained species and taxonomy classification, run natively on PyTorch and ROCm on the 32 GB card. |
| License plate redactor | A tiled vehicle detector plus a blur pass over each clip. It fails closed: a clip that was not checked is not published. |
| Re-identification embedders | Visual feature vectors, including an animal-tuned DINOv2, that help recognize recurring visitors. |
| Frigate with OpenVINO | Animal detection for the yard's moving camera, on an Intel integrated GPU. |
| SDXL and FLUX diffusion (ComfyUI) | Storyboard illustrations. Starts on demand on the 32 GB card and gives all of its video memory back after ten idle minutes. |
| BirdNET (not running yet) | Planned: name birds by their calls, keeping only the names and times and never the recordings. The yard microphone is installed; listening is not switched on. |
The Language & Vision Model Fleet
The open-weight language and vision models we run across the three GPUs, and what each does for the Observatory. Only one large model loads at a time on a card.
| Model | Size | Job |
|---|---|---|
qwen3.8:27b, deepseek-r1:14b | 17 GB and 9 GB, Q4_K_M | Planning and deep reasoning: the Dispatch's outline pass and logic-heavy questions. |
fluffy/magnum-v4-22b, magnum:27b | 13.3 GB Q4_K_M and 19 GB Q5_K_M | Creative writing: Field Dispatch editions and the character dossiers. |
mistral-small:24b | 14 GB, Q4_K_M | The fact-check pass that reads a draft against the sightings. |
qwen2.5vl:7b, minicpm-v:8b | 6.0 GB and 5.5 GB, Q4_K_M | Seeing: the people and vehicle screen, scene descriptions and frame inspection. |
codestral:22b-q5_k_m, qwen2.5-coder:14b | 15 GB each | Reviewing our own site code before it ships, on local hardware at zero cloud cost. |
qwen3.5, llama3.2:3b | 6.6 GB and 2.0 GB | Fast daily driver: per-event notes, JSON parsing and small jobs. |
gemma:2b | 1.7 GB | A tiny model that runs on a CPU when a job asks for one. |
nomic-embed-text | 274 MB, F16 | Embeddings for the Vespa vector lake and semantic search. |
Security & Supply Chain
Keeping the animals' site safe is part of keeping the neighbors' privacy safe. The short version:
- No open door at the house. The public site is reached only through an outbound-only Cloudflare Tunnel. We do not open an inbound port for it.
- Members' pages sit behind a login. The VIP pages are gated by Cloudflare Access, and a request that is not allowed in gets a plain “not found”, not a hint that something is there.
- No tracking. Public pages set no cookies and run no analytics or advertising scripts. The privacy section of the disclaimer lists the little that is loaded.
- Containers are pinned and scanned. Every third-party container image is pinned to an exact digest, and the images are scanned for known vulnerabilities with Trivy, the open-source container scanner. Nothing loads from that site until you click the link.
- A publish gate that fails closed. Before an edition goes live, every file is scanned for internal code names and private names, and only camera media may sit in the snapshot folder. If a check fails, the whole edition is refused and nothing goes live.
- Code is checked before it ships. Changes are reviewed by a local model and must pass automated tests before they deploy, with no cloud tokens spent on the review.
About These Numbers
This page is a snapshot of the hardware and models as we last checked them, not a live dashboard. We do not print run counts here yet. When we do, they will be measured values, and where we have not measured something, we will say so instead of showing a zero.