Troubleshooting

Error messages you can meet on a new TPU chip, collected from each guide’s Troubleshooting table. Longer answers are in the FAQ.

Run on Google Cloud TPU

From Run on Google Cloud TPU:

What you see	What to do
an error that mentions quota when you create the chip	request TPU quota for that zone (step 5)
no capacity in the zone	try the next zone in the step 6 table, or queue a request
the TPU API is not enabled	turn it on (step 4)
the VM instance button opens a Compute Engine VM form	that is a different product; use the Cloud TPU API tab, then Create TPU (step 6)
SSH in the browser asks you to authorize	click Authorize; it asks once (step 7)
There is no more capacity in the zone "us-east5-b"	no free chips there right now: create the chip in the next zone of the step 6 table and use that zone in every later command, or queue a request
User does not have permission to submit requests for accelerator type "v6e-1" in location ...	your project has no on-demand quota for that chip in that zone: pick a zone from the step 6 table, or ask for quota there (step 5)
ERROR: No matching distribution found for jax[tpu]==0.11.2	plain pip install "mashbox[tpu]" ran on the chip’s Python 3.10; use the install line from step 7, which gets Python 3.12 through uv
mashbox: command not found	use the full path ~/.local/bin/mashbox, or log in again so ~/.local/bin is on your PATH
429 Too Many Requests from cdn.hf.co during the download	Hugging Face’s Xet transfer is limiting the download: run the command again. mashbox downloads over plain HTTPS unless HF_HUB_DISABLE_XET=0 is set
The TPU is already in use by another process	one process at a time can use the chip: stop the other one (for example a server in another SSH session) and run again
<model> is not in mashbox <version>	that model is not in the installed version: pick one from the Models pages

Hy-Nomas on TPU

From Hy-Nomas on TPU:

What you see	Why	What to do
There is no more capacity in the zone or Insufficient capacity when you create the chip	that zone has no free v6e-4 right now	run the create command again in another zone, and use that zone in every command after it; or queue the request (see Install)
User does not have permission to submit requests for accelerator type "v6e-4" or Quota limit 'TPUV6EPerProjectPerZoneForTPUAPI'	your project has no on-demand TPU v6e quota in that zone	ask for 4 cores of “TPU v6e cores per project per zone” in that zone (step 5 of the setup guide) or pick a zone where you have it
ERROR: No matching distribution found for jax[tpu]==0.11.2	plain pip install ran on the chip’s system Python 3.10; JAX for TPU needs Python 3.12	use the install lines in Install, which get Python 3.12 through uv
hy-nomas needs a TPU v6e-4 (4 chips); found ...	the machine has no TPU or fewer than 4 chips	create a v6e-4 (Install), or use --stub to try the app on CPU
... sha256 ... does not match the pinned ...	a download was cut off or changed	delete the file named in the message and run again
... code hash ... does not match the pinned ...	Tencent’s code folder in the cache is incomplete or changed	delete the folder named in the message and run again
a log line "event": "disk" with free_gb below missing_gb	the cache disk has less free space than the files still to download	set HF_HUB_CACHE to a folder with more room (see What you need) and run again
Unable to initialize backend 'tpu': ... START_SESSION failed when mashbox starts	the chips could not be linked into one slice. It can follow a start 0.84 s after the previous run exited, or a server stopped with kill, and can last through 25 minutes and a reboot	stop the server with Ctrl-C, not kill, and wait 10 s after it exits before the next start. If it happens, delete the chip and create a new one
could not download the kernel bundle from https://huggingface.co/mashbox-labs/Hy-Nomas-TPU ...	the VM could not reach Hugging Face, or Hugging Face refused the request	check the VM’s network and run again; --no-kernels runs without the bundle, in plain JAX (slower)
TPU is already in use by process with pid ...	another mashbox server is still running on the VM	stop it with Ctrl-C in its terminal (or kill -INT <pid>), then start again
mashbox: command not found	~/.local/bin is not on your PATH until you log in again	use the full path ~/.local/bin/mashbox
ModuleNotFoundError: torch (or transformers, yaml)	the hy-nomas extra is not installed	run the uv tool install line in Install again
a message that the text is too long	texts longer than 512 tokens are refused	shorten the world or action text
the picture stops for a moment and then jumps ahead, on a slow connection	the video stream is about 6 Mbit/s. When your connection carries less, the app drops the video it cannot deliver in time and restarts from the next key frame, so the picture stays current instead of falling seconds behind. The readouts under the picture show link, in flight and dropped for network	use a faster connection, or a chip in a zone near you
Host not allowed or Origin not allowed (HTTP 403) in the browser	the page was opened under a host name other than localhost or 127.0.0.1 (your own tunnel or proxy with a domain name), or a page of another site sent the request; the app refuses both so no other site can drive or watch your session	open http://127.0.0.1:8790 through the SSH tunnel, or start with MASHBOX_ALLOW_HOSTS=your.host.name when you use your own tunnel or proxy
missing or wrong session token (HTTP 401)	a proxy in front of the app dropped the X-Mashbox-Token header or the token part of the stream address; the page sends both, with a token made at each start	pass both through unchanged

Qwen3.8-27B on TPU

From Qwen3.8-27B on TPU:

What you see	Why	What to do
one line saying the model runs on plain JAX	the kernel bundle was not built for this chip or JAX version, or --slots is not 8	use TPU v6e and the default --slots 8; ~/.local/bin/mashbox env shows the chip and versions
out of memory	another process holds the chip, or --ctx/--slots are above the defaults	run on a TPU v6e-1 with nothing else on it, with --ctx 2048 --slots 8
sha256 ... does not match the pinned ...	a download was cut off or changed	delete the file and download again, as the message says
ERROR: No matching distribution found for jax[tpu]==0.11.2	plain pip install "mashbox[tpu]" ran on the chip’s Python 3.10; JAX for TPU needs 3.12	use the install line in Install
mashbox: command not found	~/.local/bin is not on your PATH until you log in again	use the full path ~/.local/bin/mashbox
429 Too Many Requests from cdn.hf.co, then a traceback	Hugging Face’s Xet transfer is limiting the download	run the command again; mashbox downloads over plain HTTPS unless you set HF_HUB_DISABLE_XET=0, so remove that setting if you added it
The TPU is already in use by another process	one process at a time can use the chip, for example a server you started in another SSH session	stop that process (Ctrl-C in its session) and run again; if sudo lsof -w /dev/vfio/0 shows no process, the lock file is left over: sudo rm /tmp/libtpu_lockfile, as the message says
There is no more capacity in the zone when you create the chip	no free TPU v6e chips in that zone right now	create it in another zone from step 6 of the setup guide and use that zone in every later command

Track2Rig

From Track2Rig:

Error	Fix
The Track2Rig weights are not on Hugging Face at a pinned revision in this version of mashbox	pass a local copy with --weights track2rig-v2.safetensors
sha256 ... does not match the pinned ...	the file is damaged or is another file; delete it and download again
tracks has T=... frames; max 176	split or resample the clip
frame 0 is the bind pose and must be fully visible	drop the points not seen at frame 0, or start the clip at a frame where all are seen
frame t->t+1: median point jump ...	a camera cut or a lost subject; cut the clip before that frame
... bytes uncompressed; the most a tracks file needs is ...	the archive holds arrays far larger than a valid tracks file; check what wrote it
--fps: must be a finite number above 0	give the clip’s real frame rate
REFERENCE ERROR: ... after eval	the reference file is missing or does not hold one complete row per clip and method; pass the shipped reference.json with --reference, or the clips it was recorded for
OVER TOLERANCE after eval	a result moved further from the recorded row than the published tolerance; report it with the output of mashbox run track2rig info