Watchers
Run a watcher
Start laplace watch: re-run every answer and challenge the ones that differ.
laplace watch is a node that runs every answered receipt again and, when its output differs from the one posted, opens the challenge and plays the game to its end with nobody at the keyboard. This guide covers the key and the SOL it needs, what it follows, what it does on a difference, what it earns and how to keep it running.
What a watcher needs
| The CLI and the native step function | The laplace command and the laplace-vm binary: Install and invoke, The native laplace-vm binary. |
| The network's addresses | LAPLACE_RPC and LAPLACE_API, or --rpc and --api: Addresses. |
| A key | Any Solana keypair in the CLI's key file. A watcher is not registered anywhere: it needs no Operator account and no offer. |
| SOL in the key's wallet | For each challenge: a bond equal to the model's, the rent of the Dispute account, and the fees of the challenge and of every turn. |
A challenger's bond is taken from the wallet by challenge, not from a deposited balance. The bond of a model is bondLamports of GET /v1/models/:slug, shown on /models/<slug> as its bond per answer. To have n challenges open at once on a model, the wallet holds n times that bond.
Start it
Make or import the key
Shell laplace key newOr
laplace key import <keypair.json>for a keypair that already exists. Either prints the public key. The key file covers where it is kept.Fund the key
Send SOL to the public key: the bond of the models it will watch, with room for rent and fees.
Start watching
Shell laplace watch --models judge,marketIt asks for the key's password once, prints a
watchline, fetches each model'sprogram.binandweights.binfrom the API's store into its data folder (gzip-compressed on the wire), checks each against the SHA-256 the API lists for it, and printsmodels_ready. Then it starts its passes.
Which receipts it follows
Every 2 seconds, for each model in --models, the watcher reads every answered receipt from the API's index, 100 to a page, following nextCursor to the last page:
GET /v1/receipts?model=<slug>&state=answered&limit=100&cursor=<nextCursor>Each receipt is checked once per answer. The watcher remembers a receipt together with the trace root it saw, so a receipt whose answer was replaced after a dispute is checked again with its new answer. It skips receipts whose standing answer is its own.
The index only says where to look. The receipt the watcher compares against is the account read from the chain.
The re-run
For each receipt the watcher starts laplace-vm run on the model's files with the receipt's input tokens and seed, and compares two things with the receipt: the output, token by token, and the trace root.
| The run gives | The watcher | Prints |
|---|---|---|
| The same output and the same trace root | Does nothing more. | matched |
| A different output | Opens a challenge, if the receipt's window is still open. | challenged |
| The same output and a different trace root | Sends nothing. A challenge whose output equals the answer is refused with SameOutput. | trace_differs_output_same |
The run's state roots are kept in a file under the data folder for as long as the watcher holds the receipt, so every later turn is answered from it.
On a difference
The challenge
The watcher sends one
challengetransaction with its output, its trace root and its first points over the whole run. The program takes the model's bond from the watcher's wallet into the vault, opens the dispute and sets the receipt tochallenged. The watcher prints achallengedline with the receipt, the dispute's address, the first tokens of both outputs and the signature.Every turn
In a loop of its own beside the re-runs, every second, the watcher reads
GET /v1/disputes?operator=<key>, which returns the disputes its key is a side of, and for each one that is not resolved reads the Dispute account from the chain and computes its move from its run:- on its turn, while the interval is wider than one step, a
bisect: the first part of the other side's points it disagrees with, and its own state roots inside that part. Printsbisect. - while it waits for the other side, nothing.
- on its turn, while the interval is wider than one step, a
The proof
When one step is left, the watcher builds the step's payload with
laplace-vm proveand sendsprove_stepas a v1 transaction, beside the network's keeper, which proves it too. It does this whichever side's claim is being tested, since the program's result stands whoever sends the proof. Printsprovedwith the step and the payload's size.Timeouts
When the other side is on turn and its deadline has passed on the chain's clock, the watcher sends
claim_timeout, and the other side loses as if proven wrong. Printsclaimed_timeout.
A move the program refuses prints play_failed with the error's name, and the next round computes the move again from the dispute as it stands. The keeper and each side's node may all send the proof of the last step: the first to land settles the dispute and the others are refused with WrongState.
The challenge game has the rules each of these moves follows.
What it earns
When the watcher's run is the model's, the step on chain agrees with it and the dispute resolves for the challenger. Two payments follow.
| When | What arrives |
|---|---|
At resolution, in prove_step or claim_timeout | slash_winner_bps of the loser's bond, to the watcher's wallet. The rest of that bond goes to the protocol's fee recipient. |
At finalize, after the fresh window | The watcher's own bond, from the vault back to its wallet, and the receipt's fee less the protocol share, to its associated token account for the receipt's mint. |
Between the two, the watcher's output and trace root stand on the receipt as its answer, with the watcher's bond locked behind them and a full window open in which anyone can challenge it in turn. The fee is the watcher's because its answer is the one that stands.
The current split is protocol.slashWinnerBps of GET /v1/network, and the share of the fee is protocol.shareBps. A lost bond describes the split.
Running it as a service
The password is given as LAPLACE_PASSWORD, since a service has no terminal to ask in:
export LAPLACE_RPC="$rpcUrl"
export LAPLACE_API="$apiUrl"
export LAPLACE_KEY=/var/lib/laplace/key.json
export LAPLACE_PASSWORD="$password"
export LAPLACE_DATA=/var/lib/laplace/watch
export LAPLACE_VM_BIN=/usr/local/bin/laplace-vm
node dist/src/cli/main.js watch --models judge,marketLAPLACE_KEY_SEALED takes the key file's contents in place of LAPLACE_KEY, for a host that keeps secrets as variables. Global flags and variables lists each one.
The watcher writes one JSON line per event to standard output, keeps running through a failed pass (an error line) and a failed check (check_failed), and ends its loop on SIGINT or SIGTERM after the current pass.
After a restart it checks the newest answered receipts again from the start, and it goes on playing the disputes it had open: for each one it runs the receipt's input again to rebuild its run, then moves.
Seeing what it does
- Its own log:
matchedfor every receipt it agreed with,challenged,bisect,provedandclaimed_timeoutfor a dispute. What it prints lists every line and its fields. - A dispute it opened:
/disputes/<address>, with the address from thechallengedline. - Its wallet's balance, which rises by the winner's part when a dispute resolves for it.