Operators

Answering and deadlines

What the node does with a request, the answer deadline, and getting paid at finalize.

laplace serve turns a request into an answer without anyone at the keyboard. This page follows one request through the node: how it is found, run and answered, what happens at the answer deadline and when the free bond is short, when the fee arrives, and what the node does if the answer is challenged.

One pass of the node

After its models are on disk, the node repeats one pass every 1.5 seconds.

  1. Find the requests that name this key

    The node asks the API's index for them:

    Text
    GET /v1/receipts?operator=<key>&state=requested&limit=20

    It keeps the ones whose model is in --models. The index only says where to look. Everything the node acts on is read from the chain.

  2. Read the receipt from the chain

    For each one, in turn, the node reads the Receipt account and continues only if its state is requested and its operator is this key.

  3. Check the answer deadline

    It compares the receipt's answer_deadline with the block time of the latest confirmed slot. A receipt whose deadline has passed is left alone.

  4. Run the model

    The node starts laplace-vm run on the model's files with the receipt's input tokens and seed. The run gives the output, the trace root, and a file with the state root after every step, which the node holds for as long as the process lives so it can play a dispute over this answer without running again.

  5. Send the answer

    One answer transaction, signed by the key, carrying the output and the trace root. It is simulated first, then sent, then followed until it is confirmed. The node prints an answered line with the receipt, the model, the first tokens of the output and the signature.

  6. Play its games

    At the end of the pass the node reads GET /v1/disputes?operator=<key> and, for every dispute of its own that is not resolved, sends its next move. See When an answer is challenged.

Requests are answered one after another, in the order the API lists them, newest first.

What answer does on chain

answer takes two arguments, output and trace_root, and in one instruction:

  • moves the model's bond from the operator's bond_free to bond_locked and records it as the receipt's bond,
  • writes the output, the trace root and the answerer on the receipt,
  • sets answered_at to now and window_ends_at to now plus the model's window,
  • sets the receipt's state to answered,
  • adds one to the operator's answered count and sets its last_answer_at.
Refused withCodeWhen
WrongState6010The receipt is not requested.
NotNamedOperator6011The signer is not the operator the request named.
AnswerDeadlinePassed6012The clock is past answer_deadline.
InvalidOutput6009The output's length is not the model's output_len.
InsufficientFreeBond6003bond_free is smaller than the model's bond.

Retries

When the run or the transaction fails, the node prints an answer_failed line with the reason and moves on. The receipt is still requested, so the next pass, 1.5 seconds later, finds it and tries again. This repeats until the answer lands or the deadline passes. A run that completed is kept, so only the transaction is sent again.

A pass that fails as a whole, for example because the API did not answer, prints an error line and the next pass runs as usual.

The answer deadline

request sets the receipt's answer_deadline to the moment it lands plus answer_deadline_secs of the Config account. The API returns it as answerDeadline on the receipt, and the receipt's page counts it down as "to answer".

  • Up to and including the deadline, answer is accepted.
  • After it, answer fails with AnswerDeadlinePassed, and the node does not send one: it skips a receipt whose deadline is behind the chain's clock.
  • The requester can then send cancel_request, which returns the whole fee and makes the receipt void. The operator's bond is not touched, because none was locked.

A node that was stopped answers on its next start every request that is still inside its deadline.

When the free bond is short

If bond_free is smaller than the model's bond, the answer transaction fails in simulation with InsufficientFreeBond. Nothing is sent and no fee is spent. The node prints an answer_failed line whose message is The program refused it: InsufficientFreeBond. and tries again on each pass until the deadline. The answer goes through as soon as the free bond covers it: after a deposit, or after one of the operator's open receipts is finalized and its bond returns to free. Deposit bond is the task.

What live means

Every offer the API returns carries live. The API sets it to true when all of these hold:

  • the offer is active,
  • the operator's bond_free is at least the model's bond,
  • no request to this operator for this model is still requested past its answer deadline, counting only requests made after the operator's last answer.

GET /v1/offers?model= lists live offers first, then by price. The SDK's laplace.ask(model, input) and laplace ask take the first offer that is active and live, and Ask preselects it. An operator's page marks each active offer "live" or "not answering right now".

So an operator drops out of the default choice when it runs out of free bond or lets a request expire, and returns when it deposits, when the expired request is cancelled, or when it answers another request. A requester can still name any active offer directly.

Getting paid at finalize

The fee waits in escrow while the window is open. Once the window has ended with no dispute open, finalize pays it.

Who sends itAnyone. The network itself sends it for every receipt whose window has closed. The receipt's page shows Finalize once the window has closed, and the SDK has laplace.finalize(receipt).
The feeprotocol_share_bps of it, rounded down, goes to the protocol's fee recipient. The rest goes to the answerer's associated token account for the receipt's mint.
The bondThe receipt's bond moves from the operator's bond_locked back to bond_free.
The receiptState final.

The current share is protocol.shareBps of GET /v1/network. Where the fee goes covers the escrow.

TypeScript
import { createLaplace } from '@laplace/sdk'

const laplace = createLaplace({ rpcUrl, apiUrl, wallet })

const signature = await laplace.finalize(receipt)

laplace.finalize first opens the answerer's and the fee recipient's token accounts for the mint if they do not exist, paid by the sender, then sends finalize.

Refused withCodeWhen
WindowOpen6015The window has not ended.
WrongState6010The receipt is not answered: it is challenged, or already final.

The operator does not have to do anything to be paid, and the node sends no finalize itself.

When an answer is challenged

A challenge puts the receipt in challenged, pauses its window, and makes the operator the defender of a dispute. The node plays its games in a loop of their own, beside the answers, reading its disputes every second, so no burst of requests holds a turn. It finds the dispute within a second and plays it:

  1. It reads the Dispute account from the chain.
  2. It takes the run it holds for that receipt. After a restart it runs the receipt's input again first.
  3. It computes its move from that run and sends it: a bisect while the interval is wider than one step, prove_step as a v1 transaction when one step is left (the network's keeper proves it too, and the first proof settles it), claim_timeout when the challenger is on turn and its deadline has passed.

Each move prints a bisect, proved or claimed_timeout line. A move the program refuses prints play_failed with the error's name, and the next round computes the move again from the dispute as it then stands.

Every turn has a deadline, turn_deadline_secs of Config after the previous turn. The node moves within a second of the other side, so a running node stays inside it.

The dispute endsFor the operator
The operator winsIts wallet receives slash_winner_bps of the challenger's bond. The receipt returns to answered, its window resumes with the time it had left, and the fee is paid at finalize as above. The operator's disputes_won rises by one.
The operator losesThe receipt's bond leaves bond_locked and is split between the challenger and the protocol. The challenger's answer stands on the receipt and the fee goes to the challenger at finalize. The operator's disputes_lost rises by one.

Playing the game describes each turn from a side's seat, and A lost bond the split.