Inspect and recover a run

Check status, inspect failure logs and retry failed stages without repeating completed upstream work.

Reconnect to running or failed jobs, inspect their logs and outputs, and retry failed stages without repeating expensive upstream calculations.

1. Check status by ID

Every submission prints a UUID you can use to reconnect from any terminal:

What you submitted	Command to check status
Single job	`ct-cli status JOB_UUID`
Parallel Flow	`ct-cli flow status FLOW_UUID`
Benchmark evaluation	`ct-cli benchmark status EVALUATION_UUID`
MD simulation chain	`ct-cli md status RUN_UUID`
TS search chain	`ct-cli ts status RUN_UUID`
DFT energy batch	`ct-cli dft status --batch BATCH_UUID`

Tip: Multi-stage workflows (md, ts, dft, flow) also print the individual JOB_UUID for each child stage in their status output.

2. Inspect logs and partial outputs before retrying

When a job fails, check its logs and any diagnostic files it wrote to results/ before submitting a retry:

Check which of these three cases occurred:

Still queued: No enabled worker in your target --pool and --platform has free capacity (see Connect a worker).
Scientific negative: The job ran cleanly, or ts-verify rejected a candidate geometry because the reaction has no barrier or the IRC went elsewhere. That is a valid scientific outcome—do not retry it expecting a different result.
Transient worker or network error: The calculation succeeded, or a downstream verify / save stage failed due to a temporary outage. Retry from the failed stage (Step 4).

3. Recover from a dropped network connection during submit

Before sending a job to the server, ct-cli saves the request locally and prints a request UUID. If your laptop loses Wi-Fi before the server replies, reconcile that exact submission without starting a duplicate job:

4. Retry only the failed downstream stage

When a multi-stage scientific workflow fails in a later step, reuse the completed upstream results instead of starting over from scratch:

Both commands keep all completed upstream evidence and return a new UUID for the retried suffix.

Note: For MD runs, if md-parse@1 or md-publish@1 fails after md-simulate@1 has already succeeded, do not re-run the GPU simulation. Submit the downstream Program with depends_on=[simulate_job_id] so it reuses the uploaded trajectory files.

5. Cancel a running job

Pressing Ctrl-C during --wait only detaches your terminal; the job continues running on the worker. To stop the job itself, run: