Managing Orchestrations: Filters, Bulk Actions and Run History

Building an orchestration takes two minutes. Living with a few hundred of them is the part nobody writes down. This page covers the orchestrations list — how to find things in it, what you can do to several at once, what each row's menu offers, and what the run history tells you after that.

The orchestrations list: filters, select mode and everything one row can do

What the list shows you

Each row carries more than its name. Reading left to right:

Status mark A green or red mark on suites that have run. Hover it and Studio tells you the proportion — "All executions passed", or "4 of 10 executions failed". That second number is the one worth reading: a suite failing 4 times in 10 is a different problem from one that failed once.
Schedule mark Says On Demand or names the cadence, so you can see at a glance which suites are firing without you.
Last Run / Created Both sortable. Together they answer "is this thing still alive?"
Duration The last run's wall-clock time. A suite whose duration has crept from four minutes to twenty is telling you something before any test goes red.
Tests / Projects How many tests, and which projects they came from. A suite can draw from more than one project.
Owner Initials of whoever created it. In a shared account this is often the fastest way to work out who to ask.

Finding one among hundreds

Three controls sit above the table.

Search orchestrations Matches on the title. This is why a naming convention pays for itself — a prefix like Checkout - turns a search box into a folder.
Status Seven values: Processing, Created, Completed, Passed, Failed, Warning, Incomplete. Created means it has never run at all, which makes it the best filter in the list for finding suites somebody built and forgot.
Schedule Two values: On-demand and Scheduled.
The quarterly audit, in one filter
Filter Schedule to Scheduled and sort by Created, oldest first. Everything near the top is a job that has been running on a timer for a long time. Some of it is load-bearing. Some of it is spending credits on a question nobody is asking any more.

Select mode: doing something to several at once

The Select button turns every row into a checkbox and replaces the search and filter row with a count and four actions.

Run Starts every ticked orchestration. This is the fastest way to fire a release check across several suites — and the fastest way to spend a great many credits without meaning to. Count what you have ticked first.
Stop Halts runs that are in flight.
Delete Removes the orchestrations. The tests themselves are untouched; an orchestration is a grouping, not a container.
Cancel Leaves select mode and brings the filters back.

A header checkbox selects everything on the current page. Since the page size is whatever Rows per page says, that is ten rows by default rather than every orchestration in the account — but it is still worth a second look before pressing Run.

What one row can do

The … at the end of a row opens eight items.

Run Starts the job now, whatever its schedule says.
Clone Copies the whole configuration — projects, tests, schedule, run order, re-runs, alerts. This is the most useful item in the menu. A second suite that differs in one setting is a clone and one edit, not a rebuild.
Edit Reopens the same panel you created it in, with everything filled in. Changing the schedule or the advanced settings later is normal and costs nothing.
Open in new tab / Copy link address An orchestration has a stable URL. Pasting it into a ticket or a runbook is better than writing "the nightly one".
Execution Presets Chooses which data set each project uses when this suite runs — see Execution Presets: Running One Suite Against Different Data.
Copy title The exact name, for when you are referring to it somewhere else and want it to match.
Delete Removes the orchestration and its history. The tests survive.

Reading the run history

Opening an orchestration that has run gives you a summary panel: Total Runs, Avg. Time and Tests across the top, and two tabs underneath.

Run History One row per run: result, duration, how long ago, and who triggered it — a scheduled run and a run somebody started by hand are told apart here. Each row opens the individual test results underneath it.
Tests What is currently in the suite: test name, the project it came from, and when it was created. Worth checking when a suite has been around long enough that nobody is quite sure what is in it.

Avg. Time is the number people underuse. It moves slowly and then, one week, it does not — and a suite that has gone from 90 seconds to 4 minutes is usually telling you about the application rather than about itself.

Drilling into a run takes you to that test's own evidence: steps, the browser pane, and the Data tab with its Previous / Current / Generated comparison, exactly as for a single run. See Understanding a Test Run and Its Results in Functionize Studio.

The header of the panel carries Run and a More options menu with Clone, Edit, Execution Presets and Delete — the same actions as the row menu, minus the link and copy items.

Three habits worth forming

  • Name suites so search works. A prefix per application or per team turns a thousand-row list into something navigable. See Scaling From One Team to Many.
  • Clone rather than rebuild. Most second suites are a first suite with one thing changed.
  • Audit Scheduled once a quarter. Nothing else in Studio will tell you that a job has been running nightly for a year for a team that disbanded.

Where to go next

Download this article
A PDF of this page, for reading offline, printing, or passing to somebody who does not have a Studio account yet.
Managing Orchestrations: Filters, Bulk Actions and Run History (PDF, 274 KB)