One test tells you about one flow. A suite tells you whether the application still works. In this exercise you will group your tests into an orchestration and run them as a single job. About ten minutes.
You need at least two tests in your project.
Step 1 — Create the orchestration
Open Orchestrations in the left sidebar and choose New Orchestration.
Fill in four things:
- Title. Name it after what it proves, not when it runs. Guest smoke ages better than Nightly 2am, which becomes a lie the first time someone changes the schedule.
- Projects. Pick your project first — the Tests field stays locked until you do, and will tell you so.
- Tests. Add your tests. They run as one job.
- Schedule. Leave it on On Demand for now.
Click Create.
Step 2 — Run it
The orchestration opens on its own summary — total runs, average time, how many tests it contains, and a Run History tab. Click Run; the button becomes Stop while the job is in flight.
Step 3 — Read the result
When it finishes, the summary fills in and the run appears in the history with its result, duration and who triggered it:
Two tests ran as one job in 1 minute 14 seconds. The history entry covers the whole job, and you can still open any individual test to see its steps, screenshots and data.
If a run you have just triggered does not appear straight away, reload the page.
Step 4 — Think about the schedule before you set one
The schedule options are On Demand, Hourly, Every 4 Hours, Every 12 Hours, Daily and Monthly.
Two things worth saying plainly, because they are the mistakes people actually make.
Frequent schedules cost runtime. An hourly suite of fifty tests is a real commitment. Start daily and tighten it only if the feedback is genuinely too slow.
A scheduled suite nobody reads is worse than no suite, because it manufactures confidence. Decide who looks at the result, and where they will see it, before you set the schedule — not after.
For this exercise, leave it on On Demand.
How to organize suites
| Smoke | Five to ten tests. The flows that mean the product is up at all. Run often. |
| Regression | The broad suite. Run daily, or before a release. |
| Release candidate | Everything, including slow edge cases. Run on demand, when it matters. |
Keep the smoke suite genuinely small. Its value is that a failure means something, and that depends on people trusting it enough to look.
Checkpoint
You are done when your orchestration shows one completed run in its history, with a duration, and you can open one of its tests and see that run's steps.
Try this
Break one of the tests using the method from Exercise 5, then run the orchestration again. Watch how a suite reports a partial failure, and notice you can still get from the suite result down to the exact red step.
What you learned
An orchestration turns individual tests into a signal about the application. The work is in choosing which tests belong together and who will act on the result.