Parallel legs (start several, then wait for them)

Summary#

Some work has to happen together. Booking a trip means a flight and a hotel; neither is worth having on its own, both take time, and doing them one after the other means the customer waits twice as long for an answer.

Inside a Workflow.BeginSaga (a compensating saga scope) scope, a saga.Run("step", ...) written without await starts a leg and carries on. It hands back a leg handle, and await Workflow.WhenAll(...) waits for the legs you name. The distinction is the same one C# draws between var t = Foo(); and await Foo(); — start, versus start and wait.

Each leg keeps everything that made a sequential step readable: it is created by ordinary statements above the call, and its compensation sits on the call that starts it. Nothing has to be squeezed into an argument.

When the legs are many of the same kind — one per order line, say — build the list yourself and hand it to the same wait: await Workflow.WhenAll(legs). You still write the loop, the creation and the undo; only the count stops being something the source knows.

Signature#

var leg = saga.Run("step", step, () => Undo(step));   // START a leg — no `await`

await Workflow.WhenAll(a, b, c);              // every leg
await Workflow.WhenAny(a, b, c);              // the first to succeed
await Workflow.When(2, a, b, c);              // enough of them

await Workflow.WhenAll(legs);                 // …or a LIST of legs, when the count is only known at run time
await Workflow.When(2, legs);

await Workflow.WhenAny(a, b, losers: Losers.Continue);   // …and leave the rest running

await is what separates the two forms, and it is not decoration:

Description#

Every leg you start must be joined#

A leg's compensation is registered when the leg succeeds, and the wait is what learns that. So a leg you start and never wait for is work with nothing to undo it — a hotel booked with no way back. That never compiles: the compiler names the leg and the line.

For the same reason, a wait must be written with await. Without it nothing waits, and nothing observes the outcomes.

Many legs of the same kind: build the list, pass the list#

Sometimes the legs really are the same work repeated — reserve every line of this order, notify every subscriber — and how many there are is a property of the data, not of the source. Build them in a loop and hand the wait the list:

var legs = new List<Leg>();
foreach (var line in order.Lines)
  legs.Add(saga.Run("reservation", new Reservation { Line = line }, () => Release(line)));

await Workflow.WhenAll(legs);

This is the same overload pair C# gives Task.WhenAll — several tasks, or a sequence of them — which is why it is an argument form rather than a fourth verb. Everything that made a leg readable stays where it was: you write the loop, you create each step, and each carries its own undo. The wait is handed handles; it does not iterate on your behalf and it does not own any leg's body.

Everything below applies per element: the settle-then-unwind rule, which legs get compensated, and the requirement that every leg you start is joined — a list built and never waited for is refused by name, exactly as a single leg is.

Waiting for enough, not for all#

WhenAll waits for every leg. WhenAny returns as soon as one succeeds, and When(n, …) as soon as n do — they count successes, not finishes, because the question is "did enough of my work get done", and a leg that failed did not.

If a quorum becomes impossible — too many legs have failed — the join reports the failure once every leg has settled, exactly as WhenAll does.

What happens to the legs that lost#

By default they are cancelled and compensated. A leg you gave an undo to is one you said must not dangle, and work left standing with nothing to reverse it is the failure that is invisible until it matters.

Cancelling is thorough: the leg stops, and every level of it unwinds itself on the way out, so a leg that had its own compensations registered runs them before it ends. Then this run's own undo for that leg runs too.

That last part is worth being precise about, because it is the difference between correct and nearly correct: a loser's compensation runs now, rather than joining the stack that unwinds if the saga fails. A saga that goes on to Complete() drops that stack — so a loser parked there would be quietly forgotten by exactly the run that won.

Say losers: Losers.Continue when the losers really are irrelevant rather than wrong — a "first responder answers" race where the others doing their work harms nothing. It has to be written down; it is not somewhere you should be able to arrive by omission.

WhenAll has no losers, so passing losers: there is an error rather than something quietly ignored.

What WhenAll guarantees#

• It returns only when every leg has finished. One leg finishing changes nothing on its own.
• Every leg that succeeded is compensable. Their undos go onto the scope's stack in the order the legs were started, so a later failure unwinds them last-started-first — the same order a sequence of steps produces.
• A leg that failed compensates nothing. Only work that actually committed gets an undo. This is the same rule a sequential step follows, and it is why the wait, rather than the start, is where undos are registered.

When a leg fails: everything settles first#

If one leg fails while another is still running, nothing is cancelled and nothing is compensated yet. The wait lets the other legs reach their own conclusion, and only then reports the failure.

This is deliberate, and it is the difference between a compensation that is safe and one that is not. Compensating against a leg that is halfway through means undoing something whose extent nobody knows — the hardest problem in the category. Letting each leg finish first means every compensation faces a settled fact.

The cost is latency on a path that was already failing, which is the cheapest place to spend it.

What it costs you when it fails#

The failure arrives as an ordinary catchable workflow error naming the legs that failed, so the surrounding try/catch — and the scope's unwind — work exactly as they do for a single step. Nothing new to learn.

Examples#

The world the trip is booked in — two kinds of work, each its own entity, its own workflow, and its own way back:

enum TripStatus { Planning, Booked, Abandoned }
enum LegStatus  { Pending, Confirmed }

entity Trip   { [Required, MaxLength(60)] string Destination; TripStatus Status = TripStatus.Planning; }
entity Flight { [Required] Trip Trip; [Required, MaxLength(20)] string Route; LegStatus Status = LegStatus.Pending; }
entity Hotel  { [Required] Trip Trip; [Required, MaxLength(60)] string City;  LegStatus Status = LegStatus.Pending; }

void CancelFlight(Flight flight) { }
void CancelHotel(Hotel hotel)    { }

workflow FlightFlow { Tracks = Flight.Status; Autostart = false; Initial = Pending;
  event Confirm(); state Pending { subscribe Confirm(); on Confirm { goto Confirmed; } } terminal success Confirmed { } }
workflow HotelFlow { Tracks = Hotel.Status; Autostart = false; Initial = Pending;
  event Confirm(); state Pending { subscribe Confirm(); on Confirm { goto Confirmed; } } terminal success Confirmed { } }

Then the booking itself — both legs in flight at once, each with the undo that belongs to it:

workflow TripFlow {
  Tracks = Trip.Status; Autostart = false; Initial = Planning;
  state Planning {
    enter {
      await using var saga = Workflow.BeginSaga();
      try {
        // Each row is created inside a `Workflow.Once` step: the body is re-entered from the top after a wait, so a
        // bare `new` would create a second row on resume. Complex creation is just statements.
        var flight = Workflow.Once("make-flight", () => new Flight { Trip = this.Item, Route = "LHR-JFK" });
        var hotel  = Workflow.Once("make-hotel",  () => new Hotel  { Trip = this.Item, City  = this.Item.Destination });

        var bookingFlight = saga.Run("flight", flight, () => CancelFlight(flight));  // STARTED, not waited for
        var bookingHotel  = saga.Run("hotel", hotel,  () => CancelHotel(hotel));

        await Workflow.WhenAll(bookingFlight, bookingHotel);               // …now wait for both

        saga.Complete();
        goto Booked;
      } catch (WorkflowError e) { goto Abandoned; }   // whichever leg stood is cancelled on the way out
    }
  }
  terminal success Booked     { }
  terminal error   Abandoned  { Message = "trip abandoned"; }
}

If the hotel cannot be had, the flight leg is still allowed to finish — and then the flight is cancelled and the trip routes to Abandoned. If both stand, saga.Complete() drops the undos and nothing is cancelled.

Change one word and it becomes a race — book whichever is available first, and release the other:

void ReleaseFlight(Flight flight) { }
void ReleaseHotel(Hotel hotel)    { }

workflow RaceFlow {
  Tracks = Trip.Status; Autostart = false; Initial = Planning;
  state Planning {
    enter {
      await using var saga = Workflow.BeginSaga();
      try {
        var flight = Workflow.Once("make-flight", () => new Flight { Trip = this.Item, Route = "LHR-JFK" });
        var hotel  = Workflow.Once("make-hotel",  () => new Hotel  { Trip = this.Item, City  = this.Item.Destination });

        var byAir  = saga.Run("flight", flight, () => ReleaseFlight(flight));
        var byRoom = saga.Run("hotel", hotel,  () => ReleaseHotel(hotel));

        await Workflow.WhenAny(byAir, byRoom);   // the loser is cancelled AND released
        saga.Complete();
        goto Booked;
      } catch (WorkflowError e) { goto Abandoned; }
    }
  }
  terminal success Booked    { }
  terminal error   Abandoned { Message = "trip abandoned"; }
}

And when the legs are one-per-element, the loop is yours and only the wait changes:

enum OrderStatus { Placed, Reserved, Abandoned }
enum LineStatus  { Pending, Held }

entity Order {
  [Required, MaxLength(30)] string Ref;
  OrderStatus Status = OrderStatus.Placed;
  [ForeignKey(Order)] OrderLine[] Lines;
}
entity OrderLine   { [Required] Order Order; [Required, MaxLength(20)] string Sku; int Quantity; }
entity Reservation { [Required] OrderLine Line; LineStatus Status = LineStatus.Pending; }

void ReleaseReservation(Reservation reservation) { }

workflow ReservationFlow { Tracks = Reservation.Status; Autostart = false; Initial = Pending;
  event Confirm(); state Pending { subscribe Confirm(); on Confirm { goto Held; } } terminal success Held { } }

workflow OrderFlow {
  Tracks = Order.Status; Autostart = false; Initial = Placed;
  state Placed {
    enter {
      await using var saga = Workflow.BeginSaga();
      try {
        var legs = new List<Leg>();
        foreach (var line in this.Item.Lines) {
          // The step id carries a per-iteration segment, so one label inside a loop is one step PER LINE.
          var reservation = Workflow.Once("make-reservation", () => new Reservation { Line = line });
          legs.Add(saga.Run("reservation", reservation, () => ReleaseReservation(reservation)));   // one leg per line
        }

        await Workflow.WhenAll(legs);          // however many that turned out to be

        saga.Complete();
        goto Reserved;
      } catch (WorkflowError e) { goto Abandoned; }   // the lines that WERE held are released on the way out
    }
  }
  terminal success Reserved  { }
  terminal error   Abandoned { Message = "order abandoned"; }
}

Notes#

• The legs are named by your variables. A failure says which leg failed, because you gave it a name — not "leg 2".
• A leg is a workflow of its own, so it can wait on people, timers and events exactly as any workflow does. That is the case the wait exists for; legs that finish instantly work too, and simply never park anything.
• Starting a leg does not wait for anything, so the statements between the starts run immediately. The single pause in the whole shape is the wait itself.
• The count in When(n, …) is a plain number, not an expression. A join that asks for more legs than it was given can never be satisfied — a compile error when you passed the legs one by one, and a fault at the wait itself when they came from a list, which is the first moment the count exists.
• A list of legs is still your legs. The wait never iterates anything and never creates anything; it is handed handles you made. If you want the platform to fan work out over a collection for you, that is a different question — see dynamic fan-out (foreach over a runtime collection).

See also#

• Workflow.BeginSaga (a compensating saga scope) — the scope these legs live in, and how compensation unwinds.
• Workflow.Run (start a workflow) — start one workflow, and optionally wait for it.
• dynamic fan-out (foreach over a runtime collection) — many instances of the same work, one per element.