Concepts#

This page explains the ideas behind ZagalaSpec and how each one is realised in the plugin. Read the plain sections if you write or review features; open the In ZagalaSpec parts and the dropdowns when you configure or implement them.

Why executable specifications#

A specification that lives in a document goes stale the day after it is written. A specification that the engine can execute cannot: either the game still does what the sentence says, or the test is red and somebody has to look.

That gives a team three things it usually lacks:

  • A deliverable for the conversation. When designers, producers, QA and programmers agree on how a feature should behave, the output is a set of scenarios, not meeting notes.

  • A trace from intent to status. Each scenario is a line in the Session Frontend. “Is it done?” becomes “is it green?”.

  • Protection against regressions. A fixed bug gets a scenario. If it comes back, the scenario says so before a player does.

  • Falseability. A failing scenario signals the system is unreleasable or broken in some way, prompting immediate attention and correction.

ZagalaSpec stores those scenarios as Unreal assets, runs them through the standard Unreal Automation Framework and imports/exports them to the text formats other teams already use.

TDD, ATDD and BDD in four paragraphs#

Test-driven development (TDD) is the habit to design based on behavior. Behavior is reflected in writing test sections that describe the expected outcomes before implementing the code. The TDD loop is: write a failing test, implement the code to make it pass, then refactor. Red, green, refactor.

Behaviour-driven development (BDD) gives shape to tests: Given a situation, When something happens, Then this is observable. The three-part sentence is readable by everyone, and specific enough to automate.

Acceptance-test-driven development (ATDD) applies the same loop one level up. Before building a feature, the people who want it and the people who build it agree on concrete examples of it working (scenarios), written in the language of the product rather than the code. Those examples become acceptance criteria. This shifts the focus to collaboration and shared understanding that brings value to the business.

Good scenarios follow INVEST: Independent (can be implemented in any order), Negotiable (they describe outcomes, not implementations, so they can change), Valuable (give concrete value to customers), Estimable (time is measured from prior features and extrapolated), Small (scalable; should be done in a short time), and Testable (it is only finished when it passes its acceptance criteria). A scenario is not a contract carved in stone; it is the current best description of what the team wants, and editing it is cheap.

In ZagalaSpec

  • The Features are Feature assets. Each holds scenarios (acceptance criteria), each scenario holds Given / When / Then sentences.

  • The TDD loop is: write the scenario, run it in the Session Frontend and watch it fail (either a sentence has no step behind it or the game does not do it yet), implement the step or the gameplay, run again, green.

  • The sentence is the contract between the person who wrote it and the person who connects it to the game. Change the sentence and the test changes with it.

Dive deeper on ATDD, BDD and TDD

Vocabulary#

Plain term

What it is

In ZagalaSpec

Feature

A capability of the game with a short description and its scenarios

A UZagalaSpecFeatureAsset data asset

Scenario

One concrete example of the feature working, with an identifier tag

A UZagalaSpecScenario instanced inside the feature

Step

One Given / When / Then sentence

A text step (UZagalaSpecTextStep) holding the sentence

Step entry

The rule that says which sentences a piece of logic understands

A UZagalaSpecRegexStepEntry with a regex pattern and a step class

Execution step

The logic that runs for a sentence

A class derived from UZagalaSpecDefaultExecutionStep (Blueprint or C++)

Native step

A step bound to a class directly, with no sentence

A UZagalaSpecNativeStep

Context extension

A setup step that prepares the world or data before the expectations run

Entries in a scenario’s Context Extensions or a feature’s Shared Context Extensions

Expectation

The Given / When / Then steps that exercise and verify the game

Entries in a scenario’s Expectations

Examples table

Rows of values that run the same scenario several times

A UZagalaSpecTaggedDataExamples asset plus an examples iterator extension

Tagged data

A typed value or reference that steps hand to each other

A UZagalaSpecTaggedData object keyed by a gameplay tag

System under test (SUT)

The part of your game being checked

Your actors, components and objects

Two property names in the editor are historical: Context Extensions means setup steps, Expectations means the scenario’s Given / When / Then steps. Both hold the same kind of object.

The four layers#

A scenario runs through up to four layers. Each layer only talks to the one below it. The four layers were championed by Dave Farley in his work on evolutionary architecture and continuous delivery.

        flowchart LR
    TC["Test case<br/>Feature, scenario, step entry,<br/>execution step"]
    DSL["Domain language<br/>What the step means,<br/>in product terms"]
    PD["Protocol driver<br/>How to make the game do it<br/>and read the result"]
    SUT["System under test<br/>Your gameplay code"]
    TC --> DSL --> PD --> SUT
    
  • Test case: the sentence, the regex that recognised it, and the execution step that was created for it.

  • Domain language (DSL): a small action named after what the product does (“increment the counter”, “check the number”). It knows nothing about actors or components.

  • Protocol driver (PD): the action that actually touches the game: spawns, calls functions, reads values. If the game’s API changes, only this layer changes.

  • System under test: your game. ZagalaSpec never requires you to modify it, only to make it reachable.

In ZagalaSpec

Layer

Base class

Example in BlueprintFeature

Test case

UZagalaSpecDefaultExecutionStep or, when it delegates, UZagalaSpecChainLinkOnStep

BP_Step_ThenNumberShouldBe

Domain language

UZagalaSpecChainLinkOnChainLink

BP_DSL_CheckNumber

Protocol driver

UZagalaSpecDefaultChainLink

BP_PD_CheckNumberAgainstExpected

System under test

anything

the two numbers in the cache

The layers are optional. BP_Step_GivenANumber does everything in the step itself. Collapse layers while a feature is small; introduce the DSL and driver layers when several steps start sharing the same game interaction, when more than one feature touches the same system, or when you want the step text and the game API to change independently.

What ZagalaSpec adds#

The ideas above are the definition of the four-layer model. The following are ZagalaSpec’s own machinery. Each item says what the thing is and how to use it; the dropdowns name the classes for programmers.

Two chains per scenario#

Every scenario runs two sequences of steps. The extension chain holds the feature’s shared context extensions followed by the scenario’s own; it prepares state (a fresh counter, a loaded level, a focused item). The expectation chain holds the scenario’s Given / When / Then steps. If any extension fails, the expectations never run and the scenario fails.

The tagged data cache#

        flowchart LR
    TC["Test case<br/>Feature, scenario, step entry,<br/>execution step"]
    DSL["Domain language<br/>What the step means,<br/>in product terms"]
    PD["Protocol driver<br/>How to make the game do it<br/>and read the result"]
    SUT["System under test<br/>Your gameplay code"]
    TC --> DSL --> PD --> SUT
    Cache[("Tagged data cache")]
    TC -.-> Cache
    DSL -.-> Cache
    PD -.-> Cache
    

The dashed lines are the tagged data cache: a per-scenario map that every layer reads from and writes to, so a value captured from a sentence at the top is available to the driver at the bottom.

Each scenario execution owns one cache: a map from a gameplay tag to a tagged data object. It is handed to every step, DSL action and protocol driver as the assigned cacher, so they communicate by reading and writing tags rather than by holding references to each other.

Rules worth remembering:

  • The cache is cleared at the start of every pass of a scenario and after it finishes. Nothing leaks between scenarios or between iterations.

  • One object per tag. Writing the same tag again replaces the previous object. If you need several values under one tag, put an array inside the tagged data class.

  • Steps read and write through UZagalaSpecTaggedDataHandler (Blueprint nodes Get Data, Set Data, Has Data, Remove Data, Copy Data). Do not touch the map directly.

Under the hood

The cache lives in the scenario’s UZagalaSpecContext and implements IZagalaSpecTaggedDataCacher. Each chain link receives it in InternalExecute(AssignedCacher). Tagged data classes derive from UZagalaSpecTaggedData and define their key either in the DataTag property (Blueprint) or by overriding GetLoadTag() (C++). Nested chains forward the same cacher, so a protocol driver three levels down still sees what the step wrote.

Capture groups#

Most sentences carry values: Given a number 6, When I add 3 of "Arrow". The step entry’s regex marks them as capture groups. When a text step matches an entry, ZagalaSpec builds a Capture Group Customizer on the step with one Capture Group Data entry per group. That entry is where a configurator decides what the captured text becomes:

Field

Meaning

Capture Group Value

The captured text (6, Arrow). Filled automatically; editable.

Scope

Local: this entry says which tagged data class to create. Global: the captured text is looked up in Project Settings (Global Capture Group Classes), so the same word means the same object everywhere.

Group Value Class

The tagged data class to create for this group (Local scope). It can be a Blueprint subclass whose defaults carry tuned values, as the Checkpoint example does for spawn offsets.

Group Value Object

An existing asset referenced by the group instead of text, as the examples iterator does for its examples asset.

Converter Class

How to turn text plus class/object into tagged data. The default converter covers value types and object injection; replace it only for custom parsing. You can also derive it to specify ObjectTaggedDataType or ValueTaggedDataType. See ZagalaSpecDefaultCaptureGroupConverter for more details.

The conversion happens when the step calls Add Local Setup Data To Cache: every capture group is converted and written to the cache under its tag. After that call, the rest of the step, the DSL and the drivers simply read the tags.

Under the hood

UZagalaSpecTextStep::PerformMatching() asks the step registry for a match, then UZagalaSpecCaptureGroupCustomizer::FillPatternData() creates or resizes the UZagalaSpecCaptureGroupData array (previous values are preserved when the pattern still has the same groups). UZagalaSpecDefaultExecutionStep::AddLocalSetupDataToCache() writes a UZagalaSpecTaggedDataLocalSetupData carrying the customizer and then calls StoreDataIntoCache(), which runs each group’s ConverterClass. The default, UZagalaSpecDefaultCaptureGroupConverter, tries in order: its ObjectTaggedDataType (an IZagalaSpecObjectInjector, injecting GroupValueObject or a new GroupValueClass instance); its ValueTaggedDataType or a GroupValueClass that implements IZagalaSpecValueStore (the captured text is passed to StoreValue); finally any GroupValueClass derived from UZagalaSpecTaggedData, instantiated with its defaults. Global scope resolves through UZagalaSpecFeatureSettings::GlobalCaptureGroupClasses, a table of FZagalaSpecGlobalGroupData keyed by the captured text.

The step registry#

A text step finds its logic by asking the step registry: a table from regex pattern to step entry. Step entries written in C++ register themselves when the engine starts; step entries made in Blueprint register when their asset is saved. The first entry registered for a pattern wins; later duplicates are skipped with a warning in the Output Log.

In the editor, a text step shows its match in Matched Step Entry. Empty means no entry understands the sentence yet; the scenario will fail at that step until one does. Matching is re-run half a second after you stop typing in the step’s Definition.

Under the hood

UZagalaSpecStepRegistry is a UZagalaSpecDefaultRegexRegistry singleton. UZagalaSpecRegexStepEntry::PostInitProperties() registers native CDOs; PreSave() persists Blueprint entries as RegexRecords in UZagalaSpecFeatureSettings, which seed the registry when the singleton is created. FindMatch() returns an FZagalaSpecRegexMatch (pattern and soft entry path) without loading the entry asset. The debounce delay is DebounceDelaySeconds in the same settings.

Iteration and examples#

A context extension can ask for the scenario’s expectations to run again. ZagalaSpec checks every extension after each pass; if any still has a pending iteration, it clears the cache, advances all iterators and runs the pass again. This is how examples tables work without duplicating scenarios: an examples asset holds rows of tagged data, an examples iterator extension loads one row per pass, and the expectations read the current row from the cache.

Under the hood

Iterators implement IZagalaSpecIterationController (HasPendingIteration(), UpdateIteration()). UZagalaSpecContextExtensionExamplesIterator is the stock iterator: it loads a UZagalaSpecTaggedDataExamples asset whose Collections array holds one UZagalaSpecTaggedDataCollection per row, writes the current row’s tagged data to the cache and advances on UpdateIteration(). BlueprintFeature’s BP_Step_IterateTwice is a minimal hand-written controller. Imported Scenario Outline blocks become one scenario plus an examples iterator.

Native steps#

Not every step needs a sentence. A native step names an execution step class directly and runs it with no regex matching and no capture groups. Use it for setup that configurators never need to parameterise.

How a scenario runs#

        sequenceDiagram
    participant AF as Automation Framework
    participant C as Context
    participant X as Extension chain
    participant E as Expectation chain
    AF->>C: Setup(shared + scenario extensions, expectations)
    AF->>C: Develop()
    loop one pass per iteration
        C->>C: clear cache
        C->>X: run links in order
        X-->>C: success / failure
        C->>E: run links in order
        E-->>C: success / failure
        C->>E: teardown
        C->>C: any extension has pending iteration? update iterators
    end
    C->>X: teardown
    C->>C: clear cache
    C-->>AF: scenario passed / failed (reason)
    

Each scenario is one automation test named Features.<scenario tag>, for example Features.ZagalaSpec.Features.BlueprintFeature.CheckDataRetrieval. The Session Frontend, the command line and your CI all see the same list.

Under the hood

FFeaturesTest (a complex automation test registered as Features with the ProductFilter flag) enumerates UZagalaSpecFeatureSettings::ScenariosMap, builds a UZagalaSpecContext through UZagalaSpecExecutionContextSubsystem::BuildExecutionContext(), calls Develop() and waits each frame until the context’s UZagalaSpecExecutionState has finished. The failure reason passed to BroadcastFailure() is what the test reports.

What a result means#

A feature with any red scenario means the build is not releasable as far as that feature is concerned. A fully green feature means only that its scenarios pass; the game can still have bugs nobody wrote a scenario for. When you find one, write the scenario first, watch it fail, then fix it.

Glossary#

Assigned cacher

The tagged data cache handed to a link when it executes. Normally the scenario’s context cache.

Capture group

A parenthesised part of a step entry’s regex. Its text becomes tagged data through Capture Group Data.

Chain link

Anything that executes with a cache and broadcasts exactly one success or failure.

Chainer

A chain link that runs other links in sequence.

Context

The runtime object that owns one scenario execution: its two chains, its cache and its state.

Context extension

A setup step. Shared ones live on the feature; scenario ones live on the scenario.

Domain language (DSL) action

A chain link named after a product action, sitting between the step and the protocol driver.

Filter tags

Tags under ZagalaSpec.ScenarioFilter set on a scenario (Advanced section of its Details) that include or exclude it from the test list depending on the environment, such as editor or build configuration. See Usage.

Examples asset

A UZagalaSpecTaggedDataExamples asset: rows of tagged data for data-driven scenarios.

Expectation

A Given / When / Then step of a scenario.

Iteration controller

An extension that can request another pass of the scenario.

Native step

A step bound to a class, with no sentence.

Protocol driver (PD)

A chain link that touches the system under test.

Step entry

A regex pattern mapped to an execution step class.

Tagged data

A UZagalaSpecTaggedData object stored in the cache under a gameplay tag.

Teardown

The reset call every link receives after a pass, so it can run again.

Text step

A step defined by a sentence; it resolves its class through the step registry.