Subsections

The Workflow editor

This software component provides a graphical editing tool for workflows, offering features like undoing changes, performing automatic graph layouts, zooming, and printing of diagrams. Each workflow is created in its own project and can be submitted and monitored like a single job.

Figure 10: The workflow editor: editing mode
Image workflow_editor

Creating a workflow project

In order to create a new workflow project, either select File $\rightarrow$ New $\rightarrow$ Workflow Project from the workbench's menu bar or select New $\rightarrow$ Workflow Project from the context menu of the Navigator view (see Figure 6). After providing a valid name for both the parent folder and the workflow file, the project is created.

Editing mode

When creating a new workflow project or opening an existing workflow file, a new workflow editor instance is opened for setting up the workflow description (see Figure 10). Workflow descriptions are graphs consisting of nodes (commonly called activities in workflow terminology) and edges (called transitions). When a workflow diagram is created, it only displays a single activity: the starting activity of the workflow 1. Execution of the workflow begins at this activity. In order to add new elements to the workflow, select them from the palette on the left hand side and click in the diagram where you want to place them. Currently, the palette offers the following elements that can be added to the workflow:

  1. Application activities
    These activities represent jobs that are submitted to target systems during workflow execution in order to run specific applications there. For each application GUI that is installed in the client platform, the palette shows a small icon and the name of the application 2. By selecting an icon and left-clicking a free spot within the workflow editor, a new activity for the associated application will be created. This leads to the creation of a job file in the `jobs' directory of the workflow project as soon as the workflow is saved.

    When being double-clicked, application activities will open the job editor for the associated job file. The editor can be used in order to change the job description. When a job is embedded in a workflow, there are a several additional possiblities for specifying the job's inputs and outputs that are not available for single jobs:

  2. Transitions
    Transitions represent the flow of control that passes from one activity to the next. Currently, there are two types of transitions: unconditional 3 and conditional 4 ones. Only unconditional transitions can be added to the workflow manually. Conditional transitions are used in If-statements and While-loops and are added automatically. The reason for this is that conditional transitions may require a different joining behaviour: the default joining behaviour when an activity has multiple incoming transitions is called `synchronisation'. This means that the activity is only processed when all incoming transitions have been processed. As you might imagine, this behaviour is no longer appropriate when conditional transitions are used: the activity that joins the if and else branches of an If-statement would never be processed if it waited for both branches to finish. In order to hide this complexity from users that are unfamiliar with workflow processing and programming languages, If-statements and similar constructs will be modelled as sub-workflows that automatically define the appropriate joining behaviour.

  3. Workflow structures
    Workflow structures are subgraphs that bring their own semantics on how to process their child nodes. Currently, four workflow structures are provided: groups, If-statements, While-loops, and ForEach-Loops 5.
    1. Groups are the simplest of all subgraphs. They are just containers for other activities. Their content may be hidden by clicking the small minus symbol at their top.
    2. If-Statements influence the flow of control and contain two additional subgraphs (which are modelled as groups): the if-branch and the else-branch. The if-branch is processed when a certain user-defined condition holds. If the condition evaluates to false the else-branch is processed instead. Both branches can contain arbitrary activities and transitions, thus permitting nesting of workflow structures. Conditions can be altered by double clicking the conditional transition. This will open up the Properties view which displays relevant properties of workflow elements 6. Most properties can be modified through this view. There are currently two types of conditions: the first type compares the exit status of an application to a value provided by the user and the second one compares the value of a workflow variable to a user-given value.
    3. The While-loop provides a single subgraph called the loop-body that can be processed multiple times (as long as the loop's condition holds true). The While-loop declares a workflow variable that reflects the current number of loop iterations, the so-called loop `iterator'. It also declares a variable modifier that increments the loop iterator. The variable declaration can be changed in the Properties view of the red activity at the top of the loop and the variable modifier can be set up in the Properties view of the associated modifier activity (at the bottom of the while loop).
    4. ForEach-Loops can be used in order to create many similar jobs without having to set up each job individually. They have two different modes of operation. The first mode will iterate over a set of workflow variable values and run the job(s) contained in the loop body once for each value in the set. The workflow variable values can be used as input parameters for these jobs. Complex parameter sweeps are possible, as multiple workflow variables can be sweeped at the same time. The second mode is used to iterate over a set of files. The file set may consist of any combination of local, remote or workflow files. This mode provides a convenient way to process many different files simultaneously. The operational mode and the parameters to the selected mode can be modified in the Properties view of the orange activity at the top of the ForEach-loop.

      The iterations of the ForEach-loop are usually executed in parallel. However, there is an upper bound of parallel iterations which results from the workflow engine's capabilities. There is also a way to lower this boundary by providing an Integer value for the Number of parallel tasks in the Properties view of the ForEach activity. Setting this value to `1' will lead to sequential execution of the loop iterations.

  4. Variable declarations and modifiers 7
    Additional workflow variables can be declared using the appropriate Declaration activity. The Properties view of this activity allows for (re-)naming the variable and assigning it a type (e.g. String or Integer) and initial value. A Modifier activity can be used to change the value of a workflow variable later.

When the user is pleased with the workflow description, the workflow can be submitted via the editor's context menu 8 or the workbench's tool bar. It can also be exported to an XML based workflow language that the workflow engine understands 9. The exported workflow can later be submitted to the workflow engine by the UNICORE commandline client. This feature is useful e.g. in order to make predefined workflows available via a web interface (the Chemomentum web portal solution uses the commandline client for workflow submission).

Figure 11: The workflow editor: monitoring mode and the Trace Graph view
Image monitoring_tracing

Monitoring mode

The workflow editor is also used for monitoring the execution of workflows, so the basic graphical representation of a workflow stays the same before and after submission to the workflow engine (see Figure 11). This helps in identifying which part of the workflow is being executed at a given point in time. When a workflow has been submitted, a new folder is created in the `submitted' subfolder of the workflow project. This folder contains a copy of the workflow file that is automatically opened in a new workflow editor panel - in monitoring mode. In this mode, the editor disallows any changes to the workflow. It displays the progress of workflow execution by adding small icons to the nodes of the workflow graph that symbolise the execution state of these parts 1.

Outcomes of jobs can be fetched as soon as the jobs have finished. This function is available via the context menu of application activities and the fetch output files action in the global tool bar (after selecting the activity for which to fetch outcomes). Job outcomes are downloaded to the `output files' folder again, so they can easily be found later and associated with the workflow by which they were produced. Monitoring a workflow can be interrupted by simply closing the editor panel 2. By double-clicking the file that represents the submitted workflow (in the Navigator view), the editor panel will be re-opened and continue to monitor the execution of the workflow.

The Trace Graph view

In addition to monitoring the execution states of activities in the workflow, the user may trace the workflow for finding out where his jobs were submitted. This action is available via the context menu of the workflow editor. A trace graph will open, showing all messages that were sent by the workflow system during the execution of the workflow 3. By hovering the mouse over a node or edge in the trace graph, additional information about the element is displayed in a tooltip. The set of traced messages can be updated by clicking the Refresh button in the tool bar of the Trace Graph view 4. Additional buttons allow to zoom in and out (zooming can also be achieved by rotating the mouse wheel while pressing the `control' key).

Bastian Demuth
2010-05-25