Expected Preparations:

  [FND-CSC]
Software_development 		 
  The units listed above are part of this course and contain important preparatory material.  
Keywords: Structured Process Notation (SPN) for workflow modelling
Objectives:

This unit will …

  • … introduce SPN, a method to draw workflow diagrams.

 Outcomes:

After working through this unit you …

  • … can critically evaluate workflow diagrams and process models and are aware of some aspects of poor information design;

  • … can draw your own SPN diagrams, starting from an online template.

Deliverables:

Time management: Before you begin, estimate how long it will take you to complete this unit. Then, record in your course journal: the number of hours you estimated, the number of hours you worked on the unit, and the amount of time that passed between start and completion of this unit. Journal: Document your progress in your Course Journal. Some tasks may ask you to include specific items in your journal. Don’t overlook these. Insights: If you find something particularly noteworthy about this unit, make a note in your insights! page.

Evaluation:

NA: This unit is not evaluated for course marks.

Contents

SPN (Structured Process Notation) is a lightweight, expressive notation for the design of dataflow diagrams.

Engineering workflow-centric software for research laboratories requires transparent, high level models to structure the design and to document the overall organisation of the code. However current, popular methodologies are actually not well suited for this task: they tend to focus on the implementation of objects rather than on the integration of functions, and they are burdened with poor information design decisions. SPN (Structured Process Notation) is an intuitive, lightweight alternative loosely based on SADT(W) / IDEF0(W). SPN is intuitive by consistently applying a directional metaphor that is close to implementation, expressive by using icons and avoiding any superfluous elements, simple by minimizing the number of symbols and using explicit annotations, granular by supporting multiple levels of detail through nesting, and extensible.

 

The core SPN elements and their general layout. Icons are drawn on the left of the page, labels and descriptions on the right. Fonts may be formatted to emphasize correspondence. Input data enters an activity from the top, parameters from the left, database, libraries, services, repositories and similar resources from the right and output data leaves the activity at the bottom. Connections are drawn with black lines and all other lines in the diagram are not black to make them visually distinct. An activity can be annotated with functions, methods or algorithms. Notes and comments can be added for clarification, they are formatted in grey, the sense of their pointing arrows is always against the dataflow and these arrows are not connected to icons, to avoid confusion. A pale dotted line groups elements that are described in greater detail elsewhere, a reference is included. All elements of the diagram are optional. A Name identifies the process and a versioned reference ID (process prefix, number, version, page) provides a stable, traceableidentifier. The page number is optional for a single-page diagram. A summary at the top may be useful and help making the intent of the process explicit. (SPN_Core.png)

Features include:

 

Elements

 

SPN Elements
Icon Name Description

(SPN_Data.png)

 Data Any data item that is consumed or produced by an activity / procedure in the workflow. Typically this may be a file, but it can also be a stream, a series of files, an object in memory, or one or more tables in a datamodel. The nature of the data is clarified in the description. Labels are short mnemonics. Examples: ID, Tor, s3D, mapX. Data items have one incoming relationship line (a data item is produced by exactly one activity) and can have more than one outgoing relationships (data items can be used by more than one activity). The data icon can be used to mark the Begin and End state for the process. Begin icons are not produced by an activity.

(SPN_MultiData.png)

 Multi data Multiple data items with the same format and semantics are represented with the multi data icon. They are produced and consumed sequentially by an activity. For paralell processes, use braces (see below).

(SPN_ConnectingArrows.png)

 Connecting arrow

Connecting arrows depict dataflow in the process and establish the relationship between input, activity and output. All lines are orthogonal and line crossings are avoided. Drawing connections against the flow is discouraged, and lines must never enter the output (lower) edges of icons.

All connections have a direction. We never use plain lines but arrows.

Arrows at the page edge that do not connect to icons indicate that the workflow / diagram is continued on another page.

(SPN_Resource.png)

 Resource A database, repository or service that supplies data to the process through some query mechanism but is not significantly modified by the process, i.e. it is not a target of process output – any “canned” result. Labels are either acronyms, abbreviations, or proper names. Examples: PDB, SP ( = SwissProt). This icon is also used for services, i.e. activites that reside outside of the system, such as BLAST, or T-Coffee.

(SPN_Activity.png)

 Activity Activities operate on input to generate output. These can be programs, scripts, modules or functions/methods within programs. Activities are given mnemonic labels, one-sylllable action verbs are recommended. Examples: Hoard, Spin, Chew. Labels do not have to be the actual function name, their purpose is merely to be descriptive enough to support comprehension of the workflow.

(SPN_Parameter.png)

 Parameter One or more parameters for an activity. These would be values that would typically passed through command-line or initialization file. Parameters are named and default values can be given. Ellipsis {…} can be used if the list is not complete. Parameters can be Begin states.

(SPN_Function.png)

 Function A function, subroutine or algorithm that is a non-trivial part of an activity. This is optional and only used to highlight a particular functional aspect of the workflow, or choices between alternatives. Example: BioConductor, Minerva, curl.

(SPN_Branch.png)

 Branch Conditional branches fork dataflow according to a condition that is described in a right-hand side annotation. Labels are short eg. C1, C2. Use these sparingly, SPN diagrams are data flow models, not control models. It will usually be better to indicate iteration or recursion with a Note on the Activity, rather than drawing an explicit loop. But don’t be dogmatic about this, draw a loop if it improves clarity.

(SPN_Connector.png)

 Connector The connector dot is used when dataflows are merged or split. In general, it should be avoided – data icons accept up to three outgoing lines and activities accept up to three lines from input, output and resources. However connectors are required if more than three conections to the same object are needed. They make splits and joins explicit and distinguish them clearly from line crossings.

(SPN_Braces.png)

 Braces Braces provide conceptual connections to parallel processes, such as map/reduce procedures.

(SPN_Note.png)

 Note A Note contains a comment about a component of the diagram or about its functions or purpose. The sense of the arrow always goes against the dataflow to avoid confusing a note with an activity. Short information is contained in the Note, longer information is annotated at the side and referenced to the Note with a label.

(SPN_Reference.png)

 Reference

A pale, dashed box is used to indicate a reference to a nested, more detailed model and description, such as a specification, fine grained diagram, datamodel etc.

The number of incoming and outgoing connections in the subordinate diagram must be the same as those entering or leaving the reference box.

Through this, the SPN provides for multi-layered hierarchical descriptions of a system and is easily extensible with other notational conventions.

(SPN_Continuation.png)

 Continuation A pale dashed line is used to indicate that the process continues on a different page. Page numbers are labeled.

 

“Rules”

 

I - Be clear and explicit
This is “Rule Number One”. As a corollary, any other rule or convention can be suspended if this helps the clarity of the diagram or improves the message.
II - Name, reference, version
A name for the process is listed on the top left of the diagram. A reference is on the top right, it is used in other documentation and to identify nested diagrams. A textual summary may appear at the top. Diagrams and references must be versioned.
III - Begin and End states
Abstract Begin and End states of a process are not normally drawn since they should be obvious from the context. An activity can be started with an input parameter or data element, and ends with an output data element.
IV - Connecting arrows
Connecting arrows are drawn as horizontal/vertical lines, never oblique lines. Crossing lines are avoided if possible. Arrows should not move against the dataflow.
V -Multiple data items, multiple resources
If multiple data elements are produced as output or used as input, they are connected to their respective source or target element with distinct lines, unless there are more than three. If there are more than three connections, they are merged into a connector element which is attached to an activity through a single line.

If multiple instances of syntactically and semantically equivalent data items are produced, this is indicated with a multi data icon. These are connected to the activity that produces them with a single line.

If the multiple elements are connected to an activity, this indicates that are processed sequentially. Parallel processing is indicated through braces.

If the multiple elements are merged in an activity, this is indicated with a brace.

VI - Nesting
A procedure can be detailed at higher resolution on a separate sheet. All input and output must be shown at the parent level. This ensures that the expanded view of a procedure introduces no reference to streams or processes that were not present in the parent view and thus ensures that descriptions are locally complete. Nested diagrams are indicated with a box and annotated with a reference/version. The containing page is referenced like a continuation diagram.
VII - Continuation
If a process needs to be split accross several pages or diagrams, all output data items have connecting arrows that end at the bottom of the page on a pale, dotted line, and their icons are repeated at the top of the next page. The continuing page is referenced.
VIII - Color
All elements that are part of the process are drawn with black lines, all elements that provide additional structure or annotation are drawn grey or in a pale colour. Icons may be filled with a light colour to visually offset them clearly from the page background. Coloured fills may be used to visually group functionally related elements.

 

In Practice

 

I start drawing sketches by hand on a blackboard, or whiteboard to get a first overview and quickly move things around, reconnect and rearrange. A final, handdrawn sketch is fine, if an electronic diagram is required, I find after experimenting with many alternatives that Google Slides is currently the best tool to develop SPN diagrams. Here is a link to a SPN template that you can copy and use for your own diagrams.

 

Best Practice in SPN Diagrams
Labels
Good
  • Mnemonic labels;
  • Verb labels for activities;
  • Acronyms for data;
  • Actual names for resources;
  • Versioned references.
Bad
  • Wordy labels (Graph.clustering)
  • Meaningless labels (A, B, C …)
Worst
  • Wordy and redundant labels (Retrieve.sequence.from.NCBI)
  • Duplicate labels;
  • References without version numbers.
Connectors
Good
  • All connectors straight and orthogonal;
  • Minimal number of crossings;
  • No backtracking;
  • Connectors attached to proper connection points..
Bad
  • Connectors at odd angles;
  • Superfluous corners;
  • Backtracking (breaks consistent direction of information flow).
Worst
  • Unnecessarily crossing lines;
  • Connectors attached to corners or the wrong side of elements.
  • Connectors that are slightly off horizontal/vertical (these stand out visually and give a sloppy impression of the diagram).
Color
Good
  • Maximally restrained use of colour, only to separate elements from background and perhaps to group functionally related modules;
  • Consistent use of black lines and text for process components and workflow structure, vs. grey lines and text for annotations.
Bad
  • No use of color (elements blend into canvas);
  • Obtrusive color (attention is drawn away from process contents).
Worst
  • Gratuitous use of color and decorations – shadows, outlines, gradients and similar fluff …
Even Worse
than Worst
  • Inconsistent use of decorations.

 

Example

A sample process. Note that FETCH generates multiple files, PRY/MEND and TELL operate on these files sequentially. The files are merged by COUNT, which produces a single output list. Two nested diagrams for TELL and COUNT are referenced, there is no off-page continuation, the process terminates with FRA. (SPN_ExampleProcess.png)

Further Reading

Questions, comments

If in doubt, ask! If anything about this contents is not clear to you, do not proceed but ask for clarification. If you have ideas about how to make this material better, let’s hear them. We are aiming to compile a list of FAQs for all learning units, and your contributions will count towards your participation marks.

Improve this page! If you have questions or comments, please post them on the Quercus Discussion board with a subject line that includes the name of the unit.

References

Page ID: FND-CSC-SPN

Author:
Boris Steipe ( <boris.steipe@utoronto.ca> )
Created:
2017-08-05
Last modified:
2023-01-23
Version:
1.0
Version History:
–  1.0 First live version
–  0.1 First stub
Tagged with:
–  Unit
–  Live
–  Contains images
–  Has further reading

 

[END]