Difference between revisions of "FND-CSC-SPN"

From "A B C"
Jump to navigation Jump to search
m
m
 
(2 intermediate revisions by the same user not shown)
Line 1: Line 1:
<div id="BIO">
+
<div id="ABC">
  <div class="b1">
+
<div style="padding:5px; border:1px solid #000000; background-color:#b3dbce; font-size:300%; font-weight:400; color: #000000; width:100%;">
 
SPN<br />
 
SPN<br />
 
<small>Structured Process Notation</small>
 
<small>Structured Process Notation</small>
  </div>
+
<div style="padding:5px; margin-top:20px; margin-bottom:10px; background-color:#b3dbce; font-size:30%; font-weight:200; color: #000000; ">
 
+
(Structured Process Notation (SPN) for workflow modelling)
  {{Vspace}}
+
</div>
 
 
<div class="keywords">
 
<b>Keywords:</b>&nbsp;
 
Structured Process Notation (SPN) for workflow modelling
 
 
</div>
 
</div>
  
{{Vspace}}
+
{{Smallvspace}}
 
 
 
 
__TOC__
 
  
{{Vspace}}
 
  
 
+
<div style="padding:5px; border:1px solid #000000; background-color:#b3dbce33; font-size:85%;">
{{LIVE}}
+
<div style="font-size:118%;">
 
+
<b>Abstract:</b><br />
{{Vspace}}
 
 
 
 
 
</div>
 
<div id="ABC-unit-framework">
 
== Abstract ==
 
 
<section begin=abstract />
 
<section begin=abstract />
<!-- included from "./components/FND-CSC-SPN.components.txt", section: "abstract" -->
 
 
SPN (Structured Process Notation) is a lightweight, expressive notation for the design of dataflow diagrams.
 
SPN (Structured Process Notation) is a lightweight, expressive notation for the design of dataflow diagrams.
 
<section end=abstract />
 
<section end=abstract />
 
+
</div>
{{Vspace}}
+
<!-- ============================ -->
 
+
<hr>
 
+
<table>
== This unit ... ==
+
<tr>
=== Prerequisites ===
+
<td style="padding:10px;">
<!-- included from "./components/FND-CSC-SPN.components.txt", section: "prerequisites" -->
+
<b>Objectives:</b><br />
<!-- included from "./data/ABC-unit_components.txt", section: "notes-prerequisites" -->
 
You need to complete the following units before beginning this one:
 
*[[FND-CSC-Software_development|FND-CSC-Software_development (Software Development for Research Labs)]]
 
 
 
{{Vspace}}
 
 
 
 
 
=== Objectives ===
 
<!-- included from "./components/FND-CSC-SPN.components.txt", section: "objectives" -->
 
 
This unit will ...
 
This unit will ...
 
* ... introduce SPN, a method to draw workflow diagrams.
 
* ... introduce SPN, a method to draw workflow diagrams.
 +
</td>
 +
<td style="padding:10px;">
 +
<b>Outcomes:</b><br />
 +
After working through this unit you ...
 +
* ... can critically evaluate workflow diagrams and process models and are aware of some aspects of poor information design;
 +
* ... can draw your own SPN diagrams, starting from an online template.
 +
</td>
 +
</tr>
 +
</table>
 +
<!-- ============================  -->
 +
<hr>
 +
<b>Deliverables:</b><br />
 +
<section begin=deliverables />
 +
<li><b>Time management</b>: Before you begin, estimate how long it will take you to complete this unit. Then, record in your course journal: the number of hours you estimated, the number of hours you worked on the unit, and the amount of time that passed between start and completion of this unit.</li>
 +
<li><b>Journal</b>: Document your progress in your [[FND-Journal|Course Journal]]. Some tasks may ask you to include specific items in your journal. Don't overlook these.</li>
 +
<li><b>Insights</b>: If you find something particularly noteworthy about this unit, make a note in your [[ABC-Insights|'''insights!''' page]].</li>
 +
<section end=deliverables />
 +
<!-- ============================  -->
 +
<hr>
 +
<section begin=prerequisites />
 +
<b>Prerequisites:</b><br />
 +
This unit builds on material covered in the following prerequisite units:<br />
 +
*[[FND-CSC-Software_development|FND-CSC-Software_development (Software Development for Research Labs)]]
 +
<section end=prerequisites />
 +
<!-- ============================  -->
 +
</div>
  
{{Vspace}}
+
{{Smallvspace}}
  
  
=== Outcomes ===
 
<!-- included from "./components/FND-CSC-SPN.components.txt", section: "outcomes" -->
 
After working through this unit you ...
 
* ... can critically evaluate workflow diagrams and process models and are aware of some aspects of poor information design;
 
* ... can draw your own SPN diagrams, starting from an online template.
 
  
{{Vspace}}
+
{{Smallvspace}}
  
  
=== Deliverables ===
+
__TOC__
<!-- included from "./components/FND-CSC-SPN.components.txt", section: "deliverables" -->
 
<!-- included from "./data/ABC-unit_components.txt", section: "deliverables-time_management" -->
 
*<b>Time management</b>: Before you begin, estimate how long it will take you to complete this unit. Then, record in your course journal: the number of hours you estimated, the number of hours you worked on the unit, and the amount of time that passed between start and completion of this unit.
 
<!-- included from "./data/ABC-unit_components.txt", section: "deliverables-journal" -->
 
*<b>Journal</b>: Document your progress in your [[FND-Journal|Course Journal]]. Some tasks may ask you to include specific items in your journal. Don't overlook these.
 
<!-- included from "./data/ABC-unit_components.txt", section: "deliverables-insights" -->
 
*<b>Insights</b>: If you find something particularly noteworthy about this unit, make a note in your [[ABC-Insights|'''insights!''' page]].
 
  
 
{{Vspace}}
 
{{Vspace}}
  
  
</div>
+
=== Evaluation ===
<div id="BIO">
+
<b>Evaluation: NA</b><br />
 +
<div style="margin-left: 2rem;">This unit is not evaluated for course marks.</div>
 
== Contents ==
 
== Contents ==
<!-- included from "./components/FND-CSC-SPN.components.txt", section: "contents" -->
 
 
Engineering workflow-centric software for research laboratories requires transparent, high level models to structure the design and to document the overall organisation of the code. However current methodologies are not well suited for this task, focussing on the implementation of objects rather than on the integration of functions and suffering from poor information design principles. SPN (Structured Process Notation) is an intuitive, lightweight alternative loosely based on {{WP|Structured_Analysis_and_Design_Technique|SADT}} / {{WP|IDEF0}}. SPN is '''intuitive''' by consistently applying a directional metaphor that is close to implementation, '''expressive''' by using icons and avoiding any superfluous elements, '''simple''' by minimizing the number of symbols and using explicit annotations, '''granular''' by supporting multiple levels of detail through nesting, and '''extensible'''.
 
Engineering workflow-centric software for research laboratories requires transparent, high level models to structure the design and to document the overall organisation of the code. However current methodologies are not well suited for this task, focussing on the implementation of objects rather than on the integration of functions and suffering from poor information design principles. SPN (Structured Process Notation) is an intuitive, lightweight alternative loosely based on {{WP|Structured_Analysis_and_Design_Technique|SADT}} / {{WP|IDEF0}}. SPN is '''intuitive''' by consistently applying a directional metaphor that is close to implementation, '''expressive''' by using icons and avoiding any superfluous elements, '''simple''' by minimizing the number of symbols and using explicit annotations, '''granular''' by supporting multiple levels of detail through nesting, and '''extensible'''.
  
Line 387: Line 377:
 
{{FullImage|SPN_ExampleProcess.png|A sample process. Note that '''FETCH''' generates multiple files, '''PRY/MEND''' and '''TELL''' operate on these files sequentially. The files are merged by '''COUNT''', which produces a single output list. Two nested diagrams for '''TELL''' and '''COUNT''' are referenced, there is no off-page continuation, the process terminates with '''FRA'''.
 
{{FullImage|SPN_ExampleProcess.png|A sample process. Note that '''FETCH''' generates multiple files, '''PRY/MEND''' and '''TELL''' operate on these files sequentially. The files are merged by '''COUNT''', which produces a single output list. Two nested diagrams for '''TELL''' and '''COUNT''' are referenced, there is no off-page continuation, the process terminates with '''FRA'''.
 
}}
 
}}
 
 
 
{{Vspace}}
 
  
  
Line 400: Line 386:
 
{{Vspace}}
 
{{Vspace}}
  
 
== Notes ==
 
<!-- included from "./components/FND-CSC-SPN.components.txt", section: "notes" -->
 
<!-- included from "./data/ABC-unit_components.txt", section: "notes" -->
 
<references />
 
 
{{Vspace}}
 
 
 
</div>
 
<div id="ABC-unit-framework">
 
== Self-evaluation ==
 
<!-- included from "./components/FND-CSC-SPN.components.txt", section: "self-evaluation" -->
 
<!--
 
=== Question 1===
 
 
Question ...
 
 
<div class="toccolours mw-collapsible mw-collapsed" style="width:800px">
 
Answer ...
 
<div class="mw-collapsible-content">
 
Answer ...
 
 
</div>
 
  </div>
 
 
  {{Vspace}}
 
 
-->
 
 
{{Vspace}}
 
 
 
 
{{Vspace}}
 
 
 
<!-- included from "./data/ABC-unit_components.txt", section: "ABC-unit_ask" -->
 
 
----
 
 
{{Vspace}}
 
 
<b>If in doubt, ask!</b> If anything about this learning unit is not clear to you, do not proceed blindly but ask for clarification. Post your question on the course mailing list: others are likely to have similar problems. Or send an email to your instructor.
 
 
----
 
 
{{Vspace}}
 
  
 
<div class="about">
 
<div class="about">
Line 464: Line 402:
 
*0.1 First stub
 
*0.1 First stub
 
</div>
 
</div>
[[Category:ABC-units]]
 
<!-- included from "./data/ABC-unit_components.txt", section: "ABC-unit_footer" -->
 
  
 
{{CC-BY}}
 
{{CC-BY}}
  
 +
[[Category:ABC-units]]
 +
{{UNIT}}
 +
{{LIVE}}
 
</div>
 
</div>
 
<!-- [END] -->
 
<!-- [END] -->

Latest revision as of 13:12, 24 September 2020

SPN
Structured Process Notation

(Structured Process Notation (SPN) for workflow modelling)


 


Abstract:

SPN (Structured Process Notation) is a lightweight, expressive notation for the design of dataflow diagrams.


Objectives:
This unit will ...

  • ... introduce SPN, a method to draw workflow diagrams.

Outcomes:
After working through this unit you ...

  • ... can critically evaluate workflow diagrams and process models and are aware of some aspects of poor information design;
  • ... can draw your own SPN diagrams, starting from an online template.

Deliverables:

  • Time management: Before you begin, estimate how long it will take you to complete this unit. Then, record in your course journal: the number of hours you estimated, the number of hours you worked on the unit, and the amount of time that passed between start and completion of this unit.
  • Journal: Document your progress in your Course Journal. Some tasks may ask you to include specific items in your journal. Don't overlook these.
  • Insights: If you find something particularly noteworthy about this unit, make a note in your insights! page.

  • Prerequisites:
    This unit builds on material covered in the following prerequisite units:


     



     



     


    Evaluation

    Evaluation: NA

    This unit is not evaluated for course marks.

    Contents

    Engineering workflow-centric software for research laboratories requires transparent, high level models to structure the design and to document the overall organisation of the code. However current methodologies are not well suited for this task, focussing on the implementation of objects rather than on the integration of functions and suffering from poor information design principles. SPN (Structured Process Notation) is an intuitive, lightweight alternative loosely based on SADT / IDEF0. SPN is intuitive by consistently applying a directional metaphor that is close to implementation, expressive by using icons and avoiding any superfluous elements, simple by minimizing the number of symbols and using explicit annotations, granular by supporting multiple levels of detail through nesting, and extensible.


     
    SPN Core.png

    The core SPN elements and their general layout. Icons are drawn on the left of the page, labels and descriptions on the right. Fonts may be formatted to emphasize correspondence. Input data enters an activity from the top, parameters from the left, database, libraries, services, repositories and similar resources from the right and output data leaves the activity at the bottom. Connections are drawn with black lines and all other lines in the diagram are not black to make them visually distinct. An activity can be annotated with functions, methods or algorithms. Notes and comments can be added for clarification, they are formatted in grey, the sense of their pointing arrows is always against the dataflow and these arrows are not connected to icons, to avoid confusion. A pale dotted line groups elements that are described in greater detail elsewhere, a reference is included. All elements of the diagram are optional. A Name identifies the process and a versioned reference ID (process prefix, number, version, page) provides a stable, traceableidentifier. The page number is optional for a single-page diagram. A summary at the top may be usefula nd help making the intent of the process explicit.


    Features include:

    • A structured, spatial organization of components intuitively depicts data flow.
    • Information is not implicit in a complex grammar of symbols but made explicit; the set of symbols that are used is small and they are iconic, i.e. their shape is related to their semantics.
    • SPN's process-centric view of data flow is close to the implementation of the workflow in code, in particular modules and interfaces are clearly visible.
    • Icons are separated from text to allow uncluttered spatial representation of relationships, but explicit descriptions are available by placing elements and descriptions at corresponding heights on the diagram and relating them through mnemonic labels.
    • Nesting of diagrams is encouraged. Procedures can be expanded into more detail in a separate diagram. This allows to structure all components of the workflow, at any level of granularity, without overloading individual diagrams with too much detail.


     

    Elements

     
    SPN Elements
    Icon Name Description
    SPN Data.png Data

    Any data item that is consumed or produced by an activity / procedure in the workflow. Typically this may be a file, but it can also be a stream, a series of files, an object in memory, or one or more tables in a datamodel. The nature of the data is clarified in the description. Labels are short mnemonics. Examples: ID, Tor, s3D, mapX. Data items have one incoming relationship line (a data item is produced by exactly one activity) and can have more than one outgoing relationships (data items can be used by more than one activity). The data icon can be used to mark the Begin and End state for the process. Begin icons are not produced by an activity.

    SPN MultiData.png Multi data

    Multiple data items with the same format and semantics are represented with the multi data icon. They are produced and consumed sequentially by an activity. For paralell processes, use braces (see below).

    SPN ConnectingArrows.png Connecting arrow

    Connecting arrows depict dataflow in the process and establish the relationship between input, activity and output. All lines are orthogonal and line crossings are avoided. Drawing connections against the flow is discouraged, and lines must never enter the output (lower) edges of icons.

    All connections have a direction. We never use plain lines but arrows.

    Arrows at the page edge that do not connect to icons indicate that the workflow / diagram is continued on another page.

    SPN Resource.png Resource

    A database, repository or service that supplies data to the process through some query mechanism but is not significantly modified by the process, i.e. it is not a target of process output – any "canned" result. Labels are either acronyms, abbreviations, or proper names. Examples: PDB, SP ( = SwissProt). This icon is also used for services, i.e. activites that reside outside of the system, such as BLAST, or T-Coffee.

    SPN Activity.png Activity

    Activities operate on input to generate output. These can be programs, scripts, modules or functions/methods within programs. Activities are given mnemonic labels, one-sylllable action verbs are recommended. Examples: Hoard, Spin, Chew. Labels do not have to be the actual function name, their purpose is merely to be descriptive enough to support comprehension of the workflow.

    SPN Parameter.png Parameter

    One or more parameters for an activity. These would be values that would typically passed through command-line or initialization file. Parameters are named and default values can be given. Ellipsis {...} can be used if the list is not complete. Parameters can be Begin states.

    SPN Function.png Function

    A function, subroutine or algorithm that is a non-trivial part of an activity. This is optional and only used to highlight a particular functional aspect of the workflow, or choices between alternatives. Example: BioConductor, Minerva, curl.

    SPN Branch.png Branch

    Conditional branches fork dataflow according to a condition that is described in a right-hand side annotation. Labels are short eg. C1, C2. Use these sparingly, SPN diagrams are data flow models, not control models. It will usually be better to indicate iteration or recursion with a Note on the Activity, rather than drawing an explicit loop. But don't be dogmatic about this, draw a loop if it improves clarity.

    SPN Connector.png Connector

    The connector dot is used when dataflows are merged or split. In general, it should be avoided – data icons accept up to three outgoing lines and activities accept up to three lines from input, output and resources. However connectors are required if more than three conections to the same object are needed. They make splits and joins explicit and distinguish them clearly from line crossings.

    SPN Braces.png Braces

    Braces provide conceptual connections to parallel processes, such as map/reduce procedures.

    SPN Note.png Note

    A Note contains a comment about a component of the diagram or about its functions or purpose. The sense of the arrow always goes against the dataflow to avoid confusing a note with an activity. Short information is contained in the Note, longer information is annotated at the side and referenced to the Note with a label.

    SPN Reference.png Reference

    A pale, dashed box is used to indicate a reference to a nested, more detailed model and description, such as a specification, fine grained diagram, datamodel etc.

    The number of incoming and outgoing connections in the subordinate diagram must be the same as those entering or leaving the reference box.

    Through this, the SPN provides for multi-layered hierarchical descriptions of a system and is easily extensible with other notational conventions.

    SPN Continuation.png Continuation

    A pale dashed line is used to indicate that the process continues on a different page. Page numbers are labeled.


     

    "Rules"

     
    I - Be clear and explicit
    This is "Rule Number One". As a corollary, any other rule or convention can be suspended if this helps the clarity of the diagram or improves the message.
    II - Name, reference, version
    A name for the process is listed on the top left of the diagram. A reference is on the top right, it is used in other documentation and to identify nested diagrams. A textual summary may appear at the top. Diagrams and references must be versioned.
    III - Begin and End states
    Abstract Begin and End states of a process are not normally drawn since they should be obvious from the context. An activity can be started with an input parameter or data element, and ends with an output data element.
    IV - Connecting arrows
    Connecting arrows are drawn as horizontal/vertical lines, never oblique lines. Crossing lines are avoided if possible. Arrows should not move against the dataflow.
    V -Multiple data items, multiple resources
    If multiple data elements are produced as output or used as input, they are connected to their respective source or target element with distinct lines, unless there are more than three. If there are more than three connections, they are merged into a connector element which is attached to an activity through a single line.
    If multiple instances of syntactically and semantically equivalent data items are produced, this is indicated with a multi data icon. These are connected to the activity that produces them with a single line.
    If the multiple elements are connected to an activity, this indicates that are processed sequentially. Parallel processing is indicated through braces.
    If the multiple elements are merged in an activity, this is indicated with a brace.
    VI - Nesting
    A procedure can be detailed at higher resolution on a separate sheet. All input and output must be shown at the parent level. This ensures that the expanded view of a procedure introduces no reference to streams or processes that were not present in the parent view and thus ensures that descriptions are locally complete. Nested diagrams are indicated with a box and annotated with a reference/version. The containing page is referenced like a continuation diagram.
    VII - Continuation
    If a process needs to be split accross several pages or diagrams, all output data items have connecting arrows that end at the bottom of the page on a pale, dotted line, and their icons are repeated at the top of the next page. The continuing page is referenced.
    VIII - Color
    All elements that are part of the process are drawn with black lines, all elements that provide additional structure or annotation are drawn grey or in a pale colour. Icons may be filled with a light colour to visually offset them clearly from the page background. Coloured fills may be used to visually group functionally related elements.


     

    In Practice

     

    I start drawing sketches by hand on a blackboard, or whiteboard to get a first overview and quickly move things around, reconnect and rearrange. A final, handdrawn sketch is fine, if an electronic diagram is required, I find after experimenting with many alternatives that Google Slides is currently the best tool to develop SPN diagrams. Here is a link to a SPN template that you can copy and use for your own diagrams.


     
    Best Practice in SPN Diagrams
    Labels
    Good
    • Mnemonic labels;
    • Verb labels for activities;
    • Acronyms for data;
    • Actual names for resources;
    • Versioned references.
    Bad
    • Wordy labels (Graph.clustering)
    • Meaningless labels (A, B, C ...)
    Worst
    • Wordy and redundant labels (Retrieve.sequence.from.NCBI)
    • Duplicate labels;
    • References without version numbers.
    Connectors
    Good
    • All connectors straight and orthogonal;
    • Minimal number of crossings;
    • No backtracking;
    • Connectors attached to proper connection points..
    Bad
    • Connectors at odd angles;
    • Superfluous corners;
    • Backtracking (breaks consistent direction of information flow).
    Worst
    • Unnecessarily crossing lines;
    • Connectors attached to corners or the wrong side of elements.
    • Connectors that are slightly off horizontal/vertical (these stand out visually and give a sloppy impression of the diagram).
    Color
    Good
    • Maximally restrained use of colour, only to separate elements from background and perhaps to group functionally related modules;
    • Consistent use of black lines and text for process components and workflow structure, vs. grey lines and text for annotations.
    Bad
    • No use of color (elements blend into canvas);
    • Obtrusive color (attention is drawn away from process contents).
    Worst
    • Gratuitous use of color and decorations – shadows, outlines, gradients and similar fluff ...
    Even Worse
    than Worst
    • Inconsistent use of decorations.


     

    Example

     
    SPN ExampleProcess.png

    A sample process. Note that FETCH generates multiple files, PRY/MEND and TELL operate on these files sequentially. The files are merged by COUNT, which produces a single output list. Two nested diagrams for TELL and COUNT are referenced, there is no off-page continuation, the process terminates with FRA.


    Further reading, links and resources


     


    About ...
     
    Author:

    Boris Steipe <boris.steipe@utoronto.ca>

    Created:

    2017-08-05

    Modified:

    2017-10-05

    Version:

    1.0

    Version history:

    • 1.0 First live version
    • 0.1 First stub

    CreativeCommonsBy.png This copyrighted material is licensed under a Creative Commons Attribution 4.0 International License. Follow the link to learn more.