Difference between revisions of "FND-Cargo Cult"

From "A B C"
Jump to navigation Jump to search
m
m
 
(4 intermediate revisions by the same user not shown)
Line 1: Line 1:
−
<div id="BIO">
+
<div id="ABC">
−
  <div class="b1">
+
<div style="padding:5px; border:1px solid #000000; background-color:#b3dbce; font-size:300%; font-weight:400; color: #000000; width:100%;">
 
Cargo Cult Science
 
Cargo Cult Science
−
  </div>
+
<div style="padding:5px; margin-top:20px; margin-bottom:10px; background-color:#b3dbce; font-size:30%; font-weight:200; color: #000000; ">
−
 
+
(Cargo Cult science, Cargo Cult bioinformatics)
−
  {{Vspace}}
+
</div>
−
 
 
−
<div class="keywords">
 
−
<b>Keywords:</b>&nbsp;
 
−
Cargo Cult science, Cargo Cult bioinformatics
 
 
</div>
 
</div>
  
−
{{Vspace}}
+
{{Smallvspace}}
−
 
 
−
 
 
−
__TOC__
 
−
 
 
−
{{Vspace}}
 
−
 
 
  
−
{{LIVE}}
 
  
−
{{Vspace}}
+
<div style="padding:5px; border:1px solid #000000; background-color:#b3dbce33; font-size:85%;">
−
 
+
<div style="font-size:118%;">
−
 
+
<b>Abstract:</b><br />
−
</div>
 
−
<div id="ABC-unit-framework">
 
−
== Abstract ==
 
 
<section begin=abstract />
 
<section begin=abstract />
−
<!-- included from "./components/FND-Cargo_Cult.components.txt", section: "abstract" -->
 
 
Not all activities lead to valuable outcomes and "Cargo Cult Science" is an important metaphor for a class of conceptual problems that are the hallmark of "poor science". The particular issue is that activites are causally disconnected from their claimed beneficial outcomes.
 
Not all activities lead to valuable outcomes and "Cargo Cult Science" is an important metaphor for a class of conceptual problems that are the hallmark of "poor science". The particular issue is that activites are causally disconnected from their claimed beneficial outcomes.
 
<section end=abstract />
 
<section end=abstract />
−
 
+
</div>
−
{{Vspace}}
+
<!-- ============================ -->
−
 
+
<hr>
−
 
+
<table>
−
== This unit ... ==
+
<tr>
−
=== Prerequisites ===
+
<td style="padding:10px;">
−
<!-- included from "./components/FND-Cargo_Cult.components.txt", section: "prerequisites" -->
+
<b>Objectives:</b><br />
−
<!-- included from "./data/ABC-unit_components.txt", section: "notes-external_prerequisites" -->
 
−
You need the following preparation before beginning this unit. If you are not familiar with this material from courses you took previously, you need to prepare yourself from other information sources:
 
−
<!-- included from "./data/ABC-unit_prerequisites.txt", section: "inquiry" -->
 
−
*<b>Inquiry</b>: The scientific method; evidence based reasoning; how to design, execute and document an experiment; Conjecture, hypothesis and theory.
 
−
 
 
−
{{Vspace}}
 
−
 
 
−
 
 
−
=== Objectives ===
 
−
<!-- included from "./components/FND-Cargo_Cult.components.txt", section: "objectives" -->
 
 
This unit will ...
 
This unit will ...
 
* ... introduce the metaphor of a "Cargo Cult", as applied to bioinformatics and science in general;
 
* ... introduce the metaphor of a "Cargo Cult", as applied to bioinformatics and science in general;
 
* ... illustrate with examples;
 
* ... illustrate with examples;
 
* ... discuss principles to avoid the problem.
 
* ... discuss principles to avoid the problem.
−
 
+
</td>
−
{{Vspace}}
+
<td style="padding:10px;">
−
 
+
<b>Outcomes:</b><br />
−
 
 
−
=== Outcomes ===
 
−
<!-- included from "./components/FND-Cargo_Cult.components.txt", section: "outcomes" -->
 
 
After working through this unit you ...
 
After working through this unit you ...
 
* ... can identify "Cargo Cult"-type issues;
 
* ... can identify "Cargo Cult"-type issues;
Line 64: Line 36:
 
* ... are able to critically evaluate projects and activities regarding whether they can contribute to thier claimed value;
 
* ... are able to critically evaluate projects and activities regarding whether they can contribute to thier claimed value;
 
* ... are able to propose improvements.
 
* ... are able to propose improvements.
 +
</td>
 +
</tr>
 +
</table>
 +
<!-- ============================  -->
 +
<hr>
 +
<b>Deliverables:</b><br />
 +
<section begin=deliverables />
 +
<li><b>Time management</b>: Before you begin, estimate how long it will take you to complete this unit. Then, record in your course journal: the number of hours you estimated, the number of hours you worked on the unit, and the amount of time that passed between start and completion of this unit.</li>
 +
<li><b>Journal</b>: Document your progress in your [[FND-Journal|Course Journal]]. Some tasks may ask you to include specific items in your journal. Don't overlook these.</li>
 +
<li><b>Insights</b>: If you find something particularly noteworthy about this unit, make a note in your [[ABC-Insights|'''insights!''' page]].</li>
 +
<section end=deliverables />
 +
<!-- ============================  -->
 +
<hr>
 +
<section begin=prerequisites />
 +
<b>Prerequisites:</b><br />
 +
You need the following preparation before beginning this unit. If you are not familiar with this material from courses you took previously, you need to prepare yourself from other information sources:<br />
 +
*<b>Inquiry</b>: The scientific method; evidence based reasoning; how to design, execute and document an experiment; Conjecture, hypothesis and theory.
 +
<section end=prerequisites />
 +
<!-- ============================  -->
 +
</div>
 +
 +
{{Smallvspace}}
 +
 +
  
−
{{Vspace}}
+
{{Smallvspace}}
  
  
−
=== Deliverables ===
+
__TOC__
−
<!-- included from "./components/FND-Cargo_Cult.components.txt", section: "deliverables" -->
 
−
<!-- included from "./data/ABC-unit_components.txt", section: "deliverables-time_management" -->
 
−
*<b>Time management</b>: Before you begin, estimate how long it will take you to complete this unit. Then, record in your course journal: the number of hours you estimated, the number of hours you worked on the unit, and the amount of time that passed between start and completion of this unit.
 
−
<!-- included from "./data/ABC-unit_components.txt", section: "deliverables-journal" -->
 
−
*<b>Journal</b>: Document your progress in your [[FND-Journal|Course Journal]]. Some tasks may ask you to include specific items in your journal. Don't overlook these.
 
−
<!-- included from "./data/ABC-unit_components.txt", section: "deliverables-insights" -->
 
−
*<b>Insights</b>: If you find something particularly noteworthy about this unit, make a note in your [[ABC-Insights|'''insights!''' page]].
 
  
 
{{Vspace}}
 
{{Vspace}}
  
  
−
</div>
+
=== Evaluation ===
−
<div id="BIO">
+
<b>Evaluation: NA</b><br />
 +
<div style="margin-left: 2rem;">This unit is not evaluated for course marks.</div>
 
== Contents ==
 
== Contents ==
−
<!-- included from "./components/FND-Cargo_Cult.components.txt", section: "contents" -->
 
  
 
{{Vspace}}
 
{{Vspace}}
  
−
The concept of {{WP|Cargo cult science}} was popularized by {{WP|Richard Feynman}} in the 1974 [http://calteches.library.caltech.edu/51/2/CargoCult.htm Caltech Commencement address]. In a nutshell, Feynman points out how scientific practices that lack "scientific integrity" are similar to the activities of a premodern cult in the South Sea islands that developed rituals for attracting goods-bearing supply airplanes by building mock airports.
+
The concept of {{WP|Cargo cult science}} was popularized by {{WP|Richard Feynman}} in his 1974 [http://calteches.library.caltech.edu/51/2/CargoCult.htm Caltech Commencement address]. In a nutshell, Feynman pointed out how scientific practices that lack "scientific integrity" are similar to the activities of a premodern spiritual cult in the South Sea islands that developed rituals for attracting goods-bearing supply airplanes by building mock airports.
  
−
The essence of Cargo Cult is not merely poor science. The essence lies in a disconnect between form and contents: the form is compelling, but there can't be a rational expectation of a benefit from the activity because there is no causal connection between the activity and the claimed outcome. This is often, but not always due to {{WP|List of fallacies|'''logical fallacies'''}}.
+
The essence of Cargo Cult is not merely poor science. What makes it "Cargo Cult" is a disconnect between form and contents: the form is compelling, but there can't be a rational expectation of a benefit from the activity because there is no causal connection between the activity and the claimed outcome. This is often, but not always due to {{WP|List of fallacies|'''logical fallacies'''}}.
  
−
What makes the topic interesting for bioinformatics is that the deficiencies are often subtle, and hard for the non-expert to spot. To guard against Cargo Cult takes integrity, and practice. A structured approach may be helpful that first clearly identifies the hoped-for outcome, then defines the proposed activities, then asks in specific detail how the outcome would be caused by the activity. '''Causation''' is key here -  many examples of Cargo Cult behaviour derive from mistakenly arguing from correlation, where causation has not been demonstrated.
+
The topic is interesting for bioinformatics because the deficiencies are often subtle, and hard for the non-expert to spot. To guard against Cargo Cult takes integrity, and practice. A structured approach may be helpful that first clearly identifies the hoped-for outcome, then defines the proposed activities, then asks in specific detail how the outcome would be caused by the activity. '''Causation''' is key here -  many examples of Cargo Cult behaviour are based on a mistaken belief in causation, where actually merely a correlation was observed. But you have to be careful: the fact that causation has not been shown does not prove it is absent. And even if causation is absent, that does not automatically make the behaviour invalid: sometimes you are right for the wrong reason. Both cases are not Cargo Cult. Rather, it is characteristic for situations we should label as Cargo Cult that there "is no cargo in the system": you are looking in the wrong place, you don't have a control or reference value, you don't understand your data - or similar problems.
  
 
{{Vspace}}
 
{{Vspace}}
Line 100: Line 89:
 
*Also, add to the discusssion of any of the existing examples. Comments that question whether the example actually should be called ''Cargo Cult'', and that lead to improved focus and clarification are especially valuable.
 
*Also, add to the discusssion of any of the existing examples. Comments that question whether the example actually should be called ''Cargo Cult'', and that lead to improved focus and clarification are especially valuable.
 
}}
 
}}
−
 
−
 
−
{{Vspace}}
 
−
 
  
 
== Further reading, links and resources ==
 
== Further reading, links and resources ==
 
<!-- {{#pmid: 19957275}} -->
 
<!-- {{#pmid: 19957275}} -->
 
<!-- {{WWW|WWW_GMOD}} -->
 
<!-- {{WWW|WWW_GMOD}} -->
−
<!-- <div class="reference-box">[https://en.wikipedia.org/wiki/List_of_fallacies '''Wikipedia: List of Logical Fallacies'''] a very comprehensive resource. One would wish that the presence of such a list itself would have a beneficial effect on science. </div> -->
+
*[http://steipe.biochemistry.utoronto.ca/abc/students/index.php/Cargo_Cult_Science Student Contributions: Cargo Cult Examples]
−
 
+
*[http://steipe.biochemistry.utoronto.ca/abc/students/index.php/Not_(quite)_Cargo_Cult_Science Student Contributions: Not (quite) Cargo Cult Examples]
−
{{Vspace}}
 
−
 
 
  
 +
<div class="reference-box">[https://en.wikipedia.org/wiki/List_of_fallacies '''Wikipedia: List of Logical Fallacies'''] a very comprehensive resource. One would wish that the presence of such a list itself would have a beneficial effect on science. </div>
 
== Notes ==
 
== Notes ==
−
<!-- included from "./components/FND-Cargo_Cult.components.txt", section: "notes" -->
 
−
<!-- included from "./data/ABC-unit_components.txt", section: "notes" -->
 
 
<references />
 
<references />
  
 
{{Vspace}}
 
{{Vspace}}
  
−
 
−
</div>
 
−
<div id="ABC-unit-framework">
 
−
== Self-evaluation ==
 
−
<!-- included from "./components/FND-Cargo_Cult.components.txt", section: "self-evaluation" -->
 
−
<!--
 
−
=== Question 1===
 
−
 
−
Question ...
 
−
 
−
<div class="toccolours mw-collapsible mw-collapsed" style="width:800px">
 
−
Answer ...
 
−
<div class="mw-collapsible-content">
 
−
Answer ...
 
−
 
−
</div>
 
−
  </div>
 
−
 
−
  {{Vspace}}
 
−
 
−
-->
 
−
 
−
{{Vspace}}
 
−
 
−
 
−
 
−
{{Vspace}}
 
−
 
−
 
−
<!-- included from "./data/ABC-unit_components.txt", section: "ABC-unit_ask" -->
 
−
 
−
----
 
−
 
−
{{Vspace}}
 
−
 
−
<b>If in doubt, ask!</b> If anything about this learning unit is not clear to you, do not proceed blindly but ask for clarification. Post your question on the course mailing list: others are likely to have similar problems. Or send an email to your instructor.
 
−
 
−
----
 
−
 
−
{{Vspace}}
 
  
 
<div class="about">
 
<div class="about">
Line 169: Line 111:
 
:2017-08-05
 
:2017-08-05
 
<b>Modified:</b><br />
 
<b>Modified:</b><br />
−
:2017-09-11
+
:2020-09-17
 
<b>Version:</b><br />
 
<b>Version:</b><br />
−
:1.1
+
:1.2
 
<b>Version history:</b><br />
 
<b>Version history:</b><br />
 +
*1.2 Maintenance
 
*1.1 Add references to logical fallacies; review older submissions and move some into ''Not (quite) Cargo Cult'' page.
 
*1.1 Add references to logical fallacies; review older submissions and move some into ''Not (quite) Cargo Cult'' page.
 
*1.0 Completed to first live version
 
*1.0 Completed to first live version
 
*0.1 Material collected from previous tutorial
 
*0.1 Material collected from previous tutorial
 
</div>
 
</div>
−
[[Category:ABC-units]]
 
−
<!-- included from "./data/ABC-unit_components.txt", section: "ABC-unit_footer" -->
 
  
 
{{CC-BY}}
 
{{CC-BY}}
  
 +
[[Category:ABC-units]]
 +
{{UNIT}}
 +
{{LIVE}}
 
</div>
 
</div>
 
<!-- [END] -->
 
<!-- [END] -->

Latest revision as of 09:27, 25 September 2020

Cargo Cult Science

(Cargo Cult science, Cargo Cult bioinformatics)


 


Abstract:

Not all activities lead to valuable outcomes and "Cargo Cult Science" is an important metaphor for a class of conceptual problems that are the hallmark of "poor science". The particular issue is that activites are causally disconnected from their claimed beneficial outcomes.

Objectives:
This unit will ...

  • ... introduce the metaphor of a "Cargo Cult", as applied to bioinformatics and science in general;
  • ... illustrate with examples;
  • ... discuss principles to avoid the problem.

Outcomes:
After working through this unit you ...

  • ... can identify "Cargo Cult"-type issues;
  • ... have contributed an example to our collection, and/or;
  • ... have contributed to the discussion in our collection;
  • ... are able to critically evaluate projects and activities regarding whether they can contribute to thier claimed value;
  • ... are able to propose improvements.

Deliverables:

  • Time management: Before you begin, estimate how long it will take you to complete this unit. Then, record in your course journal: the number of hours you estimated, the number of hours you worked on the unit, and the amount of time that passed between start and completion of this unit.
  • Journal: Document your progress in your Course Journal. Some tasks may ask you to include specific items in your journal. Don't overlook these.
  • Insights: If you find something particularly noteworthy about this unit, make a note in your insights! page.

    • Prerequisites:
    You need the following preparation before beginning this unit. If you are not familiar with this material from courses you took previously, you need to prepare yourself from other information sources:

    • Inquiry: The scientific method; evidence based reasoning; how to design, execute and document an experiment; Conjecture, hypothesis and theory.


     



     



     


    Evaluation

    Evaluation: NA

    This unit is not evaluated for course marks.

    Contents

     

    The concept of Cargo cult science was popularized by Richard Feynman in his 1974 Caltech Commencement address. In a nutshell, Feynman pointed out how scientific practices that lack "scientific integrity" are similar to the activities of a premodern spiritual cult in the South Sea islands that developed rituals for attracting goods-bearing supply airplanes by building mock airports.

    The essence of Cargo Cult is not merely poor science. What makes it "Cargo Cult" is a disconnect between form and contents: the form is compelling, but there can't be a rational expectation of a benefit from the activity because there is no causal connection between the activity and the claimed outcome. This is often, but not always due to logical fallacies.

    The topic is interesting for bioinformatics because the deficiencies are often subtle, and hard for the non-expert to spot. To guard against Cargo Cult takes integrity, and practice. A structured approach may be helpful that first clearly identifies the hoped-for outcome, then defines the proposed activities, then asks in specific detail how the outcome would be caused by the activity. Causation is key here - many examples of Cargo Cult behaviour are based on a mistaken belief in causation, where actually merely a correlation was observed. But you have to be careful: the fact that causation has not been shown does not prove it is absent. And even if causation is absent, that does not automatically make the behaviour invalid: sometimes you are right for the wrong reason. Both cases are not Cargo Cult. Rather, it is characteristic for situations we should label as Cargo Cult that there "is no cargo in the system": you are looking in the wrong place, you don't have a control or reference value, you don't understand your data - or similar problems.


     

    Task:

    Further reading, links and resources

    Wikipedia: List of Logical Fallacies a very comprehensive resource. One would wish that the presence of such a list itself would have a beneficial effect on science.

    Notes

    1. ↑ Make sure your example has not already been posted by someone else - it would be Cargo Cult to post it again.


     


    About ...
     
    Author:

    Boris Steipe <boris.steipe@utoronto.ca>

    Created:

    2017-08-05

    Modified:

    2020-09-17

    Version:

    1.2

    Version history:

    • 1.2 Maintenance
    • 1.1 Add references to logical fallacies; review older submissions and move some into Not (quite) Cargo Cult page.
    • 1.0 Completed to first live version
    • 0.1 Material collected from previous tutorial

    CreativeCommonsBy.png This copyrighted material is licensed under a Creative Commons Attribution 4.0 International License. Follow the link to learn more.

    Retrieved from "http://steipe.biochemistry.utoronto.ca/abc/index.php?title=FND-Cargo_Cult&oldid=11719"