View source for RPR-Literate programming

<div id="ABC">
<div style="padding:5px; border:1px solid #000000; background-color:#b3dbce; font-size:300%; font-weight:400; color: #000000; width:100%;">
Literate Programming with R
<div style="padding:5px; margin-top:20px; margin-bottom:10px; background-color:#b3dbce; font-size:30%; font-weight:200; color: #000000; ">
((Draft) Literate programming principles; R Markdown; R Notebooks)
</div>
</div>

{{Smallvspace}}


<div style="padding:5px; border:1px solid #000000; background-color:#b3dbce33; font-size:85%;">
<div style="font-size:118%;">
<b>Abstract:</b><br />
<section begin=abstract />
Documentation of results using R markdown and R notebooks.
<section end=abstract />
</div>
<!-- ============================  -->
<hr>
<table>
<tr>
<td style="padding:10px;">
<b>Objectives:</b><br />
This unit will ...
* ... introduce the philosophy behind "Literate Programming";
* ... teach the practice with an example that uses knitr in the RStudio environment;
* ... point you to R notebooks;
</td>
<td style="padding:10px;">
<b>Outcomes:</b><br />
After working through this unit you ...
* ... can produce your own "Literate Programs" with knitr or in an R notebook.
</td>
</tr>
</table>
<!-- ============================  -->
<hr>
<b>Deliverables:</b><br />
<section begin=deliverables />
<li><b>Time management</b>: Before you begin, estimate how long it will take you to complete this unit. Then, record in your course journal: the number of hours you estimated, the number of hours you worked on the unit, and the amount of time that passed between start and completion of this unit.</li>
<li><b>Journal</b>: Document your progress in your [[FND-Journal|Course Journal]]. Some tasks may ask you to include specific items in your journal. Don't overlook these.</li>
<li><b>Insights</b>: If you find something particularly noteworthy about this unit, make a note in your [[ABC-Insights|'''insights!''' page]].</li>
<section end=deliverables />
<!-- ============================  -->
<hr>
<section begin=prerequisites />
<b>Prerequisites:</b><br />
*[[RPR-Introduction|RPR-Introduction (Introduction to R)]]
<section end=prerequisites />
<!-- ============================  -->
</div>

{{Smallvspace}}



{{Smallvspace}}


__TOC__

{{Vspace}}


=== Evaluation ===
<b>Evaluation: NA</b><br />
<div style="margin-left: 2rem;">This unit is not evaluated for course marks.</div>
== Contents ==
{{WP|Literate programming|'''Literate programming'''}} is an idea that software is best described in a natural language, focussing on the logic of the program, i.e. the '''why''' of code, not the '''what'''. The goal is to ensure that model, code, and documentation become a single unit, and that all this information is stored in one and only one location. The product should be ''consistent'' between its described goals and its implementation, ''seamless'' in capturing the process from start (data input) to end (visualization, interpretation), and ''reversible'' (between analysis, design and implementation).

In literate programming, narrative and computer code are kept in the same file. This source document is typically written in ''Markdown'' or ''LaTeX'' syntax and includes the programming code as well as text annotations, tables, formulas etc. The supporting software can '''weave''' human-readable documentation from this, or '''tangle''' executable code. Literate programming with both Markdown and LaTex is supported by '''R Studio''' and this makes the '''R Studio''' interface a useful development environment for this paradigm. While it is easy to edit source files with a different editor and process files in base '''R''' after loading the {{R|utils|Sweave()|Sweave()</code> and <code>Stangle()}} functions or the {{R|knitr}} package. In our context here we will use '''R Studio''' because it conveniently integrates the functionality we need.

{{R|knitr}} is an '''R''' package for {{WP|literate programming}}. It is integrated with [http://www.rstudio.com '''R Studio'''].

{{Vspace}}

==RMarkdown==

Markdown is an extremely simple and informal way of structuring documents that is useful if for some reason you feel html is too complicated. That's really all it does: format documents in a simple way so they can be displayed as Web pages. For Markdown documentation, [http://daringfireball.net/projects/markdown/basics '''see here.''']. The concept is quite similar to Wiki markup syntax, the syntax is (regrettably) different, and for a number of features there there are (regrettably) several different ways to achieve the same results.

[http://rmarkdown.rstudio.com '''RMarkdown'''] is an '''R''' package that is integrated with '''R Studio''' and allows integrating '''R''' code with Markdown documents. knitr can work with Markdown files, and this gives additional output options, such as PDF and MSWord documents.


Let's give it a try: we'll write and document an '''R''' function that will find us a random phobia to ponder on.

{{task|1=
* Open your normal '''R Studio''' session
* Select '''File &rarr; New File &rarr; RMarkdown...'''. When you do this the first time, '''R Studio''' will ask you whether you want to install/update a number of required packages. Click '''Yes'''.
* Enter "Random Phobia" as the '''Title''' and your name as the '''Author''', select to create a '''Document''', and check '''HTML''' as the default output option.
'''R Studio''' will load some default text and markup into the script pane which we can edit.

* Choose '''Help &rarr; Cheatssheets &rarr; R Markdown Cheat Sheet''' and '''R Markdown Reference Guide''' to download two PDFs via your browser. Browse the contents to get an idea where you can clarify concepts as you go through this example.

Let's introduce our plan:

* We'll create a markdown document containing code and explanations
* We'll knit it into an HTML document and examine it
* Then we'll have a look at its structure, to learn how it works.

; Creating the Document
* First, delete everything except the header block from your new Markdown document.
* There is a document in the data folder called <tt>data/RandomPhobiaPage.txt</tt>. Open that, copy its entire contents, and paste it under the header block.
* Save the document as <tt>myScripts/RandomPhobias.Rmd</tt>

; Knitting the Document to HTML
* Right above the edit window, next to the search (looking glass) icon, there is an icon of a ball of wool with a knitting needle ... click on '''Knit'''.
* The .HTML document will be created and opened.


; Inspecting the Markdown source


Note the following Markdown elements in the code:

*After the header-block, three "backticks" delimit a functional block (curly braces). In this case the block injects a local bit of .css to create striped tables later on. This is entirely optional and can be deleted in your own code if you have no need for it.

* Then there is another code block. This one is crucial. the first letter in the curly braces is <code>r</code> and this bit of code will be run as R code. <code>include=FALSE</code> ensures the code block is not actually shown in the output, but it sets a knitr:: option. This is what is know as a "code chunk". It is delimited by three backticks <code>```</code> and has directives and options for the chunk in the first line. It is labelled as '''R''' code, and note that after the <code>{r </code> we have added an (optional) label for the chunk. That is useful, because we can rapidly navigate between chunks (click on the navigation menu at the ''bottom'' of the script pane), and we can refer to the labels to execute chunks that are coded later in the document at an earlier stage. This is an important idea of literate programming: the flow of the document should not be determined by the requirements of the code, but by the logic of the narrative.  '''TLDR;''' label your chunks. It's useful.

:Other options can be added after a comma, for example we can suppress printing of a chunk into the document altogether, if we think it is not relevant for the document, by adding the option <code>echo{{=}}FALSE</code><ref>For a complete list of chunk options, see [http://yihui.name/knitr/options/ the documentation by knitr's author, Xie Yihui].</ref>.

* Then the text begins. It is writting in markdown syntax which is a simple way to annotate text. Note the conventions to create headers and links etc.

* To execute a particular chunk, simply place the cursor into the chunk and select '''Chunks &rarr; Run Current Chunk''' from the menu at the top of the script pane.

* More text describes how we screen-scrape tables from a Wikipedia page, and the code chunks run that code.

:Some things to note here:

** Enclosing a piece of text in "backticks" <code>`Text...`</code> formats that text as "code" - typically in a fixed-width font.
** For this chunk we have set the option <code>cache</code> as <code>TRUE</code>. This is a very useful and well conceived mechanism that avoids recomputing code that takes a long time or should otherwise be limited. The results of a cached chunk of code are stored locally and retrieved when the file is ''weaved''. Only if anything within the chunk is changed (or <code>cache</code> is set to <code>FALSE</code>), is the chunk evaluated again. This prevents us from excessively pounding on Websites as we develop our script, which can save a lot of time as our projects become large and the calculations become complex.

* As the code continues, we have more of this mix of markup code and '''R''' code. Two important options in the chunk header:
* <code>echo{{=}}FALSE</code> prevents the contents of the chunk to be printed. We don't want this code in our output, we only want the result.
* <code>results{{=}}'asis'</code> prevents the results from being marked up. The raw HTML is sent to the output document.

But note the following: the code chunk that creates the table calls a function <code>randRow(M)</code> that we have not defined yet. In an '''R''' script this would not work. But in a knitr document we can '''reference''' a chunk of code anywhere in (and outside) of the document and thus define our function before the <code>renderPhobiaTable</code> chunk is executed. This is important for literate programming, where we don't want to be constrained by the requirements of the code.

You have a working markdown page, and that should go a long way to help you write your own.

}}


{{Vspace}}

==R Notebooks==

R Notebooks take the concept into the RStudio editor itself, rather than constructing a Webpage. On one hand, you become dependent on the RStudio editor, on the other hand, you directly edit and comment as you are developing. This is '''true''' "Literate Programming".

{{task|1=
Read about the concept [http://rmarkdown.rstudio.com/r_notebooks.html '''here'''] and follow along with the exercise.

}}

{{Vspace}}

== Further reading, links and resources ==
<div class="reference-box">[https://www.rstudio.com/wp-content/uploads/2015/03/rmarkdown-reference.pdf R Markdown Reference Guide] (PDF @ RStudio)</div>
<div class="reference-box">[https://www.r-bloggers.com/using-rvest-to-scrape-an-html-table/ Using rvest to Scrape an HTML Table] (R Bloggers)</div>
{{Smallvspace}}
<div class="reference-box">[https://blog.ouseful.info/2017/11/15/programming-meh-lets-teach-how-to-write-computational-essays-instead/ Thoughts on notebooks and literate programming] (Tony Hirst, via R Bloggers)</div>


== Notes ==
<references />

{{Vspace}}


<div class="about">
<b>About ... </b><br />
&nbsp;<br />
<b>Author:</b><br />
:Boris Steipe (boris.steipe@utoronto.ca)
<b>Created:</b><br />
:2017-09-17
<b>Modified:</b><br />
:2020-09-25
<b>Version:</b><br />
:2.0
<b>Version history:</b><br />
*2.0 Update after complete rewrite of sample .Rmd - don't assemble the page piecewise
*1.2 Change from require() to requireNamespace() and use &lt;package&gt;::&lt;function&gt;() idiom.
*1.1 bugfix, comment on header tags in a table, add an eval question
*1.0 First live version
*0.1 First stub
</div>
<!-- included from "./data/ABC-unit_components.txt", section: "ABC-unit_footer" -->

{{CC-BY}}

[[Category:ABC-units]]
{{UNIT}}
{{LIVE}}
</div>
<!-- [END] -->