Difference between revisions of "RPR-OBJECTS-Data frames"

From "A B C"
Jump to navigation Jump to search
m
 
(15 intermediate revisions by the same user not shown)
Line 1: Line 1:
<div id="BIO">
+
<div id="ABC">
  <div class="b1">
+
<div style="padding:5px; border:1px solid #000000; background-color:#b3dbce; font-size:300%; font-weight:400; color: #000000; width:100%;">
R Dataframes
+
R "data frames""
  </div>
+
<div style="padding:5px; margin-top:20px; margin-bottom:10px; background-color:#b3dbce; font-size:30%; font-weight:200; color: #000000; ">
 
+
(R data frames)
  {{Vspace}}
+
</div>
 
 
<div class="keywords">
 
<b>Keywords:</b>&nbsp;
 
Dataframes
 
 
</div>
 
</div>
  
{{Vspace}}
+
{{Smallvspace}}
  
  
__TOC__
+
<div style="padding:5px; border:1px solid #000000; background-color:#b3dbce33; font-size:85%;">
 
+
<div style="font-size:118%;">
{{Vspace}}
+
<b>Abstract:</b><br />
 
 
 
 
{{DEV}}
 
 
 
{{Vspace}}
 
 
 
 
 
</div>
 
<div id="ABC-unit-framework">
 
== Abstract ==
 
 
<section begin=abstract />
 
<section begin=abstract />
<!-- included from "../components/RPR-Objects-Dataframes.components.wtxt", section: "abstract" -->
+
Introduction to data frames: how to create, and modify them and how to retrieve data.
...
 
 
<section end=abstract />
 
<section end=abstract />
 +
</div>
 +
<!-- ============================  -->
 +
<hr>
 +
<table>
 +
<tr>
 +
<td style="padding:10px;">
 +
<b>Objectives:</b><br />
 +
This unit will ...
 +
* ... introduce R data frames;
 +
* ... cover a number of basic operations.
 +
</td>
 +
<td style="padding:10px;">
 +
<b>Outcomes:</b><br />
 +
After working through this unit you ...
 +
* ... know how to create and manipulate data frames;
 +
* ... can access and change individual elements;
 +
* ... can extract rows, columns, and append new data rows;
 +
</td>
 +
</tr>
 +
</table>
 +
<!-- ============================  -->
 +
<hr>
 +
<b>Deliverables:</b><br />
 +
<section begin=deliverables />
 +
<ul>
 +
<li><b>Time management</b>: Before you begin, estimate how long it will take you to complete this unit. Then, record in your course journal: the number of hours you estimated, the number of hours you worked on the unit, and the amount of time that passed between start and completion of this unit.</li>
 +
<li><b>Journal</b>: Document your progress in your [[FND-Journal|Course Journal]]. Some tasks may ask you to include specific items in your journal. Don't overlook these.</li>
 +
<li><b>Insights</b>: If you find something particularly noteworthy about this unit, make a note in your [[ABC-Insights|'''insights!''' page]].</li>
 +
</ul>
 +
<section end=deliverables />
 +
<!-- ============================  -->
 +
<hr>
 +
<section begin=prerequisites />
 +
<b>Prerequisites:</b><br />
 +
This unit builds on material covered in the following prerequisite units:<br />
 +
*[[RPR-Objects-Vectors|RPR-Objects-Vectors (R scalars and vectors)]]
 +
<section end=prerequisites />
 +
<!-- ============================  -->
 +
</div>
  
{{Vspace}}
+
{{Smallvspace}}
 
 
  
== This unit ... ==
 
=== Prerequisites ===
 
<!-- included from "../components/RPR-Objects-Dataframes.components.wtxt", section: "prerequisites" -->
 
<!-- included from "ABC-unit_components.wtxt", section: "notes-prerequisites" -->
 
You need to complete the following units before beginning this one:
 
*[[RPR-Objects-Vectors]]
 
  
{{Vspace}}
 
  
 +
{{Smallvspace}}
  
=== Objectives ===
 
<!-- included from "../components/RPR-Objects-Dataframes.components.wtxt", section: "objectives" -->
 
...
 
  
{{Vspace}}
+
__TOC__
 
 
 
 
=== Outcomes ===
 
<!-- included from "../components/RPR-Objects-Dataframes.components.wtxt", section: "outcomes" -->
 
...
 
 
 
{{Vspace}}
 
 
 
 
 
=== Deliverables ===
 
<!-- included from "../components/RPR-Objects-Dataframes.components.wtxt", section: "deliverables" -->
 
<!-- included from "ABC-unit_components.wtxt", section: "deliverables-time_management" -->
 
*<b>Time management</b>: Before you begin, estimate how long it will take you to complete this unit. Then, record in your course journal: the number of hours you estimated, the number of hours you worked on the unit, and the amount of time that passed between start and completion of this unit.
 
<!-- included from "ABC-unit_components.wtxt", section: "deliverables-journal" -->
 
*<b>Journal</b>: Document your progress in your [[FND-Journal|Course Journal]]. Some tasks may ask you to include specific items in your journal. Don't overlook these.
 
<!-- included from "ABC-unit_components.wtxt", section: "deliverables-insights" -->
 
*<b>Insights</b>: If you find something particularly noteworthy about this unit, make a note in your [[ABC-Insights|'''insights!''' page]].
 
  
 
{{Vspace}}
 
{{Vspace}}
Line 72: Line 69:
  
 
=== Evaluation ===
 
=== Evaluation ===
<!-- included from "../components/RPR-Objects-Dataframes.components.wtxt", section: "evaluation" -->
 
<!-- included from "ABC-unit_components.wtxt", section: "eval-none" -->
 
 
<b>Evaluation: NA</b><br />
 
<b>Evaluation: NA</b><br />
:This unit is not evaluated for course marks.
+
<div style="margin-left: 2rem;">This unit is not evaluated for course marks.</div>
 +
== Contents ==
  
{{Vspace}}
+
{{task| 1=
 +
* Load the <code>R-Exercise_BasicSetup</code> project in RStudio if you don't already have it open.
 +
* Type <code>init()</code> as instructed after the project has loaded.
 +
* Continue below.
 +
}}
  
  
</div>
 
<div id="BIO">
 
== Contents ==
 
<!-- included from "../components/RPR-Objects-Dataframes.components.wtxt", section: "contents" -->
 
 
===Data frames===
 
===Data frames===
  
 +
Data frames are  the most frequently used type of data object for bioinformatics in R; they emulate our mental model of data in a spreadsheet and can be used to implement entity-relationship datamodels. They are more flexible than vectors or matrices, but they are easier to work with than lists.
  
Data frames are one of the most important types of data object in bioinformatics because they emulate our mental model of data in a spreadsheet and can be used to implement datamodels.
+
Usually the result of reading external data from an input file is a data frame. The file below is included with the <code>R-Exercise-BasicSetup</code> project files - it is called <code>plasmidData.tsv</code>,<ref>The two most important formats for generic text-based datafiles are "'''tab'''"-separated values (<code>.tsv</code>) and "'''comma'''"-separated values (<code>.csv</code>).</ref> and you can click on it in the Files Pane to open and inspect it.
 
 
Usually the result of reading external data from an input file is a data frame. The file below is included with the BasicSetup project files - it is called <code>plasmidData.tsv</code>, and you can click on it in the Files Pane to open and inspect it.
 
  
 
  Name Size Marker Ori Sites
 
  Name Size Marker Ori Sites
Line 96: Line 91:
 
  pACYC184 4245 Tet, Cam p15A ClaI, HindIII
 
  pACYC184 4245 Tet, Cam p15A ClaI, HindIII
  
This data set uses tabs as column separators and it has a header line. Similar files can be exported from Excel or other spreadsheet programs. Read this as a data frame as follows:
+
This data set uses tabs as value separators and it has a header line. Similar files can be exported from Excel or other spreadsheet programs. Note that the file contains commas '''within''' fields. Read this as a data frame as follows:
  
<source lang="rsplus">
+
<pre>
plasmidData <- read.table("plasmidData.tsv", sep="\t", header=TRUE, stringsAsFactors = FALSE)
+
( plasmidData <- read.table("plasmidData.tsv",
plasmidData   # show what the data frame contains
+
                            sep = "\t",
</source>
+
                            header = TRUE )
 
+
objectInfo(plasmidData)
Note the argument {{c|stringsAsFactors {{=}} FALSE}}. If this is {{c|TRUE}} instead, '''R''' will convert all strings in the input to factors and this may lead to problems. Make it a habit to turn this behaviour off, you can always turn a column of strings into factors when you actually mean to have factors.
+
</pre>
  
 
You can view the data frame contents by clicking on the spreadsheet icon behind its name in the Environment Pane.
 
You can view the data frame contents by clicking on the spreadsheet icon behind its name in the Environment Pane.
Line 109: Line 104:
 
{{Vspace}}
 
{{Vspace}}
  
 +
===Basic operations===
  
 +
Here are some basic operations with the data frame. Try them, and experiment. If you break the object by mistake, you can just recreate it by reading the source file again:
  
{{Vspace}}
+
<pre>
 +
rownames(plasmidData) <- plasmidData[ , 1]  # assigns the contents of column 1 as rownames
 +
nrow(plasmidData)
 +
ncol(plasmidData)
 +
objectInfo(plasmidData)
  
  
== Further reading, links and resources ==
+
x <- plasmidData[2, ]  # assign one row to a variable
<!-- {{#pmid: 19957275}} -->
+
objectInfo(x)  # This is also a data frame! One row. It has to be, because
<!-- {{WWW|WWW_GMOD}} -->
+
              # it contains elements of type chr and of type int!
<!-- <div class="reference-box">[http://www.ncbi.nlm.nih.gov]</div> -->
 
  
{{Vspace}}
+
plasmidData["pBR322", ]  # retrieve one row: different syntax, same thing
  
 +
( s <- plasmidData["pBR322", "Size"] )  # one element
 +
plasmidData["pBR322", "Size"] <- "???"  # change one element
 +
plasmidData["pBR322", ]                # Note that this is noew a string, not a number
 +
objectInfo(plasmidData)                # In fact, the assignment has changed the
 +
                                        # type of the the whole column. Remember:
 +
                                        # in a data.frame, all elements of one column
 +
                                        # have the same type.
  
== Notes ==
+
plasmidData <- plasmidData[-2, ]  # remove one row
<!-- included from "../components/RPR-Objects-Dataframes.components.wtxt", section: "notes" -->
+
objectInfo(plasmidData)
<!-- included from "ABC-unit_components.wtxt", section: "notes" -->
 
<references />
 
  
{{Vspace}}
+
plasmidData <- rbind(plasmidData, x)  # add it back at the end
 +
objectInfo(plasmidData)
  
 +
# add a new row from scratch:
 +
plasmidData <- rbind(plasmidData, data.frame(Name = "pMAL-p5x",
 +
                                                    Size = 5752,
 +
                                                    Marker = "Amp",
 +
                                                    Ori = "pMB1",
 +
                                                    Sites = "SacI, AvaI, HindIII"))
 +
objectInfo(plasmidData)
  
</div>
+
( x <- plasmidData[ , 2] )    # retrieve one column by index
<div id="ABC-unit-framework">
+
  plasmidData[ , "Size"]      # retrieve one column by name
== Self-evaluation ==
+
objectInfo(plasmidData)      # now a vector!
<!-- included from "../components/RPR-Objects-Dataframes.components.wtxt", section: "self-evaluation" -->
 
<!--
 
=== Question 1===
 
  
Question ...
+
# That may be surprising behaviour. When you retrieve a single column from a
 +
# dataframe it is (silently) turned into a vector (unless you explicitly
 +
# tell R not to do that - e.g. plasmidData[ , "Size", drop = FALSE]). To make the
 +
# nature of this data as a vector more expolicit, I usually use a different
 +
# and equivalent syntax: the "$" operator
  
<div class="toccolours mw-collapsible mw-collapsed" style="width:800px">
+
plasmidData$Size
Answer ...
+
objectInfo(plasmidData$Size)
<div class="mw-collapsible-content">
 
Answer ...
 
  
</div>
+
# Note: the $ operator always returns a vector. And, the column name is _NOT_
  </div>
+
# placed in quotation marks. This is the syntax we usually will use throughout
 +
# the course.
  
  {{Vspace}}
+
</pre>
  
-->
+
{{task|1=
  
{{Vspace}}
+
The rowname of the new row of <code>plasmidData</code> is now "1". It should be "pMAL-p5x". Fix this.
  
 +
}}
  
  
 
{{Vspace}}
 
{{Vspace}}
  
 
+
== Notes ==
<!-- included from "ABC-unit_components.wtxt", section: "ABC-unit_ask" -->
+
<references />
 
 
----
 
  
 
{{Vspace}}
 
{{Vspace}}
  
<b>If in doubt, ask!</b> If anything about this learning unit is not clear to you, do not proceed blindly but ask for clarification. Post your question on the course mailing list: others are likely to have similar problems. Or send an email to your instructor.
 
 
----
 
 
{{Vspace}}
 
  
 
<div class="about">
 
<div class="about">
Line 178: Line 185:
 
:2017-08-05
 
:2017-08-05
 
<b>Modified:</b><br />
 
<b>Modified:</b><br />
:2017-08-05
+
:2020-09-18
 
<b>Version:</b><br />
 
<b>Version:</b><br />
:0.1
+
:1.1
 
<b>Version history:</b><br />
 
<b>Version history:</b><br />
*0.1 First stub
+
*1.1 Remove stringsAsFactors, no longer an issue
 +
*1.0.1 Maintenance
 +
*1.0 Completed to first live version
 +
*0.1 Material collected from previous tutorial
 
</div>
 
</div>
[[Category:ABC-units]]
 
<!-- included from "ABC-unit_components.wtxt", section: "ABC-unit_footer" -->
 
  
 
{{CC-BY}}
 
{{CC-BY}}
  
 +
[[Category:ABC-units]]
 +
{{UNIT}}
 +
{{LIVE}}
 
</div>
 
</div>
 
<!-- [END] -->
 
<!-- [END] -->

Latest revision as of 01:06, 6 September 2021

R "data frames""

(R data frames)


 


Abstract:

Introduction to data frames: how to create, and modify them and how to retrieve data.


Objectives:
This unit will ...

  • ... introduce R data frames;
  • ... cover a number of basic operations.

Outcomes:
After working through this unit you ...

  • ... know how to create and manipulate data frames;
  • ... can access and change individual elements;
  • ... can extract rows, columns, and append new data rows;

Deliverables:

  • Time management: Before you begin, estimate how long it will take you to complete this unit. Then, record in your course journal: the number of hours you estimated, the number of hours you worked on the unit, and the amount of time that passed between start and completion of this unit.
  • Journal: Document your progress in your Course Journal. Some tasks may ask you to include specific items in your journal. Don't overlook these.
  • Insights: If you find something particularly noteworthy about this unit, make a note in your insights! page.

Prerequisites:
This unit builds on material covered in the following prerequisite units:


 



 



 


Evaluation

Evaluation: NA

This unit is not evaluated for course marks.

Contents

Task:

  • Load the R-Exercise_BasicSetup project in RStudio if you don't already have it open.
  • Type init() as instructed after the project has loaded.
  • Continue below.


Data frames

Data frames are the most frequently used type of data object for bioinformatics in R; they emulate our mental model of data in a spreadsheet and can be used to implement entity-relationship datamodels. They are more flexible than vectors or matrices, but they are easier to work with than lists.

Usually the result of reading external data from an input file is a data frame. The file below is included with the R-Exercise-BasicSetup project files - it is called plasmidData.tsv,[1] and you can click on it in the Files Pane to open and inspect it.

Name	Size	Marker	Ori	Sites
pUC19	2686	Amp	ColE1	EcoRI, SacI, SmaI, BamHI, XbaI, PstI, HindIII
pBR322	4361	Amp, Tet	ColE1	EcoRI, ClaI, HindIII
pACYC184	4245	Tet, Cam	p15A	ClaI, HindIII

This data set uses tabs as value separators and it has a header line. Similar files can be exported from Excel or other spreadsheet programs. Note that the file contains commas within fields. Read this as a data frame as follows:

( plasmidData <- read.table("plasmidData.tsv",
                            sep = "\t",
                            header = TRUE )
objectInfo(plasmidData)

You can view the data frame contents by clicking on the spreadsheet icon behind its name in the Environment Pane.


 

Basic operations

Here are some basic operations with the data frame. Try them, and experiment. If you break the object by mistake, you can just recreate it by reading the source file again:

rownames(plasmidData) <- plasmidData[ , 1]  # assigns the contents of column 1 as rownames
nrow(plasmidData)
ncol(plasmidData)
objectInfo(plasmidData)


x <- plasmidData[2, ]  # assign one row to a variable
objectInfo(x)  # This is also a data frame! One row. It has to be, because
               # it contains elements of type chr and of type int!

plasmidData["pBR322", ]  # retrieve one row: different syntax, same thing

( s <- plasmidData["pBR322", "Size"] )  # one element
plasmidData["pBR322", "Size"] <- "???"  # change one element
plasmidData["pBR322", ]                 # Note that this is noew a string, not a number
objectInfo(plasmidData)                 # In fact, the assignment has changed the
                                        # type of the the whole column. Remember:
                                        # in a data.frame, all elements of one column
                                        # have the same type.

plasmidData <- plasmidData[-2, ]  # remove one row
objectInfo(plasmidData)

plasmidData <- rbind(plasmidData, x)  # add it back at the end
objectInfo(plasmidData)

# add a new row from scratch:
plasmidData <- rbind(plasmidData, data.frame(Name = "pMAL-p5x",
                                                     Size = 5752,
                                                     Marker = "Amp",
                                                     Ori = "pMB1",
                                                     Sites = "SacI, AvaI, HindIII"))
objectInfo(plasmidData)

( x <- plasmidData[ , 2] )    # retrieve one column by index
  plasmidData[ , "Size"]      # retrieve one column by name
objectInfo(plasmidData)       # now a vector!

# That may be surprising behaviour. When you retrieve a single column from a
# dataframe it is (silently) turned into a vector (unless you explicitly
# tell R not to do that - e.g. plasmidData[ , "Size", drop = FALSE]). To make the
# nature of this data as a vector more expolicit, I usually use a different
# and equivalent syntax: the "$" operator

plasmidData$Size
objectInfo(plasmidData$Size)

# Note: the $ operator always returns a vector. And, the column name is _NOT_
# placed in quotation marks. This is the syntax we usually will use throughout
# the course.

Task:
The rowname of the new row of plasmidData is now "1". It should be "pMAL-p5x". Fix this.


 

Notes

  1. The two most important formats for generic text-based datafiles are "tab"-separated values (.tsv) and "comma"-separated values (.csv).


 


About ...
 
Author:

Boris Steipe <boris.steipe@utoronto.ca>

Created:

2017-08-05

Modified:

2020-09-18

Version:

1.1

Version history:

  • 1.1 Remove stringsAsFactors, no longer an issue
  • 1.0.1 Maintenance
  • 1.0 Completed to first live version
  • 0.1 Material collected from previous tutorial

CreativeCommonsBy.png This copyrighted material is licensed under a Creative Commons Attribution 4.0 International License. Follow the link to learn more.