Difference between revisions of "RPR-Data-Import"

From "A B C"
Jump to navigation Jump to search
m
m
 
(2 intermediate revisions by the same user not shown)
Line 1: Line 1:
 
<div id="ABC">
 
<div id="ABC">
−
<div style="padding:5px; border:1px solid #000000; background-color:#b3dbce; font-size:300%; font-weight:400; color: #000000; width:100%;">
+
<div style="padding:5px; border:1px solid #000000; background-color:#f2fafa; font-size:300%; font-weight:400; color: #000000; width:100%;">
 
Importing data in R
 
Importing data in R
−
<div style="padding:5px; margin-top:20px; margin-bottom:10px; background-color:#b3dbce; font-size:30%; font-weight:200; color: #000000; ">
+
<div style="padding:5px; margin-top:20px; margin-bottom:10px; background-color:#f2fafa; font-size:30%; font-weight:200; color: #000000; ">
 
(Data import from unstructured text, structured text, spreadsheets, online repositories, and Web pages)
 
(Data import from unstructured text, structured text, spreadsheets, online repositories, and Web pages)
 
</div>
 
</div>
Line 10: Line 10:
  
  
−
<div style="padding:5px; border:1px solid #000000; background-color:#b3dbce33; font-size:85%;">
+
<div style="padding:5px; border:1px solid #000000; background-color:#f2fafa33; font-size:85%;">
 
<div style="font-size:118%;">
 
<div style="font-size:118%;">
 
<b>Abstract:</b><br />
 
<b>Abstract:</b><br />
Line 35: Line 35:
 
<b>Deliverables:</b><br />
 
<b>Deliverables:</b><br />
 
<section begin=deliverables />
 
<section begin=deliverables />
−
<!-- included from "./data/ABC-unit_components.txt", section: "deliverables-time_management" -->
+
<li><b>Time management</b>: Before you begin, estimate how long it will take you to complete this unit. Then, record in your course journal: the number of hours you estimated, the number of hours you worked on the unit, and the amount of time that passed between start and completion of this unit.</li>
−
*<b>Time management</b>: Before you begin, estimate how long it will take you to complete this unit. Then, record in your course journal: the number of hours you estimated, the number of hours you worked on the unit, and the amount of time that passed between start and completion of this unit.
+
<li><b>Journal</b>: Document your progress in your [[FND-Journal|Course Journal]]. Some tasks may ask you to include specific items in your journal. Don't overlook these.</li>
−
<!-- included from "./data/ABC-unit_components.txt", section: "deliverables-journal" -->
+
<li><b>Insights</b>: If you find something particularly noteworthy about this unit, make a note in your [[ABC-Insights|'''insights!''' page]].</li>
−
*<b>Journal</b>: Document your progress in your [[FND-Journal|Course Journal]]. Some tasks may ask you to include specific items in your journal. Don't overlook these.
 
−
<!-- included from "./data/ABC-unit_components.txt", section: "deliverables-insights" -->
 
−
*<b>Insights</b>: If you find something particularly noteworthy about this unit, make a note in your [[ABC-Insights|'''insights!''' page]].
 
 
<section end=deliverables />
 
<section end=deliverables />
 
<!-- ============================  -->
 
<!-- ============================  -->
Line 54: Line 51:
  
  
 +
{{SLEEP}}
  
 
{{Smallvspace}}
 
{{Smallvspace}}
Line 63: Line 61:
  
  
 +
=== Evaluation ===
 +
<b>Evaluation: NA</b><br />
 +
<div style="margin-left: 2rem;">This unit is not evaluated for course marks.</div>
 
== Contents ==
 
== Contents ==
−
<!-- included from "./components/RPR-Data-Import.components.txt", section: "contents" -->
 
 
* Line endings
 
* Line endings
 
* Unstructured text files, headers, skip, omit, and rownames
 
* Unstructured text files, headers, skip, omit, and rownames
Line 77: Line 77:
 
* XML - libraries and cpath
 
* XML - libraries and cpath
 
* Binary Data
 
* Binary Data
−
* R objects: save() and load()
+
* text objects: readLines() and writeLines()
 +
* R objects: save() and load(); saveRDS() and readRDS().
  
  
Line 133: Line 134:
  
 
-->
 
-->
−
== Notes ==
 
−
<!-- included from "./components/RPR-Data-Import.components.txt", section: "notes" -->
 
−
<!-- included from "./data/ABC-unit_components.txt", section: "notes" -->
 
−
<references />
 
 
== Further reading, links and resources ==
 
== Further reading, links and resources ==
 
<!-- Formatting exqmples:
 
<!-- Formatting exqmples:
Line 142: Line 139:
 
<div class="reference-box">[http://www.ncbi.nlm.nih.gov]</div>
 
<div class="reference-box">[http://www.ncbi.nlm.nih.gov]</div>
 
-->
 
-->
 +
== Notes ==
 +
<references />
  
 
{{Vspace}}
 
{{Vspace}}
  
−
 
−
<!-- included from "./data/ABC-unit_components.txt", section: "ABC-unit_ask" -->
 
−
 
−
----
 
−
 
−
{{Vspace}}
 
−
 
−
<b>If in doubt, ask!</b> If anything about this learning unit is not clear to you, do not proceed blindly but ask for clarification. Post your question on the course mailing list: others are likely to have similar problems. Or send an email to your instructor.
 
−
 
−
----
 
−
 
−
{{Vspace}}
 
  
 
<div class="about">
 
<div class="about">
Line 174: Line 161:
 
*0.1 First stub
 
*0.1 First stub
 
</div>
 
</div>
−
[[Category:ABC-units]]
 
 
<!-- included from "./data/ABC-unit_components.txt", section: "ABC-unit_footer" -->
 
<!-- included from "./data/ABC-unit_components.txt", section: "ABC-unit_footer" -->
  
 
{{CC-BY}}
 
{{CC-BY}}
  
 +
[[Category:ABC-units]]
 +
{{UNIT}}
 +
{{SLEEP}}
 
</div>
 
</div>
 
<!-- [END] -->
 
<!-- [END] -->

Latest revision as of 01:41, 23 September 2020

Importing data in R

(Data import from unstructured text, structured text, spreadsheets, online repositories, and Web pages)


 


Abstract:

Practical for data import from text files, spreadsheets and Web pages.

Objectives:
...

Outcomes:
...

Deliverables:

  • Time management: Before you begin, estimate how long it will take you to complete this unit. Then, record in your course journal: the number of hours you estimated, the number of hours you worked on the unit, and the amount of time that passed between start and completion of this unit.
  • Journal: Document your progress in your Course Journal. Some tasks may ask you to include specific items in your journal. Don't overlook these.
  • Insights: If you find something particularly noteworthy about this unit, make a note in your insights! page.

    • Prerequisites:


     


    This page is not currently being maintained since it is not part of active learning sections.


     



     


    Evaluation

    Evaluation: NA

    This unit is not evaluated for course marks.

    Contents

    • Line endings
    • Unstructured text files, headers, skip, omit, and rownames
    • Caution with stringsAsFactors
    • Caution with coerced data
    • Text files with keywords
    • Text-files: changing state
    • Text-files: csv and tsv (and Excel sheets)
    • Text-files: slurping, chunking and streaming
    • curl
    • httr GET and POST
    • XML - libraries and cpath
    • Binary Data
    • text objects: readLines() and writeLines()
    • R objects: save() and load(); saveRDS() and readRDS().


    Regex for screenscraping example:

    Screenscraping

    Write a PHP program that screenscrapes resolution data for a protein structure file from the PDB. 

     Here is a link to a PDB record to illustrate the URL format.

       Fetch the contents of the URL into a string. Use a regex that captures the data you want to retrieve as part of some unique pattern in the HTML source. Print.

       ;The regex:
    :/\s*(\d+\.\d+)/ *   identifying tag for the information we are looking for, ... 
       *\s*   ... probably followed by whitespace, ...
   *(\d+\.\d+)   ... the "payload" of the match: one or more digits, a literal dot and and one or more digits.

       ;The code:
   <source lang="PHP">
   <?php
 $URLpath = "http://www.pdb.org/pdb/explore/explore.do?structureId=";
 $PDBid = "2imm";
 $source = file_get_contents($URLpath . $PDBid);
    preg_match('/\s*?(\d+\.\d+)/', $source, $resolution); 
     print($resolution[1]);
 ?>




    Self-evaluation

    Further reading, links and resources

    Notes


     


    About ...
     
    Author:

    Boris Steipe <boris.steipe@utoronto.ca>

    Created:

    2017-09-17

    Modified:

    2018-01-31

    Version:

    1.0

    Version history:

    • 1.0 first live version
    • 0.2 Contents outline
    • 0.1 First stub

    CreativeCommonsBy.png This copyrighted material is licensed under a Creative Commons Attribution 4.0 International License. Follow the link to learn more.

    This page is not currently being maintained since it is not part of active learning sections.
    Retrieved from "http://steipe.biochemistry.utoronto.ca/abc/index.php?title=RPR-Data-Import&oldid=11595"