Difference between revisions of "BIN-Data integration"

From "A B C"
Jump to navigation Jump to search
m
m
 
(10 intermediate revisions by the same user not shown)
Line 1: Line 1:
−
<div id="BIO">
+
<div id="ABC">
−
  <div class="b1">
+
<div style="padding:5px; border:1px solid #000000; background-color:#b3dbce; font-size:300%; font-weight:400; color: #000000; width:100%;">
 
Data Integration
 
Data Integration
−
  </div>
+
<div style="padding:5px; margin-top:20px; margin-bottom:10px; background-color:#b3dbce; font-size:30%; font-weight:200; color: #000000; ">
−
 
+
(Integration of biological data; Identifier mapping; Entrez; UniProt; BioMart. ID mapping service and match() function.)
−
  {{Vspace}}
+
</div>
−
 
 
−
<div class="keywords">
 
−
<b>Keywords:</b>&nbsp;
 
−
Integration of biological data; Identifier mapping; Source and result of integration; Domain of validity of annotations (time, tissue, position...);
 
 
</div>
 
</div>
  
−
{{Vspace}}
+
{{Smallvspace}}
  
  
−
__TOC__
+
<div style="padding:5px; border:1px solid #000000; background-color:#b3dbce33; font-size:85%;">
−
 
+
<div style="font-size:118%;">
−
{{Vspace}}
+
<b>Abstract:</b><br />
−
 
 
−
 
 
−
{{DEV}}
 
−
 
 
−
{{Vspace}}
 
−
 
 
−
 
 
−
</div>
 
−
<div id="ABC-unit-framework">
 
−
== Abstract ==
 
 
<section begin=abstract />
 
<section begin=abstract />
−
<!-- included from "../components/BIN-Data_integration.components.wtxt", section: "abstract" -->
+
Data integration is a challenging problem. This unit discusses the issues and how the large databases solve this with NCBI's Entrez system and the EBI's UniProt Knoledeg Base and BioMart System. R coding exercises put some technical issues in practice.
−
...
 
 
<section end=abstract />
 
<section end=abstract />
 +
</div>
 +
<!-- ============================  -->
 +
<hr>
 +
<table>
 +
<tr>
 +
<td style="padding:10px;">
 +
<b>Objectives:</b><br />
 +
This unit will ...
 +
* ... introduce issue of database integration and how the NCBI and the EBI address this;
 +
* ... demonstrate use of Entrez, UniProt and BioMart;
 +
* ... teach ID mapping techniques with R.
 +
</td>
 +
<td style="padding:10px;">
 +
<b>Outcomes:</b><br />
 +
After working through this unit you ...
 +
* ... are familar with the NCBI and EBI query and retrieval systems;
 +
* ... can use BioMart bot online and in R code;
 +
* ... can retrieve ID cross references via scripts and match IDs in large tables with R's <code>match()</code> function.
 +
</td>
 +
</tr>
 +
</table>
 +
<!-- ============================  -->
 +
<hr>
 +
<b>Deliverables:</b><br />
 +
<section begin=deliverables />
 +
<li><b>Time management</b>: Before you begin, estimate how long it will take you to complete this unit. Then, record in your course journal: the number of hours you estimated, the number of hours you worked on the unit, and the amount of time that passed between start and completion of this unit.</li>
 +
<li><b>Journal</b>: Document your progress in your [[FND-Journal|Course Journal]]. Some tasks may ask you to include specific items in your journal. Don't overlook these.</li>
 +
<li><b>Insights</b>: If you find something particularly noteworthy about this unit, make a note in your [[ABC-Insights|'''insights!''' page]].</li>
 +
<section end=deliverables />
 +
<!-- ============================  -->
 +
<hr>
 +
<section begin=prerequisites />
 +
<b>Prerequisites:</b><br />
 +
This unit builds on material covered in the following prerequisite units:<br />
 +
*[[BIN-EBI|BIN-EBI (Databases and services at the EBI)]]
 +
*[[BIN-FUNC-Databases|BIN-FUNC-Databases (Molecular Function Databases)]]
 +
*[[BIN-Miscellaneous_DB|BIN-Miscellaneous_DB (Miscellaneous Databases for Bioinformatics)]]
 +
*[[BIN-NCBI|BIN-NCBI (The NCBI Database and Services)]]
 +
*[[BIN-PDB|BIN-PDB (The RCSB-PDB Structure Database)]]
 +
<section end=prerequisites />
 +
<!-- ============================  -->
 +
</div>
  
−
{{Vspace}}
+
{{Smallvspace}}
  
  
−
== This unit ... ==
 
−
=== Prerequisites ===
 
−
<!-- included from "../components/BIN-Data_integration.components.wtxt", section: "prerequisites" -->
 
−
<!-- included from "ABC-unit_components.wtxt", section: "notes-prerequisites" -->
 
−
You need to complete the following units before beginning this one:
 
−
*[[BIN-EBI]]
 
−
*[[BIN-FUNC-Databases]]
 
−
*[[BIN-Miscellaneous_DB]]
 
−
*[[BIN-NCBI]]
 
−
*[[BIN-PDB]]
 
  
−
{{Vspace}}
+
{{Smallvspace}}
  
  
−
=== Objectives ===
+
__TOC__
−
<!-- included from "../components/BIN-Data_integration.components.wtxt", section: "objectives" -->
 
−
...
 
−
 
 
−
{{Vspace}}
 
−
 
 
−
 
 
−
=== Outcomes ===
 
−
<!-- included from "../components/BIN-Data_integration.components.wtxt", section: "outcomes" -->
 
−
...
 
−
 
 
−
{{Vspace}}
 
−
 
 
−
 
 
−
=== Deliverables ===
 
−
<!-- included from "../components/BIN-Data_integration.components.wtxt", section: "deliverables" -->
 
−
<!-- included from "ABC-unit_components.wtxt", section: "deliverables-time_management" -->
 
−
*<b>Time management</b>: Before you begin, estimate how long it will take you to complete this unit. Then, record in your course journal: the number of hours you estimated, the number of hours you worked on the unit, and the amount of time that passed between start and completion of this unit.
 
−
<!-- included from "ABC-unit_components.wtxt", section: "deliverables-journal" -->
 
−
*<b>Journal</b>: Document your progress in your [[FND-Journal|Course Journal]]. Some tasks may ask you to include specific items in your journal. Don't overlook these.
 
−
<!-- included from "ABC-unit_components.wtxt", section: "deliverables-insights" -->
 
−
*<b>Insights</b>: If you find something particularly noteworthy about this unit, make a note in your [[ABC-Insights|'''insights!''' page]].
 
  
 
{{Vspace}}
 
{{Vspace}}
Line 76: Line 72:
  
 
=== Evaluation ===
 
=== Evaluation ===
−
<!-- included from "../components/BIN-Data_integration.components.wtxt", section: "evaluation" -->
 
−
<!-- included from "ABC-unit_components.wtxt", section: "eval-none" -->
 
 
<b>Evaluation: NA</b><br />
 
<b>Evaluation: NA</b><br />
−
:This unit is not evaluated for course marks.
+
<div style="margin-left: 2rem;">This unit is not evaluated for course marks.</div>
−
 
+
== Contents ==
−
{{Vspace}}
 
−
 
 
  
−
</div>
 
−
<div id="BIO">
 
−
== Contents ==
 
−
<!-- included from "../components/BIN-Data_integration.components.wtxt", section: "contents" -->
 
 
{{Task|1=
 
{{Task|1=
 
*Read the introductory notes on {{ABC-PDF|BIN-Data_integration|concepts and approaches to data integration in bioinformatics}}.
 
*Read the introductory notes on {{ABC-PDF|BIN-Data_integration|concepts and approaches to data integration in bioinformatics}}.
 
}}
 
}}
−
 
  
 
{{Vspace}}
 
{{Vspace}}
  
−
 
+
{{Task|1=
−
== Further reading, links and resources ==
+
*Visit the [http://www.uniprot.org/mapping/ '''UniProt ID mapping service'''], enter <code>NP_010227</code> into the identifier field, select options from '''RefSeq Protein''' to '''UniProtKB''' and click '''Go'''.
−
 
+
* Confirm that this retrieved the right identifier.
−
{{#pmid: 20652519}}
+
* Also note that you could have searched with a list of IDs, and downloaded the results, e.g. for further processing in R.
−
 
+
}}
  
 
{{Vspace}}
 
{{Vspace}}
  
−
 
+
{{ABC-unit|BIN-Data_integration.R}}
−
== Notes ==
 
−
<!-- included from "../components/BIN-Data_integration.components.wtxt", section: "notes" -->
 
−
<!-- included from "ABC-unit_components.wtxt", section: "notes" -->
 
−
<references />
 
  
 
{{Vspace}}
 
{{Vspace}}
  
 +
{{Task|1=
 +
The <code>biomartr</code> bioconductor package is a second-generation R interface to BioMart that extends the <code>biomaRt</code> package. It has a good quick start introduction to "Functional Annotation".
 +
* Navigate to https://cran.r-project.org/web/packages/biomartr/vignettes/Functional_Annotation.html
 +
* Work through the tutorial.
  
−
</div>
+
}}
−
<div id="ABC-unit-framework">
 
−
== Self-evaluation ==
 
−
<!-- included from "../components/BIN-Data_integration.components.wtxt", section: "self-evaluation" -->
 
−
<!--
 
−
=== Question 1===
 
  
−
Question ...
 
  
−
<div class="toccolours mw-collapsible mw-collapsed" style="width:800px">
 
−
Answer ...
 
−
<div class="mw-collapsible-content">
 
−
Answer ...
 
  
−
</div>
+
== Further reading, links and resources ==
−
  </div>
 
  
−
  {{Vspace}}
+
<div class="reference-box">[http://www.uniprot.org/help/ncbi_mappings '''UniProt - NCBI ID mapping'''] -  detailed information on how it works.</div>
  
−
-->
+
{{#pmid: 20652519}}
  
−
{{Vspace}}
+
== Notes ==
−
 
+
<references />
−
 
 
−
 
 
−
{{Vspace}}
 
−
 
 
−
 
 
−
<!-- included from "ABC-unit_components.wtxt", section: "ABC-unit_ask" -->
 
−
 
 
−
----
 
  
 
{{Vspace}}
 
{{Vspace}}
  
−
<b>If in doubt, ask!</b> If anything about this learning unit is not clear to you, do not proceed blindly but ask for clarification. Post your question on the course mailing list: others are likely to have similar problems. Or send an email to your instructor.
 
−
 
−
----
 
−
 
−
{{Vspace}}
 
  
 
<div class="about">
 
<div class="about">
Line 160: Line 123:
 
:2017-08-05
 
:2017-08-05
 
<b>Modified:</b><br />
 
<b>Modified:</b><br />
−
:2017-08-05
+
:2020-09-24
 
<b>Version:</b><br />
 
<b>Version:</b><br />
−
:0.1
+
:1.1
 
<b>Version history:</b><br />
 
<b>Version history:</b><br />
 +
*1.1 2020 Maintenance
 +
*1.0 First live version.
 
*0.1 First stub
 
*0.1 First stub
 
</div>
 
</div>
−
[[Category:ABC-units]]
 
−
<!-- included from "ABC-unit_components.wtxt", section: "ABC-unit_footer" -->
 
  
 
{{CC-BY}}
 
{{CC-BY}}
  
 +
[[Category:ABC-units]]
 +
{{UNIT}}
 +
{{LIVE}}
 
</div>
 
</div>
 
<!-- [END] -->
 
<!-- [END] -->

Latest revision as of 16:32, 24 September 2020

Data Integration

(Integration of biological data; Identifier mapping; Entrez; UniProt; BioMart. ID mapping service and match() function.)


 


Abstract:

Data integration is a challenging problem. This unit discusses the issues and how the large databases solve this with NCBI's Entrez system and the EBI's UniProt Knoledeg Base and BioMart System. R coding exercises put some technical issues in practice.

Objectives:
This unit will ...

  • ... introduce issue of database integration and how the NCBI and the EBI address this;
  • ... demonstrate use of Entrez, UniProt and BioMart;
  • ... teach ID mapping techniques with R.

Outcomes:
After working through this unit you ...

  • ... are familar with the NCBI and EBI query and retrieval systems;
  • ... can use BioMart bot online and in R code;
  • ... can retrieve ID cross references via scripts and match IDs in large tables with R's match() function.

Deliverables:

  • Time management: Before you begin, estimate how long it will take you to complete this unit. Then, record in your course journal: the number of hours you estimated, the number of hours you worked on the unit, and the amount of time that passed between start and completion of this unit.
  • Journal: Document your progress in your Course Journal. Some tasks may ask you to include specific items in your journal. Don't overlook these.
  • Insights: If you find something particularly noteworthy about this unit, make a note in your insights! page.

    • Prerequisites:
    This unit builds on material covered in the following prerequisite units:


     



     



     


    Evaluation

    Evaluation: NA

    This unit is not evaluated for course marks.

    Contents

    Task:


     

    Task:

    • Visit the UniProt ID mapping service, enter NP_010227 into the identifier field, select options from RefSeq Protein to UniProtKB and click Go.
    • Confirm that this retrieved the right identifier.
    • Also note that you could have searched with a list of IDs, and downloaded the results, e.g. for further processing in R.


     

    Task:

     
    • Open RStudio and load the ABC-units R project. If you have loaded it before, choose File → Recent projects → ABC-Units. If you have not loaded it before, follow the instructions in the RPR-Introduction unit.
    • Choose Tools → Version Control → Pull Branches to fetch the most recent version of the project from its GitHub repository with all changes and bug fixes included.
    • Type init() if requested.
    • Open the file BIN-Data_integration.R and follow the instructions.


     

    Note: take care that you understand all of the code in the script. Evaluation in this course is cumulative and you may be asked to explain any part of code.


     


     

    Task:
    The biomartr bioconductor package is a second-generation R interface to BioMart that extends the biomaRt package. It has a good quick start introduction to "Functional Annotation".


    Further reading, links and resources

    UniProt - NCBI ID mapping - detailed information on how it works.
    Xie & Ahn (2010) Statistical methods for integrating multiple types of high-throughput data. Methods Mol Biol 620:511-29. (pmid: 20652519)

    [ PubMed ] [ DOI ] Large-scale sequencing, copy number, mRNA, and protein data have given great promise to the biomedical research, while posing great challenges to data management and data analysis. Integrating different types of high-throughput data from diverse sources can increase the statistical power of data analysis and provide deeper biological understanding. This chapter uses two biomedical research examples to illustrate why there is an urgent need to develop reliable and robust methods for integrating the heterogeneous data. We then introduce and review some recently developed statistical methods for integrative analysis for both statistical inference and classification purposes. Finally, we present some useful public access databases and program code to facilitate the integrative analysis in practice.

    Notes


     


    About ...
     
    Author:

    Boris Steipe <boris.steipe@utoronto.ca>

    Created:

    2017-08-05

    Modified:

    2020-09-24

    Version:

    1.1

    Version history:

    • 1.1 2020 Maintenance
    • 1.0 First live version.
    • 0.1 First stub

    CreativeCommonsBy.png This copyrighted material is licensed under a Creative Commons Attribution 4.0 International License. Follow the link to learn more.

    Retrieved from "http://steipe.biochemistry.utoronto.ca/abc/index.php?title=BIN-Data_integration&oldid=11644"