NAME
Bio::DB::EUtilities - Webagent which interacts with and retrieves data from NCBI's eUtils.
VERSION
version 1.77
SYNOPSIS
my $eutil = Bio::DB::EUtilities->new(-eutil => 'esearch',-term => 'BRCA1',-db => 'pubmed',-retmax => 10000,-email => '[email protected]'); # please use your real email# eutil => any of esearch, esummary, elink@ids = $eutil->get_ids(); # returns array or array ref of IDs# eutil => any of egquery, espell$term = $eutil->get_term(); # returns array or array ref of IDs# eutil => any of elink, einfo$db = $eutil->get_database(); # returns database# Query-related methods (esearch, egquery, espell data)# eutil data centered on use of search termsmy $ct = $eutil->get_count; # uses optional database for egquery countmy $translation = $eutil->get_count;my $corrected = $eutil->get_corrected_query; # espellwhile (my $gquery = $eutil->next_GlobalQuery) {# iterates through egquery data}# Info-related methods (einfo data)# database-related informationmy $desc = $eutil->get_description;my $update = $eutil->get_last_update;my $nm = $eutil->get_menu_name;my $ct = $eutil->get_record_count;while (my $field = $eutil->next_FieldInfo) {# ...}while (my $field = $eutil->next_LinkInfo) {# ...}# History methods (epost data, some data returned from elink)# data which enables one to retrieve and query against user-stored# information on the NCBI serverwhile (my $cookie = $eutil->next_History) {# ...}my @hists = $eutil->get_Histories;# Bio::Tools::EUtilities::Summary (esummary data)# information on a specific database record# retrieve nested docsum datawhile (my $docsum = $eutil->next_DocSum) {print "ID:",$docsum->get_ids,"\n";while (my $item = $docsum->next_Item) {# do stuff here...while (my $listitem = $docsum->next_ListItem) {# do stuff here...while (my $listitem = $docsum->next_Structure) {# do stuff here...}}}}# retrieve flattened item list per DocSumwhile (my $docsum = $eutil->next_DocSum) {my @items = $docsum->get_all_DocSum_Items;}
DESCRIPTION
This is a general webagent which posts and retrieves data to NCBI's eUtils service using their CGI interface. A separate project, currently present in BioPerl-Run, utilizes the SOAP-based interface for eUtils.
A full explanation of the eUtils interfaces are not possible within this short module; please see the BioPerl wiki HOWTO for more details.
METHODS
TODO
- •
-
Finish documentation
HOWTOs (both standard and Cookbook).
- •
-
Cookbook tests
Set up dev-only tests for Cookbook examples to make sure they are consistently updated.
- •
-
API
Mark Jensen has written up the SOAP-based eUtil modules, maybe we should coalesce around a consistent API between the two (they are close).
- •
-
Carryover of parameters
Maybe add a default but configurable list of parameters that can be carried over between calls.
- •
-
Make setting certain parameters consistent
Setting history is a bit inconsistent, so maybe use a common alias for this?
- •
-
Moosify?
Not necessary, but it might make things easier and more maintainable in the long run.
Bio::DB::GenericWebAgent methods
GenericWebDBI methods
parameter_base
Title : parameter_baseUsage : $dbi->parameter_base($pobj);Function: Get/Set Bio::ParameterBaseI.Returns : Bio::ParameterBaseI objectArgs : Bio::ParameterBaseI object
ua
Title : uaUsage : $dbi->ua;Function: Get/Set LWP::UserAgent.Returns : LWP::UserAgentArgs : LWP::UserAgent
get_Response
Title : get_ResponseUsage : $agent->get_Response;Function: Get the HTTP::Response object by passing it an HTTP::Request (generated fromBio::ParameterBaseI implementation).Returns : HTTP::Response object or data if callback is usedArgs : (optional)-cache_response - flag to cache HTTP::Response object;Default is 1 (TRUE, caching ON)These are passed on to LWP::UserAgent::request() if stipulated-cb - use a LWP::UserAgent-compliant callback-file - dumps the response to a file (handy for large responses)Note: can't use file and callback at the same time-read_size_hint - bytes of content to read in at a time to pass to callbackNote : Caching and parameter checking are set
delay
Title : delayUsage : $secs = $self->delay([$secs])Function: get/set number of seconds to delay between fetchesReturns : number of seconds to delayArgs : new value
NOTE: the default is to use the value specified by delay_policy(). This can be overridden by calling this method.
LWP::UserAgent related methods
proxy
Title : proxyUsage : $httpproxy = $db->proxy('http') or$db->proxy(['http','ftp'], 'http://myproxy' )Function: Get/Set a proxy for use of proxyReturns : a string indicating the proxyArgs : $protocol : an array ref of the protocol(s) to set/get$proxyurl : url of the proxy to use for the specified protocol$username : username (if proxy requires authentication)$password : password (if proxy requires authentication)
authentication
Title : authenticationUsage : $db->authentication($user,$pass)Function: Get/Set authentication credentialsReturns : Array of user/passArgs : Array or user/pass
delay_policy
Title : delay_policyUsage : $secs = $self->delay_policyFunction: return number of seconds to delay between calls to remote dbReturns : number of seconds to delayArgs : noneNOTE: NCBI requests a delay of 3 requests per second. This method implementsthat policy. This will likely change to check time of day or number ofrequests for lengthening delays.
get_Parser
Title : get_ParserUsage : $agent->get_Parser;Function: Retrieve the parser used for last agent requestReturns : The Bio::Tools::EUtilities parser used to parse the HTTP::ResponsecontentArgs : NoneNote : Abstract method; defined by implementation
Bio::Tools::EUtilities::EUtilParameters-delegating methods
This is only a subset of parameters available from Bio::Tools::EUtilities::EUtilParameters (the ones deemed absolutely necessary). All others are available by calling 'parameter_base->method' when needed.
set_parameters
Title : set_parametersUsage : $pobj->set_parameters(@params);Function: sets the NCBI parameters listed in the hash or arrayReturns : NoneArgs : [optional] hash or array of parameter/values.Note : This sets any parameter (i.e. doesn't screen them). In addition toregular eutil-specific parameters, you can set the following:-eutil - the eUtil to be used (default 'efetch')-history - pass a HistoryI-implementing object, whichsets the WebEnv, query_key, and possibly db and linkname(the latter two only for LinkSets)-correspondence - Boolean flag, set to TRUE or FALSE; indicates howIDs are to be added together for elink request whereID correspondence might be needed(default 0)
reset_parameters
Title : reset_parametersUsage : resets valuesFunction: resets parameters to either undef or value in passed hashReturns : noneArgs : [optional] hash of parameter-value pairsNote : this also resets eutil(), correspondence(), and the history and requestcache
available_parameters
Title : available_parametersUsage : @params = $pobj->available_parameters()Function: Returns a list of the available parametersReturns : Array of available parameters (no values)Args : [optional] A string; either eutil name (for returning eutil-specificparameters) or 'history' (for those parameters allowed when retrievingdata stored on the remote server using a 'History' object).
get_parameters
Title : get_parametersUsage : @params = $pobj->get_parameters;%params = $pobj->get_parameters;Function: Returns list of key/value pairs, parameter => valueReturns : Flattened list of key-value pairs. All key-value pairs returned,though subsets can be returned based on the '-type' parameter.Data passed as an array ref are returned based on whether the'-join_id' flag is set (default is the same array ref).Args : -type : the eutil name or 'history', for returning a subset ofparameters (Default: returns all)-join_ids : Boolean; join IDs based on correspondence (Default: no join)
get_parameter_values
Title : get_parameter_valuesUsage : @vals = $factory->get_parameter_value('id'); # always get arrayFunction: Returns the specific parameter values.Returns : For consistency returns a list of values for this parameter. If onlyone is expected, use:($val) = $factory->get_parameter_value('id');Args : parameter expected
Bio::Tools::EUtilities-delegating methods
Bio::Tools::EUtilities::EUtilDataI methods
eutil
Title : eutilUsage : $eutil->$foo->eutilFunction : Get/Set eutilReturns : stringArgs : string (eutil)Throws : on invalid eutil
datatype
Title : datatypeUsage : $type = $foo->datatype;Function : Get/Set data object typeReturns : stringArgs : string
to_string
Title : to_stringUsage : $foo->to_string()Function : converts current object to stringReturns : noneArgs : (optional) simple data for text formattingNote : Implemented in plugins
print_all
Title : print_allUsage : $info->print_all();$info->print_all(-fh => $fh, -cb => $coderef);Function : prints (dumps) all data in parser. Unless a coderef is supplied,this just dumps the parser-specific to_string method to either afile/fh or STDOUTReturns : noneArgs : [optional]-file : file to print to-fh : filehandle to print to (cannot be used concurrently with file)-cb : coderef to use in place of default print method. This is passedin a LinkSet object-wrap : number of columns to wrap default text output to (def = 80)Notes : only applicable for einfo. If -file or -fh are not defined,prints to STDOUT
Methods useful for multiple eutils
get_ids
Title : get_idsUsage : my @ids = $parser->get_idsFunction : returns array of requested IDs (see Notes for more specifics)Returns : arrayArgs : [conditional] not required except when running elink queries againstmultiple databases. In case of the latter, the database name isoptional but recommended when retrieving IDs as the ID list willbe globbed together. In such cases, if a db name isn't provided awarning is issued as a reminder.Notes : esearch : returned ID listelink : returned ID list (see Args above for caveats)all others : from parameter_base->id or undef
get_database
Title : get_databaseUsage : my $db = $info->get_database;Function : returns single database name (eutil-compatible). This is the querieddatabase. For most eutils this is straightforward. For elinks(which have 'db' and 'dbfrom') this is dbto, for egquery, it is the firstdb in the list (you probably want get_databases instead)Returns : stringArgs : noneNotes : egquery : first db in the query (you probably want get_databases)einfo : the queried databaseespell : the queried databaseelink : from parameter_base->dbfrom or undefall others : from parameter_base->db or undef
get_db (alias for get_database)
get_databases
Title : get_databasesUsage : my @dbs = $parser->get_databasesFunction : returns list of databasesReturns : array of stringsArgs : noneNotes : This is guaranteed to return a list of databases. For a singledatabase use the convenience method get_db/get_databaseegquery : list of all databases in the queryeinfo : the queried databaseespell : the queried databaseall others : from parameter_base->db or undef
get_dbs (alias for get_databases)
next_History
Title : next_HistoryUsage : while (my $hist=$parser->next_History) {...}Function : returns next HistoryI (if present).Returns : Bio::Tools::EUtilities::HistoryI (Cookie or LinkSet)Args : noneNote : esearch, epost, and elink are all capable of returning data whichindicates search results (in the form of UIDs) is stored on theremote server. Access to this data is wrapped up in simple interface(HistoryI), which is implemented in two classes:Bio::DB::EUtilities::History (the simplest) andBio::DB::EUtilities::LinkSet. In general, calls to epost and esearchwill only return a single HistoryI object (formerly known as aCookie), but calls to elink can generate many depending on thenumber of IDs, the correspondence, etc. Hence this iterator, whichallows one to retrieve said data one piece at a time.
next_cookie (alias for next_History)
get_Histories
Title : get_HistoriesUsage : my @hists = $parser->get_HistoriesFunction : returns list of HistoryI objects.Returns : list of Bio::Tools::EUtilities::HistoryI (Cookie or LinkSet)Args : none
Query-related methods
get_count
Title : get_countUsage : my $ct = $parser->get_countFunction : returns the count (hits for a search)Returns : integerArgs : [CONDITIONAL] string with database name - used to retrievecount from specific database when using egqueryNotes : egquery : count for specified database (specified above)esearch : count for last searchall others : undef
get_term
Title : get_termUsage : $st = $qd->get_term;Function : retrieve the term for the global searchReturns : stringArgs : noneNotes : egquery : search termespell : search termesearch : from parameter_base->term or undefall others : undef
get_translation_from
Title : get_translation_fromUsage : $string = $qd->get_translation_from();Function: portion of the original query replaced with translated_to()Returns : stringArgs : noneNote : only applicable for esearch
get_translation_to
Title : get_translation_toUsage : $string = $qd->get_translation_to();Function: replaced string used in place of the original query term in translation_from()Returns : stringArgs : noneNote : only applicable for esearch
get_retstart
Title : get_retstartUsage : $start = $qd->get_retstart();Function : retstart setting for the query (either set or NCBI default)Returns : IntegerArgs : noneNotes : esearch : retstartesummary : retstartall others : from parameter_base->retstart or undef
get_retmax
Title : get_retmaxUsage : $max = $qd->get_retmax();Function : retmax setting for the query (either set or NCBI default)Returns : IntegerArgs : noneNotes : esearch : retmaxesummary : retmaxall others : from parameter_base->retmax or undef
get_query_translation
Title : get_query_translationUsage : $string = $qd->get_query_translation();Function: returns the translated query used for the search (if any)Returns : stringArgs : noneNotes : only applicable for esearch. This is the actual term used foresearch.
get_corrected_query
Title : get_corrected_queryUsage : my $cor = $eutil->get_corrected_query;Function : retrieves the corrected query when using espellReturns : stringArgs : noneNotes : only applicable for espell.
get_replaced_terms
Title : get_replaced_termsUsage : my $term = $eutil->get_replaced_termsFunction : returns array of strings replaced in the queryReturns : stringArgs : noneNotes : only applicable for espell
next_GlobalQuery
Title : next_GlobalQueryUsage : while (my $query = $eutil->next_GlobalQuery) {...}Function : iterates through the queries returned from an egquery searchReturns : GlobalQuery objectArgs : noneNotes : only applicable for egquery
get_GlobalQueries
Title : get_GlobalQueriesUsage : @queries = $eutil->get_GlobalQueriesFunction : returns list of GlobalQuery objectsReturns : array of GlobalQuery objectsArgs : noneNotes : only applicable for egquery
print_GlobalQueries
Title : print_GlobalQueriesUsage : $docsum->print_GlobalQueries();$docsum->print_GlobalQueries(-fh => $fh, -cb => $coderef);Function : prints item data for all global queries. The default printingmethod is each item per DocSum is printed with relevant values ifpresent in a simple table using Text::Wrap.Returns : noneArgs : [optional]-file : file to print to-fh : filehandle to print to (cannot be used concurrently with file)-cb : coderef to use in place of default print method. This is passedin a GlobalQuery object;-wrap : number of columns to wrap default text output to (def = 80)Notes : only applicable for esummary. If -file or -fh are not defined,prints to STDOUT
Summary-related methods
next_DocSum
Title : next_DocSumUsage : while (my $ds = $esum->next_DocSum) {...}Function : iterate through DocSum instancesReturns : single Bio::Tools::EUtilities::Summary::DocSumArgs : none yetNotes : only applicable for esummary
get_DocSums
Title : get_DocSumsUsage : my @docsums = $esum->get_DocSumsFunction : retrieve a list of DocSum instancesReturns : array of Bio::Tools::EUtilities::Summary::DocSumArgs : noneNotes : only applicable for esummary
print_DocSums
Title : print_DocSumsUsage : $docsum->print_DocSums();$docsum->print_DocSums(-fh => $fh, -cb => $coderef);Function : prints item data for all docsums. The default printing method iseach item per DocSum is printed with relevant values if presentin a simple table using Text::Wrap.Returns : noneArgs : [optional]-file : file to print to-fh : filehandle to print to (cannot be used concurrently with file)-cb : coderef to use in place of default print method. This is passedin a DocSum object;-wrap : number of columns to wrap default text output to (def = 80)Notes : only applicable for esummary. If -file or -fh are not defined,prints to STDOUT
Info-related methods
get_available_databases
Title : get_available_databasesUsage : my @dbs = $info->get_available_databasesFunction : returns list of available eutil-compatible database namesReturns : Array of stringsArgs : noneNotes : only applicable for einfo.
get_record_count
Title : get_record_countUsage : my $ct = $eutil->get_record_count;Function : returns database record countReturns : integerArgs : noneNotes : only applicable for einfo.
get_last_update
Title : get_last_updateUsage : my $time = $info->get_last_update;Function : returns string containing time/date stamp for last database updateReturns : integerArgs : noneNotes : only applicable for einfo.
get_menu_name
Title : get_menu_nameUsage : my $nm = $info->get_menu_name;Function : returns string of database menu nameReturns : stringArgs : noneNotes : only applicable for einfo.
get_description
Title : get_descriptionUsage : my $desc = $info->get_description;Function : returns database descriptionReturns : stringArgs : noneNotes : only applicable for einfo.
next_FieldInfo
Title : next_FieldInfoUsage : while (my $field = $info->next_FieldInfo) {...}Function : iterate through FieldInfo objectsReturns : Field objectArgs : noneNotes : only applicable for einfo. Uses callback() for filtering if definedfor 'fields'
get_FieldInfo
Title : get_FieldInfoUsage : my @fields = $info->get_FieldInfo;Function : returns list of FieldInfo objectsReturns : array (FieldInfo objects)Args : noneNotes : only applicable for einfo.
next_LinkInfo
Title : next_LinkInfoUsage : while (my $link = $info->next_LinkInfo) {...}Function : iterate through LinkInfo objectsReturns : LinkInfo objectArgs : noneNotes : only applicable for einfo. Uses callback() for filtering if definedfor 'linkinfo'
get_LinkInfo
Title : get_LinkInfoUsage : my @links = $info->get_LinkInfo;Function : returns list of LinkInfo objectsReturns : array (LinkInfo objects)Args : noneNotes : only applicable for einfo.
print_FieldInfo
Title : print_FieldInfoUsage : $info->print_FieldInfo();$info->print_FieldInfo(-fh => $fh, -cb => $coderef);Function : prints field data for each FieldInfo object. The default methodprints data from each FieldInfo in a simple table using Text::Wrap.Returns : noneArgs : [optional]-file : file to print to-fh : filehandle to print to (cannot be used concurrently with file)-cb : coderef to use in place of default print method.-wrap : number of columns to wrap default text output to (def = 80)Note : if -file or -fh are not defined, prints to STDOUT
print_LinkInfo
Title : print_LinkInfoUsage : $info->print_LinkInfo();$info->print_LinkInfo(-fh => $fh, -cb => $coderef);Function : prints link data for each LinkInfo object. The default is generatedvia LinkInfo::to_stringReturns : noneArgs : [optional]-file : file to print to-fh : filehandle to print to (cannot be used concurrently with file)-cb : coderef to use in place of default print method. This is passedin a LinkInfo object;-wrap : number of columns to wrap default text output to (def = 80)Notes : only applicable for einfo. If -file or -fh are not defined,prints to STDOUT
Bio::Tools::EUtilities::Link-related methods
next_LinkSet
Title : next_LinkSetUsage : while (my $ls = $eutil->next_LinkSet {...}Function : iterate through LinkSet objectsReturns : LinkSet objectArgs : noneNotes : only applicable for elink. Uses callback() for filtering if definedfor 'linksets'
get_LinkSets
Title : get_LinkSetsUsage : my @links = $info->get_LinkSets;Function : returns list of LinkSets objectsReturns : array (LinkSet objects)Args : noneNotes : only applicable for elink.
print_LinkSets
Title : print_LinkSetsUsage : $info->print_LinkSets();$info->print_LinkSets(-fh => $fh, -cb => $coderef);Function : prints link data for each LinkSet object. The default is generatedvia LinkSet::to_stringReturns : noneArgs : [optional]-file : file to print to-fh : filehandle to print to (cannot be used concurrently with file)-cb : coderef to use in place of default print method. This is passedin a LinkSet object-wrap : number of columns to wrap default text output to (def = 80)Notes : only applicable for einfo. If -file or -fh are not defined,prints to STDOUT
get_linked_databases
Title : get_linked_databasesUsage : my @dbs = $eutil->get_linked_databasesFunction : returns list of databases linked to in linksetsReturns : array of databasesArgs : noneNotes : only applicable for elink.
Iterator- and callback-related methods
rewind
Title : rewindUsage : $esum->rewind()$esum->rewind('recursive')Function : retrieve a list of DocSum instancesReturns : array of Bio::Tools::EUtilities::Summary::DocSumArgs : [optional] Scalar; string ('all') to reset all iterators, or stringdescribing the specific main object iterator to reset. The followingare recognized (case-insensitive):'all' - rewind all objects and also recursively resets nested objectiterators (such as LinkSets and DocSums).'globalqueries''fieldinfo' or 'fieldinfos''linkinfo' or 'linkinfos''linksets''docsums'
generate_iterator
Title : generate_iteratorUsage : my $coderef = $esum->generate_iterator('linkinfo')Function : generates an iterator (code reference) which iterates throughthe relevant object indicated by the argsReturns : code referenceArgs : [REQUIRED] Scalar; string describing the specific object to iterate.The following are currently recognized (case-insensitive):'globalqueries''fieldinfo' or 'fieldinfos''linkinfo' or 'linkinfos''linksets''docsums'A second argument can also be passed to generate a 'lazy' iterator,which loops through and returns objects as they are created (insteadof creating all data instances up front, then iterating through,which is the default). Use of these iterators precludes use ofrewind() for the time being as we can't guarantee you can rewind(),as this depends on whether the data source is seek()able and thus'rewindable'. We will add rewind() support at a later time whichwill work for 'seekable' data.A callback specified using callback() will be used to filter objectsfor any generated iterator. This behaviour is implemented for bothnormal and lazy iterator types and is the default. If you don't wantthis, make sure to reset any previously set callbacks viareset_callback() (which just deletes the code ref).TODO : generate seekable iterators ala HOP for seekable fh data
callback
Title : callbackUsage : $parser->callback(sub {$_[0]->get_database eq 'protein'});Function : Get/set callback code ref used to filter returned data objectsReturns : code ref if previously setArgs : single argument:code ref - evaluates a passed object and returns true or false value(used in iterators)'reset' - string, resets the iterator.returns upon any other args
FEEDBACK
Mailing lists
User feedback is an integral part of the evolution of this and other Bioperl modules. Send your comments and suggestions preferably to the Bioperl mailing list. Your participation is much appreciated.
[email protected] - General discussionhttps://bioperl.org/Support.html - About the mailing lists
Support
Please direct usage questions or support issues to the mailing list: [email protected] rather than to the module maintainer directly. Many experienced and reponsive experts will be able look at the problem and quickly address it. Please include a thorough description of the problem with code and data examples if at all possible.
Reporting bugs
Report bugs to the Bioperl bug tracking system to help us keep track of the bugs and their resolution. Bug reports can be submitted via the web:
https://github.com/bioperl/bio-eutilities/issues
AUTHOR
Chris Fields <[email protected]>
COPYRIGHT
This software is copyright (c) 2006-2013 by Chris Fields.
This software is available under the same terms as the perl 5 programming language system itself.