WWW::Search::CraigsList(3pm)

NAME

WWW::Search::HotJobs - class for searching HotJobs

SYNOPSIS

    require WWW::Search;

    $search = new WWW::Search('HotJobs');

DESCRIPTION

This class is an HotJobs specialization of WWW::Search. It handles making and interpreting HotJobs searches http://www.HotJobs.com.

This class exports no public interface; all interaction should be done through WWW::Search objects.

OPTIONS

None at this time (2001.04.25)

search_url=URL: Specifies who to query with the HotJobs protocol. The default is at "http://www.HotJobs.com/cgi-bin/job-search".

search_debug, search_parse_debug, search_ref Specified at the WWW::Search manpage.

Internet/Web Engineering Category options:
<null> - ALL JOBS
art - web design jobs
bus - business jobs
mar - marketing jobs
eng - internet engineering jobs
etc - etcetera jobs
wri - writing jobs
sof - software jobs
acc - finance jobs
ofc - office jobs
med - media jobs
hea - health science jobs
ret - retail jobs
npo - nonprofit jobs
lgl - legal jobs
egr - engineering jobs
sls - sales jobs
sad - sys admin jobs
tel - network jobs
tfr - tv video radio jobs
hum - human resource jobs
tch - tech support jobs
edu - education jobs
trd - skilled trades jobs

Checkboxes - additive to search(?)

addOne value=telecommuting - telecommute addTwo value=contract - contract addThree value=internship - internships addFour value=part-time - part-time addFive value=non-profit - non-profit

HOW DOES IT WORK?

"native_setup_search" is called before we do anything. It initializes our private variables (which all begin with underscores) and sets up a URL to the first results page in "{_next_url}".

"native_retrieve_some" is called (from "WWW::Search::retrieve_some") whenever more hits are needed. It calls the LWP library to fetch the page specified by "{_next_url}". It parses this page, appending any search hits it finds to "{cache}". If it finds a ``next'' button in the text, it sets "{_next_url}" to point to the page for the next set of results, otherwise it sets it to undef to indicate we're done.

AUTHOR and CURRENT VERSION

"WWW::Search::HotJobs" is written and maintained by Glenn Wood, <[email protected]>.

The best place to obtain "WWW::Search::HotJobs" is from Martin Thurn's WWW::Search releases on CPAN. Because HotJobs sometimes changes its format in between his releases, sometimes more up-to-date versions can be found at http://alumni.caltech.edu/~glenwood/SOFTWARE/WWW_SEARCH_HotJobs/index.html.

COPYRIGHT

This program is free software; you can redistribute it and/or modify it under the same terms as Perl itself.

--------------------------

Redistribution and use in source and binary forms are permitted provided that the above copyright notice and this paragraph are duplicated in all such forms and that any documentation, advertising materials, and other materials related to such distribution and use acknowledge that the software was developed by the University of Southern California, Information Sciences Institute. The name of the University may not be used to endorse or promote products derived from this software without specific prior written permission.

THIS SOFTWARE IS PROVIDED "AS IS" AND WITHOUT ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE.

XML Scaffolding

Look at the idea from the perspective of the XML "scaffold" I'm suggesting for parsing the response HTML.

(This is XML, but looks superficially like HTML)

This scaffold describes the relevant skeleton of an HTML document; there's HTML and BODY elements, of course. Then the <TABLE> entry tells our parser to skip to the TABLE in the HTML named "name", or skip "number" TABLE entries (default=0, to pick up first TABLE element.) Then the TABLE is described. The first <TR> is described as a "header" row. The parser throws that one away. The second <TR> is a "detail" row (the "*" means multiple detail rows, of course). The parser picks up each <TD> element, extracts it's content, and places that in the hash entry corresponding to its BIND= attribute. Thus, the first TD goes into $result->_elem('title') (I needed to learn to use LWP::MemberMixin. Thanks, another lesson learned!) The second TD goes into $result->_elem('description'), etc. (Of course, some of these are _elem_array, but these details will be resolved later). The PARSE= in the url TD suggests a way for our parser to do special handling of a data element. The generic scaffold parser would take this XML and convert it to a hash/array to be processed at run time; we wouldn't actually use XML at run time. A backend author would use that hash/array in his native_setup_search() code, calling the "scaffolder" scanner with that hash as a parameter.

As I said, this works great if the response is TABLE structured, but I haven't seen any responses that aren't that way already.

This converts to an array tree that looks like this:

    my $scaffold = [ 'HTML', 

                     [ [ 'BODY', 

                       [ [ 'TABLE', 'name' ,                  # or 'name' = undef; multiple <TABLE number=n> mean n 'TABLE's here ,

                         [ [ 'NEXT', 1, 'NEXT &gt;' ] ,       # meaning how to find the NEXT button.

                           [ 'TR', 1 ] ,                      # meaning "header".

                           [ 'TR', 2 ,                        # meaning "detail*"

                             [ [ 'TD', 1, 'title' ] ,         # meaning clear text binding to _elem('title').

                               [ 'TD', 1, 'description' ] ,

                               [ 'TD', 1, 'location' ] ,

                               [ 'TD', 2, 'url' ]             # meaning anchor parsed text binding to _elem('title').

                             ]

                         ] ]

                       ] ]

                     ] ]

                  ];