Conference proceeding
Link contexts in classifier-guided topical crawlers
IEEE transactions on knowledge and data engineering, Vol.18(1), pp.107-122
01/2006
DOI: 10.1109/TKDE.2006.12
Abstract
Context of a hyperlink or link context is defined as the terms that appear in the text around a hyperlink within a Web page. Link contexts have been applied to a variety of Web information retrieval and categorization tasks. Topical or focused Web crawlers have a special reliance on link contexts. These crawlers automatically navigate the hyperlinked structure of the Web while using link contexts to predict the benefit of following the corresponding hyperlinks with respect to some initiating topic or theme. Using topical crawlers that are guided by a support vector machine, we investigate the effects of various definitions of link contexts on the crawling performance. We find that a crawler that exploits words both in the immediate vicinity of a hyperlink as well as the entire parent page performs significantly better than a crawler that depends on just one of those cues. Also, we find that a crawler that uses the tag tree hierarchy within Web pages provides effective coverage. We analyze our results along various dimensions such as link context quality, topic difficulty, length of crawl, training data, and topic domain. The study was done using multiple crawls over 100 topics covering millions of pages allowing us to derive statistically strong results.
Details
- Title: Subtitle
- Link contexts in classifier-guided topical crawlers
- Creators
- Gautam Pant - Sch. of Accounting & Inf. Syst., Utah Univ., Salt Lake City, UT, USAPadmini Srinivasan - University of Iowa, Computer Science
- Resource Type
- Conference proceeding
- Publication Details
- IEEE transactions on knowledge and data engineering, Vol.18(1), pp.107-122
- Publisher
- IEEE
- DOI
- 10.1109/TKDE.2006.12
- ISSN
- 1041-4347
- eISSN
- 1558-2191
- Language
- English
- Date published
- 01/2006
- Academic Unit
- Business Analytics; Computer Science; Nursing
- Record Identifier
- 9984003795902771
Metrics
18 Record Views