File Download
  Links for fulltext
     (May Require Subscription)
Supplementary

Article: Automated functional testing of online search services

TitleAutomated functional testing of online search services
Authors
Zhou, ZQZhang, SHagenbuchner, MTse, THKuo, FCChen, TY
KeywordsSoftware testing
Metamorphic testing
Verification and validation
Internet search
Issue Date2012
PublisherJohn Wiley & Sons Ltd. The Journal's web site is located at http://www3.interscience.wiley.com/cgi-bin/jhome/13635
Citation
Software Testing, Verification and Reliability, 2012, v. 22 n. 4, p. 221-243 How to Cite?
DOI: http://dx.doi.org/10.1002/stvr.437
AbstractSearch services are the main interface through which people discover information on the Internet. A fundamental challenge in testing search services is the lack of oracles. The sheer volume of data on the Internet prohibits testers from verifying the results. Furthermore, it is difficult to objectively assess the ranking quality because different assessors can have very different opinions on the relevance of a Web page to a query. This paper presents a novel method for automatically testing search services without the need of a human oracle. The experimental findings reveal that some commonly used search engines, including Google, Yahoo!, and Live Search, are not as reliable as what most users would expect. For example, they may fail to find pages that exist in their own repositories, or rank pages in a way that is logically inconsistent. Suggestions are made for search service providers to improve their service quality. Copyright © 2010 John Wiley & Sons, Ltd. A novel method for automatically testing search services without the need of a human oracle is presented. The experimental findings reveal that some commonly used search engines, including Google, Yahoo!, and Live Search, are not as reliable as what most users would expect. For example, they may fail to find pages that exist in their own repositories, or rank pages in a way that is logically inconsistent. Suggestions are made for search service providers to improve their service quality. Copyright © 2010 John Wiley & Sons, Ltd.
Persistent Identifierhttp://hdl.handle.net/10722/123864
ISSN
0960-0833
2023 Impact Factor: 1.5
2023 SCImago Journal Rankings: 0.431
ISI Accession Number ID
WOS:000304001500002
Funding AgencyGrant Number
Microsoft Research
University of Wollongong
Research Grants Council of Hong Kong717308
Australian Research CouncilARC DP0771733
Funding Information:

This project is supported in part by a Virtual Earth Award from Microsoft Research, a Small Grant from University of Wollongong, the General Research Fund of the Research Grants Council of Hong Kong (project no. 717308), and a Discovery Grant of the Australian Research Council (project no. ARC DP0771733). Preliminary results of this research were presented by invitation at the Microsoft Research Virtual Earth and Location Summit, Redmond, WA, U.S.A. held on 30 April-1 May 2008. The techniques presented in this paper are currently in a patent application process.

References
References in Scopus

 

DC FieldValueLanguage
dc.contributor.authorZhou, ZQ-
dc.contributor.authorZhang, S-
dc.contributor.authorHagenbuchner, M-
dc.contributor.authorTse, TH-
dc.contributor.authorKuo, FC-
dc.contributor.authorChen, TY-
dc.date.accessioned2010-10-06T04:18:59Z-
dc.date.available2010-10-06T04:18:59Z-
dc.date.issued2012-
dc.identifier.citationSoftware Testing, Verification and Reliability, 2012, v. 22 n. 4, p. 221-243-
dc.identifier.issn0960-0833-
dc.identifier.urihttp://hdl.handle.net/10722/123864-
dc.description.abstractSearch services are the main interface through which people discover information on the Internet. A fundamental challenge in testing search services is the lack of oracles. The sheer volume of data on the Internet prohibits testers from verifying the results. Furthermore, it is difficult to objectively assess the ranking quality because different assessors can have very different opinions on the relevance of a Web page to a query. This paper presents a novel method for automatically testing search services without the need of a human oracle. The experimental findings reveal that some commonly used search engines, including Google, Yahoo!, and Live Search, are not as reliable as what most users would expect. For example, they may fail to find pages that exist in their own repositories, or rank pages in a way that is logically inconsistent. Suggestions are made for search service providers to improve their service quality. Copyright © 2010 John Wiley & Sons, Ltd. A novel method for automatically testing search services without the need of a human oracle is presented. The experimental findings reveal that some commonly used search engines, including Google, Yahoo!, and Live Search, are not as reliable as what most users would expect. For example, they may fail to find pages that exist in their own repositories, or rank pages in a way that is logically inconsistent. Suggestions are made for search service providers to improve their service quality. Copyright © 2010 John Wiley & Sons, Ltd.-
dc.languageeng-
dc.publisherJohn Wiley & Sons Ltd. The Journal's web site is located at http://www3.interscience.wiley.com/cgi-bin/jhome/13635-
dc.relation.ispartofSoftware Testing, Verification and Reliability-
dc.rightsThis is the peer reviewed version of the following article: Software Testing, Verification and Reliability, 2012, v. 22 n. 4, p. 221-243, which has been published in final form at https://doi-org.eproxy.lib.hku.hk/10.1002/stvr.437. This article may be used for non-commercial purposes in accordance with Wiley Terms and Conditions for Use of Self-Archived Versions.-
dc.subjectSoftware testing-
dc.subjectMetamorphic testing-
dc.subjectVerification and validation-
dc.subjectInternet search-
dc.titleAutomated functional testing of online search services-
dc.typeArticle-
dc.identifier.openurlhttp://library.hku.hk:4550/resserv?sid=HKU:IR&issn=0960-0833&volume=&spage=&epage=&date=2010&atitle=Automated+functional+testing+of+online+search+services-
dc.identifier.emailZhou, ZQ: zhiquan@uow.edu.au-
dc.identifier.emailTse, TH: thtse@cs.hku.hk-
dc.identifier.authorityTse, TH=rp00546-
dc.description.naturepostprint-
dc.identifier.doi10.1002/stvr.437-
dc.identifier.scopuseid_2-s2.0-84862066626en_HK
dc.identifier.hkuros170797-
dc.identifier.hkuros203906-
dc.relation.referenceshttp://www.scopus.com/mlt/select.url?eid=2-s2.0-84862066626&selection=ref&src=s&origin=recordpageen_HK
dc.identifier.volume22-
dc.identifier.issue4-
dc.identifier.spage221-
dc.identifier.epage243-
dc.identifier.isiWOS:000304001500002-
dc.publisher.placeUnited Kingdom-
dc.identifier.scopusauthoridZhou, ZQ=15924687500en_HK
dc.identifier.scopusauthoridZhang, S=55247551800en_HK
dc.identifier.scopusauthoridHagenbuchner, M=6602995698en_HK
dc.identifier.scopusauthoridTse, TH=54971059300en_HK
dc.identifier.scopusauthoridKuo, FC=7006690410en_HK
dc.identifier.scopusauthoridChen, TY=35247605500en_HK
dc.identifier.citeulike11169292-
dc.identifier.issnl0960-0833-

Export via OAI-PMH Interface in XML Formats

OR

Export to Other Non-XML Formats