Links for fulltext
(May Require Subscription)
- Publisher Website: 10.1109/TSE.2019.2915065
- Scopus: eid_2-s2.0-85111103140
- WOS: WOS:000660633800004
- Find via
Supplementary
- Citations:
- Appears in Collections:
Article: Metamorphic Robustness Testing: Exposing Hidden Defects in Citation Statistics and Journal Impact Factors
Title | Metamorphic Robustness Testing: Exposing Hidden Defects in Citation Statistics and Journal Impact Factors |
---|---|
Authors | |
Keywords | Robustness Database systems Software Fuzzing |
Issue Date | 2021 |
Publisher | IEEE. The Journal's web site is located at http://www.computer.org/tse |
Citation | IEEE Transactions on Software Engineering, 2021, v. 47 n. 6, p. 1164-1183 How to Cite? |
Abstract | We propose a robustness testing approach for software systems that process large amounts of data. Our method uses metamorphic relations to check software output for erroneous input in the absence of a tangible test oracle. We use this technique to test two major citation database systems: Scopus and the Web of Science. We report a surprising finding that the inclusion of hyphens in paper titles impedes citation counts, and that this is a result of the lack of robustness of the citation database systems in handling hyphenated paper titles. Our results are valid for the entire literature as well as for individual fields such as chemistry. We further find a strong and significant negative correlation between the journal impact factor (JIF) of IEEE Transactions on Software Engineering (TSE) and the percentage of hyphenated paper titles published in TSE. Similar results are found for ACM Transactions on Software Engineering and Methodology. A software engineering field-wide study reveals that the higher JIF-ranked journals are publishing a lower percentage of papers with hyphenated titles. Our results challenge the common belief that citation counts and JIFs are reliable measures of the impact of papers and journals, as they can be distorted simply by the presence of hyphens in paper titles. |
Persistent Identifier | http://hdl.handle.net/10722/271350 |
ISSN | 2023 Impact Factor: 6.5 2023 SCImago Journal Rankings: 1.868 |
ISI Accession Number ID |
DC Field | Value | Language |
---|---|---|
dc.contributor.author | Zhou, ZQ | - |
dc.contributor.author | Tse, TH | - |
dc.contributor.author | Witheridge, M | - |
dc.date.accessioned | 2019-06-24T01:08:10Z | - |
dc.date.available | 2019-06-24T01:08:10Z | - |
dc.date.issued | 2021 | - |
dc.identifier.citation | IEEE Transactions on Software Engineering, 2021, v. 47 n. 6, p. 1164-1183 | - |
dc.identifier.issn | 0098-5589 | - |
dc.identifier.uri | http://hdl.handle.net/10722/271350 | - |
dc.description.abstract | We propose a robustness testing approach for software systems that process large amounts of data. Our method uses metamorphic relations to check software output for erroneous input in the absence of a tangible test oracle. We use this technique to test two major citation database systems: Scopus and the Web of Science. We report a surprising finding that the inclusion of hyphens in paper titles impedes citation counts, and that this is a result of the lack of robustness of the citation database systems in handling hyphenated paper titles. Our results are valid for the entire literature as well as for individual fields such as chemistry. We further find a strong and significant negative correlation between the journal impact factor (JIF) of IEEE Transactions on Software Engineering (TSE) and the percentage of hyphenated paper titles published in TSE. Similar results are found for ACM Transactions on Software Engineering and Methodology. A software engineering field-wide study reveals that the higher JIF-ranked journals are publishing a lower percentage of papers with hyphenated titles. Our results challenge the common belief that citation counts and JIFs are reliable measures of the impact of papers and journals, as they can be distorted simply by the presence of hyphens in paper titles. | - |
dc.language | eng | - |
dc.publisher | IEEE. The Journal's web site is located at http://www.computer.org/tse | - |
dc.relation.ispartof | IEEE Transactions on Software Engineering | - |
dc.rights | IEEE Transactions on Software Engineering. Copyright © IEEE. | - |
dc.rights | ©2019 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. | - |
dc.subject | Robustness | - |
dc.subject | Database systems | - |
dc.subject | Software | - |
dc.subject | Fuzzing | - |
dc.subject | - | |
dc.title | Metamorphic Robustness Testing: Exposing Hidden Defects in Citation Statistics and Journal Impact Factors | - |
dc.type | Article | - |
dc.identifier.email | Tse, TH: thtse@cs.hku.hk | - |
dc.identifier.authority | Tse, TH=rp00546 | - |
dc.description.nature | postprint | - |
dc.identifier.doi | 10.1109/TSE.2019.2915065 | - |
dc.identifier.scopus | eid_2-s2.0-85111103140 | - |
dc.identifier.hkuros | 298100 | - |
dc.identifier.volume | 47 | - |
dc.identifier.issue | 6 | - |
dc.identifier.spage | 1164 | - |
dc.identifier.epage | 1183 | - |
dc.identifier.isi | WOS:000660633800004 | - |
dc.publisher.place | United States | - |
dc.identifier.issnl | 0098-5589 | - |