{rfName}
Th

Indexed in

License and use

Citations

26

Altmetrics

Analysis of institutional authors

Rees, Geraint PaulCorresponding Author

Share

January 11, 2024
Publications
>
Article
No

The Effectiveness of OpenAI GPT-Generated Definitions Versus Definitions from an English Learners' Dictionary in a Lexically Orientated Reading Task

Publicated to:International Journal Of Lexicography. 37 (1): 50-74 - 2023-12-13 37(1), DOI: 10.1093/ijl/ecad030

Authors: Rees, Geraint Paul; Lew, Robert

Affiliations

Adam Mickiewicz Univ, Poznan, Poland - Author
Univ Rovira i Virgili, Dept English & German Studies, Tarragona, Spain - Author

Abstract

In metalexicographical research, experts have judged the performance of technologies such as OpenAI Generative Pretrained Transformer (GPT) in lexicographic production tasks as promising yet inferior to human lexicographers. It remains unclear whether this perceived inferiority limits the effectiveness of AI-generated lexicography in resolving practical language doubts. Accordingly, this study compares the effectiveness of AI-generated definitions to those from the Macmillan English Dictionary (MED) in resolving vocabulary doubts in a multiple-choice reading task designed to test lexical knowledge. It involves 43 L2 English users in the third year of an English studies degree at a Spanish university. Students provided with MED definitions performed better on the reading task than those without access to definitions. However, there was no significant difference between the performance of students with either MED definitions or without definitions altogether, and those provided with AI-definitions. The implications of these findings are discussed along with avenues for further research.

Keywords

AiChatgptEflGptLexicographyMonolingual learners' dictionariesPedagogical lexicographyReadingReading comprehensionVocabulary testing

Quality index

Bibliometric impact. Analysis of the contribution and dissemination channel

The work has been published in the journal International Journal Of Lexicography due to its progression and the good impact it has achieved in recent years, according to the agency Scopus (SJR), it has become a reference in its field. In the year of publication of the work, 2023, it was in position , thus managing to position itself as a Q2 (Segundo Cuartil), in the category Linguistics and Language. Notably, the journal is positioned en el Cuartil Q3 for the agency WoS (JCR) in the category Linguistics.

From a relative perspective, and based on the normalized impact indicator calculated from World Citations provided by WoS (ESI, Clarivate), it yields a value for the citation normalization relative to the expected citation rate of: 9.89. This indicates that, compared to works in the same discipline and in the same year of publication, it ranks as a work cited above average. (source consulted: ESI Nov 14, 2024)

This information is reinforced by other indicators of the same type, which, although dynamic over time and dependent on the set of average global citations at the time of their calculation, consistently position the work at some point among the top 50% most cited in its field:

  • Field Citation Ratio (FCR) from Dimensions: 27.06 (source consulted: Dimensions Aug 2025)

Specifically, and according to different indexing agencies, this work has accumulated citations as of 2025-08-02, the following number of citations:

  • WoS: 9

Impact and social visibility

From the perspective of influence or social adoption, and based on metrics associated with mentions and interactions provided by agencies specializing in calculating the so-called "Alternative or Social Metrics," we can highlight as of 2025-08-02:

  • The use, from an academic perspective evidenced by the Altmetric agency indicator referring to aggregations made by the personal bibliographic manager Mendeley, gives us a total of: 41.
  • The use of this contribution in bookmarks, code forks, additions to favorite lists for recurrent reading, as well as general views, indicates that someone is using the publication as a basis for their current work. This may be a notable indicator of future more formal and academic citations. This claim is supported by the result of the "Capture" indicator, which yields a total of: 54 (PlumX).

With a more dissemination-oriented intent and targeting more general audiences, we can observe other more global scores such as:

  • The Total Score from Altmetric: 11.15.
  • The number of mentions on the social network X (formerly Twitter): 14 (Altmetric).

Leadership analysis of institutional authors

This work has been carried out with international collaboration, specifically with researchers from: Poland.

There is a significant leadership presence as some of the institution’s authors appear as the first or last signer, detailed as follows: First Author (Rees, Geraint Paul) .

the author responsible for correspondence tasks has been Rees, Geraint Paul.