Information Retrieval Performance in Text Generation using Knowledge from Generative Pre-trained Transformer (GPT-3)

Kaira Milani Fitria

doi:10.34312/jjom.v5i2.20574

Information Retrieval Performance in Text Generation using Knowledge from Generative Pre-trained Transformer (GPT-3)

Kaira Milani Fitria

Abstract

The rise of advanced language models like GPT-3 and text generation has witnessed remarkable progress. However, leveraging the vast amount of knowledge within these models to enhance information retrieval performance remains an area that needs to be explored. This research used Artificial Intelligence, specifically the OpenAI GPT-3 language model, to create an application to help make written content. This research investigates the impact of incorporating GPT-3's knowledge into text generation processes and evaluates its influence on information retrieval tasks. Several features in text generation generate text that requires exact information, such as specifications for a product and accurate descriptions of a job or product, which are included in the concept of information retrieval in text creation by language models. The research used the few-shot learning method in the GPT-3 language model. The generated responses are then evaluated using established information retrieval metrics such as precision, recall, and F1-score. The findings of this research reveal the effectiveness of utilizing GPT-3's knowledge in enhancing information retrieval performance. The generated responses demonstrate improved relevance to user queries, resulting in the same performance precision and recall scores compared to other paid text generator websites. Application results are testing in capabilities of retrieving some information. Application capabilities tested on other commercial text generator engines. The test results obtained BERTscore 86\% (precision), 88\% (recall), and 87\% (F1-Score).

Keywords

Information Retrieval; Language Model; GPT; Text Generation; Few-shot Learning

Full Text:

PDF

References

Z. Latinovic and S. C. Chatterjee, “Achieving the promise of ai and ml in delivering economic and relational customer value in b2b,” Journal of Business Research, vol. 144, pp. 966–974, 2022, doi: 10.1016/j.jbusres.2022.01.052.

M. Bahja, “Natural language processing applications in business,” 2021, doi: 10.5772/intechopen.92203.

S. F. Chen and J. Goodman, “An empirical study of smoothing techniques for language modeling,” Computer Speech & Language, vol. 13, no. 4, pp. 359–393, 1999, doi: 10.1006/csla.1999.0128.

A. Radford, K. Narashiman, T. Salimans, and I. Sutskever1, “Improving language understanding by generative pre-training,” OpenAI, 2018, [online] available: https://openai.com/research/language-unsupervised.

C. M. Gevaert, M. Carman, B. Rosman, Y. Georgiadou, and R. Soden, “Fairness and accountability of ai in disaster risk management: Opportunities and challenges,” Patterns, vol. 2, no. 11, p. 100363, 2021, doi: 10.1016/j.patter.2021.100363.

A. Chan, “Gpt-3 and instructgpt: technological dystopianism, utopianism, and “contextual” perspectives in ai ethics and industry,” AI and Ethics, vol. 3, no. 1, pp. 53–64, 2023, doi: 10.1007/s43681-022-00148-6.

S. Y. Kim, H. Park, K. Shin, and K. Kim, “Ask me what you need: Product retrieval using knowledge from gpt-3,” arxiv, 2022, [online] available: http://arxiv.org/abs/2207.02516.

G. P. Transformer, A. O. Thunstrom, and S. Steingrimsson, “Can gpt-3 write an academic paper on itself, with minimal human input?” HAL open science, vol. 1, p. 03701250, 2022.

R. Dale, “Gpt-3: What’s it good for?” Natural Language Engineering, vol. 27, no. 1, pp. 113–118, 2021, doi: 10.1017/S1351324920000601.

Y. Wang, Q. Yao, J. T. Kwok, and L. M. Ni, “Generalizing from a few examples,” ACM Computing Surveys, vol. 53, no. 3, pp. 1–34, 2021, doi: 10.1145/3386252.

D. Haluza and D. Jungwirth, “Artificial intelligence and ten societal megatrends: An exploratory study using gpt-3,” Systems, vol. 11, no. 3, p. 120, 2023, doi: 10.3390/systems11030120.

R. Singh and V. Garg, “Human factors in nde 4.0 development decisions,” Journal of Nondestructive Evaluation, vol. 40, no. 3, p. 71, 2021, doi: 10.1007/s10921-021-00808-3.

T. J. Ackermann, “Gpt-3: a robot wrote this entire article. are you scared yet, human?” Artificial Intelligence: ANI, LogicGate Computing, AGI, ASI, 2020.

M. Zhang and J. Li, “A commentary of gpt-3 in mit technology review 2021,” Fundamental Research, vol. 1, no. 6, pp. 831–833, 2021, doi: 10.1016/j.fmre.2021.11.011.

B. Ding, C. Qin, L. Liu, L. Bing, S. Joty, and B. Li, “Is gpt-3 a good data annotator?” arxiv, 2022, [online] available: http://arxiv.org/abs/2212.10450.

B. Lester, R. Al-Rfou, and N. Constant, “The power of scale for parameter-efficient prompt tuning.” Association for Computational Linguistics, 2021, pp. 3045–3059, doi: 10.18653/v1/2021.emnlp-main.243.

A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. u. Kaiser, and I. Polosukhin, “Attention is all you need,” in Advances in Neural Information Processing Systems, I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, Eds., vol. 30. Curran Associates, Inc., 2017.

J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “Bert: Pre-training of deep bidirectional transformers for language understanding,” arxiv, vol. 2, 2019, [online] available: https://arxiv.org/abs/1810.04805.

A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, and I. Sutskever, “Language models are unsupervised multitask learners,” paperswithcode, 2019, [online] available: https://paperswithcode.com/paper/language-models-are-unsupervised-multitask.

T. B. Brown, et al., “Language models are few-shot learners,” Computation and Language, vol. 2, 2020.

DOI: https://doi.org/10.34312/jjom.v5i2.20574