The ChatGPT Artificial Intelligence Chatbot: How Well Does It Answer Accounting Assessment Questions?-Reference-Cited by-同舟云学术

The ChatGPT Artificial Intelligence Chatbot: How Well Does It Answer Accounting Assessment Questions?

Published:2023-10-26 Issue:4 Volume:38 Page:81-108
ISSN:0739-3172
Container-title:Issues in Accounting Education
language:en
Short-container-title:

Author:

Wood David A.¹^ORCID,Achhpilia Muskan P.²,Adams Mollie T.³^ORCID,Aghazadeh Sanaz⁴,Akinyele Kazeem⁵^ORCID,Akpan Mfon⁶^ORCID,Allee Kristian D.⁷,Allen Abigail M.¹,Almer Elizabeth D.⁸^ORCID,Ames Daniel⁹,Arity Viktor¹⁰,Barr-Pulliam Dereck¹¹^ORCID,Basoglu K. Asli¹²,Belnap Andrew¹³,Bentley Jeremiah W.¹⁴¹⁵^ORCID,Berg Terje¹⁶,Berglund Nathan R.¹⁷,Berry Erica¹⁸,Bhandari Avishek¹⁹,Bhuyan Md Nazmul Hasan²⁰,Black Paul W.²¹^ORCID,Blondeel Eva²²,Bond David²³,Bonrath Annika²⁴^ORCID,Borthick A. Faye²⁵^ORCID,Boyle Erik S.²⁶^ORCID,Bradford Marianne²⁷^ORCID,Brandon Duane M.²¹,Brazel Joseph F.²⁷^ORCID,Brockbank Bryan G.²⁸^ORCID,Burger Marcus²⁹,Byzalov Dmitri³⁰^ORCID,Cannon James N.³¹^ORCID,Caro Cecil³²,Carr Abraham H.³³^ORCID,Cathey Jack³⁴,Cating Ryan³⁵^ORCID,Charron Kimberly³⁶,Chavez Stacy³⁷^ORCID,Chen Jason³⁸^ORCID,Chen Jennifer C.³⁹^ORCID,Chen Jennifer W.⁴⁰^ORCID,Cheng Christine⁴¹^ORCID,Cheng Xu²¹^ORCID,Christensen Brant E.¹^ORCID,Church Kimberly Swanson³^ORCID,Cicone N. J.¹⁷,Constance Patience⁴²^ORCID,Cooper Lauren A.⁴³^ORCID,Correia Candice L.⁴⁴^ORCID,Coyne Joshua⁴⁵,Cram W. Alec⁴⁶,Curtis Asher⁴⁷^ORCID,Daigle Ronald J.⁴⁸,Dannemiller Steve⁴⁹^ORCID,Davenport Stephan A.⁵⁰,Dawson Gregory S.⁵¹^ORCID,De Meyst Karen J. L.²^ORCID,Dell Scott⁵²^ORCID,Demirkan Sebahattin⁵³^ORCID,Denison Christine A.⁵⁴^ORCID,Desai Hrishikesh⁵⁵^ORCID,DeSimone Steven⁵⁶,Diehl Leah M.⁵⁷^ORCID,Dimes Ruth⁵⁸,Dong Bei⁵⁹^ORCID,Donnelly Amy⁶⁰^ORCID,du Pon Adam⁶¹^ORCID,Duan Huijue Kelly⁶²^ORCID,Duffey Ada⁶³,Dunn Ryan T.²¹^ORCID,Durkin Mary P.¹⁸,Dzuranin Ann C.⁶⁴^ORCID,Eberle Rachel M.⁷,Ege Matthew S.⁶⁵^ORCID,El Mahdy Dina⁶⁶^ORCID,Esplin Adam⁶⁷,Eulerich Marc⁶⁸,Everaert Patricia²²,Farah Nusrat⁶⁹^ORCID,Farish Lauren⁷⁰^ORCID,Favere-Marchesi Michael⁷¹^ORCID,Fayard Dutch⁵⁰,Filosa Jessica R.⁷²,Ford Melinda⁷³,Franz Diana R.⁷⁴^ORCID,Fulmer Bachman P.⁷⁵^ORCID,Fulmer Sarah⁷⁵^ORCID,Furner Zhan Z.⁷⁶,Gantman Sonia⁷⁷^ORCID,Garner Steve⁷⁸^ORCID,Garrett Jace⁷⁹^ORCID,Geng Xin⁸⁰^ORCID,Golden Joanna⁸¹^ORCID,Goldman William⁸²^ORCID,Gomez Josue¹,Gooley Mark⁸²^ORCID,Granitto Shawn P.⁸³,Green Karen Y.⁷⁴,Greenman Cindy L.⁸⁴^ORCID,Gupta Gaurav⁸⁵^ORCID,Guymon Ronald N.⁸⁶,Hale Kevin⁸⁵^ORCID,Harper Christopher J.⁸⁷,Hartt S. Allen⁸⁸^ORCID,Hawk Holly⁷⁹^ORCID,Hawkins Steven R.⁷³^ORCID,Hawkins Erin M.⁷⁹^ORCID,Hay David C.⁵⁸^ORCID,Heinzelmann Rafael⁸⁹,Henderson Cassy D.⁴⁸,Hendricks Bradley E.⁹⁰^ORCID,Heninger William G.¹,Hill Mary S.⁹¹^ORCID,Holden Nicole⁵¹,Holderness D. Kip⁴³^ORCID,Holt Travis P.²¹^ORCID,Hoopes Jeffrey L.⁹⁰,Hsieh Sheng-Feng⁹²^ORCID,Huang Feiqi⁹³^ORCID,Huang Hua-Wei⁹⁴^ORCID,Huang Ting-Chiao⁹⁵^ORCID,Huels Brian W.¹⁹,Hunter Kara⁹⁶^ORCID,Hurley Patrick J.⁸²^ORCID,Inger Kerry²¹^ORCID,Islam Sharif⁶⁹^ORCID,Ison Isaac⁹⁷,Issa Hussein⁹⁸^ORCID,Jackson Andrew B.⁹⁹^ORCID,Jackson Scott C.¹⁰⁰^ORCID,Janvrin Diane J.⁵⁴,Jimenez Peggy D.¹⁰¹^ORCID,Johanson Daniel¹⁰²^ORCID,Judd J. Scott¹⁰³,Kawada Brett S.¹⁰⁴,Kelton Andrea Seaton¹⁰⁵^ORCID,Kern Sara¹⁰⁶^ORCID,Kerr Jon N.¹^ORCID,Keune Marsha B.¹⁰⁷,Kim Mindy^ORCID,Knox Brian D.⁸⁸^ORCID,Kogan Gregory⁷²^ORCID,Kotb Amr¹⁰⁸,Krane Ronja²⁴^ORCID,Kremin Joleen⁸^ORCID,Krieg Kimberly S.¹⁸,Kugel Jonathan¹⁰⁹^ORCID,Kulset Ellen M.¹¹⁰^ORCID,Kuruppu Chamara¹¹⁰,LaDuca Garrison¹¹¹,Lamberton Barbara A.¹¹²,Lamboy-Ruiz Melvin A.⁶¹^ORCID,Lang Bradley¹⁷^ORCID,Larocque Stephannie A.¹¹³^ORCID,Larson Melissa P.¹,Lawson Bradley P.²⁸^ORCID,Lawson James G.¹¹⁴^ORCID,Lee Lorraine⁸⁵^ORCID,Lenk Margarita M.¹¹⁵,Li-Kuehne Michelle⁴⁴^ORCID,Liljegren Jonathan¹,Lin Yi-Hung⁹⁵^ORCID,Liu Wu-Po⁹⁴^ORCID,Liu Zishang¹¹⁶^ORCID,Lock Brandon¹¹⁷^ORCID,Long James H.²¹^ORCID,Loraas Tina²¹,Lowensohn Suzanne¹¹⁸,Loy Thomas R.¹¹⁹,Lyngstadaas Hakim¹²⁰,Maas Wim¹⁵^ORCID,MacGregor Jason E.⁷⁰^ORCID,Madsen Dag Øivind¹¹⁰^ORCID,Malone Carissa L.²⁷,Margolin Maximilian¹²¹^ORCID,Marshall Mary E.⁸,Martin Rachel M.³¹^ORCID,Mpofu Colleen McClain⁴⁰,McCoy Chris¹²²,McGuigan Nicholas C.⁹⁵,McSwain Dwayne N.¹²³^ORCID,Meckfessel Michele D.⁴⁰,Mellon Mark J.⁶⁴^ORCID,Melton Olivia S.¹²⁴,Mercado Julie M.⁸⁸,Mitsuda Steven⁵⁷,Modugu Kennedy¹²⁵,Moehrle Stephen⁴⁰^ORCID,Chaghervand Amirali Moeini¹¹⁶^ORCID,Moffitt Kevin⁹⁸,Moon Joon Seok¹²⁶^ORCID,Muehlmann Brigitte¹²⁷,Murray Johnna⁴⁰,Mwaungulu Emmanuel S.¹²⁸^ORCID,Myers Noah¹²⁹,Naegle J. Conrad³^ORCID,Ndicu Martin J.⁶⁴,Nelson Aaron S.⁶⁷,Nguyen Anh L.¹⁰,Niederkofler Thomas²^ORCID,Nikbakht Ehsan¹³⁰,O'Brien Ann D.¹³¹^ORCID,Ogunade Kehinde M.⁸¹^ORCID,O'Leary Daniel¹³²^ORCID,Oler Mitchell J.¹³³^ORCID,Oler Derek K.⁴⁵^ORCID,Olsen Kari Joseph¹²⁹^ORCID,Otalor John I.¹³⁴^ORCID,Outlaw Kyle W.¹¹¹^ORCID,Ozlanski Michael. E.¹³⁵,Parlier Jenny⁸⁵^ORCID,Paterson Jeffrey S.¹²²^ORCID,Pearson Christopher A.²⁶,Petersen Michael J.¹³⁶^ORCID,Petra Steven T.¹³⁰,Pickard Matthew D.⁶⁴,Pickerd Jeffrey⁴¹^ORCID,Pinsker Robert¹³⁷^ORCID,Plante Catherine¹³⁸^ORCID,Plečnik James M.¹³⁹,Price Richard A.¹⁴⁰^ORCID,Quick Linda A.⁷⁶^ORCID,Raedy Jana⁹⁰,Raschke Robyn³⁶^ORCID,Ravenscraft Julie³^ORCID,Richardson Vernon⁷^ORCID,Rixom Brett A.¹⁴¹^ORCID,Robertson John F.⁵⁵^ORCID,Rock Iyad¹⁴²^ORCID,Romney Miles A.¹²²,Rozario Andrea⁸⁶,Ruff Michael F.⁸²,Rupley Kathleen⁸^ORCID,Saeedi Ali¹⁴³^ORCID,Saiewitz Aaron³⁶^ORCID,Salzsieder Leigh W.⁶⁰,Sarkar Sayan¹⁴⁴,Saulls Michael¹^ORCID,Scanlan Tialei A.³⁹^ORCID,Schaefer Tammie J.⁶⁰^ORCID,Schaupp Daniel¹⁴⁵,Schneider Gary P.¹⁴⁶,Seebeck Andreas¹⁴⁷^ORCID,Sellers R. Drew¹⁴⁸,Seto Samantha C.⁷¹^ORCID,Sevel Romi-Lee¹⁴⁹^ORCID,Shan Yuxin¹⁵⁰,Sherwood Matthew G.¹⁴^ORCID,Singorahardjo Maggie¹⁵¹^ORCID,Skaftadottir Hanna Kristin¹⁵²^ORCID,Skomra Justyna¹⁵³^ORCID,Smith Jason L.³⁶^ORCID,Smith Dallin O.¹⁵⁴^ORCID,Smith James¹⁵⁵^ORCID,Snow Mason C.¹⁵⁶^ORCID,Sommerfeldt Ryan D.¹⁵⁷^ORCID,Sorensen Kate B.⁸¹^ORCID,Sorensen Trevor L.¹³³^ORCID,Spieler Andrew C.¹³⁰,Stallings Matthew A.⁸⁸^ORCID,Stallings Lesya⁹¹^ORCID,Stancill Alan¹⁷,Stanley Jonathan D.²¹^ORCID,Stefaniak Chad M.¹⁵⁸^ORCID,Stephens Nathaniel M.³⁹,Stewart Bryan W.⁸⁴^ORCID,Stratopoulos Theophanis C.⁴⁶,Street Daniel A.¹¹⁴^ORCID,Subedi Meena¹⁹,Summers Scott L.¹,Sundkvist Charlotte H.¹¹⁰,Synn Christina¹⁵⁹^ORCID,Tadesse Amanuel¹⁶⁰^ORCID,Tapis Gregory P.³^ORCID,Tassin Kerri³^ORCID,Taylor Samantha¹⁶¹^ORCID,Teal Mary¹⁶²^ORCID,Teeter Ryan¹⁶³^ORCID,Tharapos Meredith¹⁰^ORCID,Theis Jochen C.¹⁶⁴^ORCID,Thomas Jack¹,Thompson Kristen S.²⁶,Thornock Todd A.¹⁶⁵^ORCID,Tietz Wendy¹⁴⁸^ORCID,Travalent Anthony M.¹⁶⁶^ORCID,Trinkle Brad S.⁵⁴,Truelson J. Mike¹⁷^ORCID,Turner Michael C.¹⁶⁰,Vagner Brandon¹⁰⁵^ORCID,Vakilzadeh Hamid¹⁹,van der Geest Jesse¹⁵,van Pelt Victor¹⁴⁵^ORCID,Vandervelde Scott D.³⁴^ORCID,Vega Jose¹⁶⁷^ORCID,Vera-Muñoz Sandra¹¹³^ORCID,Villanueva Brigham¹,Vincent Nishani Edirisinghe⁵⁰,Wagener Martin²⁴^ORCID,Walton Stephanie⁴^ORCID,Warne Rick C.¹⁸,Watanabe Olena V.⁵⁴^ORCID,Watson David⁵⁹^ORCID,Watson Marcia Weidenmier¹⁶⁸^ORCID,Weber Jill¹⁹^ORCID,Weirich Thomas¹⁶⁹^ORCID,West Ashley N.¹⁷⁰^ORCID,Wilford Amanda L.¹⁷¹^ORCID,Wilson Aaron B.¹⁷²^ORCID,Winrow Brian¹²²^ORCID,Winrow Timothy¹⁷³^ORCID,Winrow Tasia S.¹⁷⁴^ORCID,Wiseman Denise¹⁷⁵^ORCID,Witte Annie L.⁸²,Wood Bryan D.³²^ORCID,Wood Jessica¹^ORCID,Woolley Darryl¹⁷⁶^ORCID,Wright Nicole S.¹⁷⁷^ORCID,Wu Juan¹⁷⁴,Xiong Xiaomei,Yatsenko Dimitri¹⁹^ORCID,Yazzie Courtney E.⁷⁹^ORCID,Young Glen M.¹⁷⁸,Zhang Chanyuan,Zimmerman Aleksandra B.¹²²,Zoet Emily¹⁷⁹^ORCID

Affiliation:

1. Brigham Young University

2. Radboud University

3. Missouri State University

4. Louisiana State University

5. University of Wisconsin–Oshkosh

6. Southern Methodist University

7. University of Arkansas

8. Portland State University

9. Bryant University

10. RMIT University

11. University of Louisville

12. University of Delaware

13. The University of Texas at Austin

14. University of Massachusetts Amherst

15. Tilburg University

16. Norwegian University of Science and Technology

17. Mississippi State University

18. University of San Diego

19. University of Wisconsin–Whitewater

20. North Carolina Agricultural and Technical State University

21. Auburn University

22. Ghent University

23. University of Technology, Sydney

24. University of Duisburg-Essen

25. Georgia State University

26. Idaho State University

27. North Carolina State University

28. Oklahoma State University

29. The University of North Carolina at Pembroke

30. Temple University

31. Utah State University

32. Unaffiliated

33. Creighton University

34. The University of North Carolina at Charlotte

35. University of Central Arkansas

36. University of Nevada, Las Vegas

37. Loyola University Maryland

38. Central Connecticut State University

39. Brigham Young University–Hawaii

40. University of Missouri–St. Louis

41. The University of Mississippi

42. The University of Kansas

43. West Virginia University

44. Whitworth University

45. Texas Tech University

46. University of Waterloo

47. University of Washington

48. Sam Houston State University

49. The University of Alabama

50. The University of Tennessee at Chattanooga

51. Arizona State University

52. Francis Marion University

53. Manhattan College

54. Iowa State University

55. Arkansas State University

56. College of the Holy Cross

57. University of Montana

58. The University of Auckland

59. University of South Florida St. Petersburg

60. University of Missouri–Kansas City

61. Georgia Southern University

62. Sacred Heart University

63. University of Wisconsin–Stevens Point

64. Northern Illinois University

65. Texas A&M University

66. Morgan State University

67. The University of Texas at El Paso

68. University Duisburg-Essen

69. Southern Illinois University Carbondale

70. Baylor University

71. Simon Fraser University

72. Virginia Polytechnic Institute and State University

73. Southern Utah University

74. The University of Toledo

75. The University of Tampa

76. East Carolina University

77. Bentley University

78. Tennessee Tech University

79. Clemson University

80. Berry College

81. The University of Memphis

82. Northeastern University

83. Florida Gulf Coast University

84. Utah Tech University

85. The University of North Carolina Wilmington

86. University of Illinois at Urbana-Champaign

87. Grand Valley State University

88. Boise State University

89. University of Agder

90. The University of North Carolina at Chapel Hill

91. Kennesaw State University

92. National Taiwan University

93. Pace University

94. National Cheng Kung University

95. Monash University

96. Fairfield University

97. Truman State University

98. Rutgers, The State University of New Jersey

99. UNSW Sydney

100. University of South Dakota

101. University of North Texas

102. Norwegian School of Economics

103. University of Illinois at Chicago

104. Woodbury University

105. Middle Tennessee State University

106. Gonzaga University

107. University of Dayton

108. Zayed University

109. Christopher Newport University

110. University of South-Eastern Norway

111. Samford University

112. University of Hartford

113. University of Notre Dame

114. Bucknell University

115. Colorado State University

116. University of Houston–Downtown

117. Baruch College–CUNY

118. The University of Vermont

119. University of Bremen

120. BI Norwegian Business School

121. Erasmus University Rotterdam

122. Florida State University

123. Appalachian State University

124. Augustana College

125. HCT Business School Dubai

126. California State University, Northridge

127. Babson College

128. Palm Beach Atlantic University

129. Utah Valley University

130. Hofstra University

131. University of Wisconsin–Madison

132. University of Southern California

133. University of Wyoming

134. University of Calabar

135. Susquehanna University

136. North Dakota State University

137. Florida Atlantic University

138. University of New Hampshire

139. Loyola Marymount University

140. The University of Oklahoma

141. University of Nevada, Reno

142. Lewis University

143. University of Minnesota, Crookston

144. University of Mary Washington

145. WHU-Otto Beisheim School of Management

146. California State University, Monterey Bay

147. Jacobs University Bremen

148. Kent State University

149. York University

150. University of Wisconsin–Eau Claire

151. The University of Melbourne

152. Bifrost University

153. The Pennsylvania State University, Behrend College

154. West Texas A&M University

155. University of Lethbridge

156. California State University, Fullerton

157. Washington State University

158. University of South Carolina

159. American University

160. The University of New Orleans

161. Dalhousie University

162. Emporia State University

163. University of Pittsburgh

164. University of Southern Denmark

165. University of Nebraska–Lincoln

166. Lone Star College

167. Stephen F. Austin State University

168. Trinity University

169. Central Michigan University

170. Kansas State University

171. IE University

172. Ohio University

173. Eastern New Mexico University

174. Minnesota State University, Mankato

175. University of Central Florida

176. University of Idaho

177. James Madison University

178. Texas State University

179. Ferris State University

Abstract

ABSTRACT ChatGPT, a language-learning model chatbot, has garnered considerable attention for its ability to respond to users’ questions. Using data from 14 countries and 186 institutions, we compare ChatGPT and student performance for 28,085 questions from accounting assessments and textbook test banks. As of January 2023, ChatGPT provides correct answers for 56.5 percent of questions and partially correct answers for an additional 9.4 percent of questions. When considering point values for questions, students significantly outperform ChatGPT with a 76.7 percent average on assessments compared to 47.5 percent for ChatGPT if no partial credit is awarded and 56.5 percent if partial credit is awarded. Still, ChatGPT performs better than the student average for 15.8 percent of assessments when we include partial credit. We provide evidence of how ChatGPT performs on different question types, accounting topics, class levels, open/closed assessments, and test bank questions. We also discuss implications for accounting education and research.

Publisher

American Accounting Association

Subject

Education,Accounting

Link

https://publications.aaahq.org/iae/article-pdf/38/4/81/101197/i0739-3172-38-4-81.pdf

Reference53 articles.

1. Chatbots: History, technology, and applications;Adamopoulou;Machine Learning with Applications,2020

2. Agomuoh, F. 2023. The best ChatGPT alternatives (according to ChatGPT).https://www.digitaltrends.com/computing/the-best-chatgpt-alternatives-according-to-chatgpt/

3. The data analytics journey: Interactions among auditors, managers, regulation, and technology;Austin;Contemporary Accounting Research,2021

4. Integrating artificial intelligence into the accounting curriculum;Baldwin-Morgan;Accounting Education,1995

Cited by 32 articles. 订阅此论文施引文献订阅此论文施引文献，注册后可以免费订阅5篇论文的施引文献，订阅后可以查看论文全部施引文献

1. Using chat GPT to evaluate police threats, risk and harm;International Journal of Law, Crime and Justice;2024-09

2. Artificial intelligence co-piloted auditing;International Journal of Accounting Information Systems;2024-09

3. Can ChatGPT Be a Certified Accountant? Assessing the Responses of ChatGPT for the Professional Access Exam in Portugal;Administrative Sciences;2024-07-16

4. Is it all hype? ChatGPT’s performance and disruptive potential in the accounting and auditing industries;Review of Accounting Studies;2024-06-27

5. ChatGPT: reflections from the UK higher education institutions, accountancy bodies and BIG4s;Accounting Research Journal;2024-06-18