ChatGPT answers more than half of software engineering questions incorrectly

The results showed that out of the 512 questions, 259 (52%) of ChatGPT’s answers were incorrect and only 248 (48%) were correct. Moreover, a whopping 77% of the answers were verbose.

[…]

According to the study, the well-articulated responses ChatGPT outputs caused the users to overlook incorrect information in the answers.

“Users overlook incorrect information in ChatGPT answers (39.34% of the time) due to the comprehensive, well-articulated, and humanoid insights in ChatGPT answers,” the authors wrote.

The generation of plausible-sounding answers that are incorrect is a significant issue across all chatbots because it enables the spread of misinformation. In addition to that risk, the low accuracy scores should be enough to make you reconsider using ChatGPT for these types of prompts. —ZDNet

Post was last modified on 10 Aug 2023 2:43 pm

Share
Published by
Dennis G. Jerz
Tags: ai

Recent Posts

Another corner building. Designed and textured. Needs an interior. #blender3d #design #aesthetics #medievalyork #mysteryplay

Another corner building. Designed and textured. Needs an interior. #blender3d #design #aesthetics #medievalyork #mysteryplay

3 hours ago

There’s No Longer Any Doubt That Hollywood Writing Is Powering AI

Two years after the release of ChatGPT, it may not be surprising that creative work…

2 days ago

The complex geometry on this wedge building took me all weekend.  #blender3d #medievalyork #mysteryplay #cgi #aesthetics #design

The complex geometry on this wedge building took me all weekend. The interior walls still…

4 days ago

Sesame Street had a big plot twist in November 1986

My older siblings say they remember our mother sitting them down to watch a new…

5 days ago