ChatGPT answers more than half of software engineering questions incorrectly

The results showed that out of the 512 questions, 259 (52%) of ChatGPT’s answers were incorrect and only 248 (48%) were correct. Moreover, a whopping 77% of the answers were verbose.

[…]

According to the study, the well-articulated responses ChatGPT outputs caused the users to overlook incorrect information in the answers.

“Users overlook incorrect information in ChatGPT answers (39.34% of the time) due to the comprehensive, well-articulated, and humanoid insights in ChatGPT answers,” the authors wrote.

The generation of plausible-sounding answers that are incorrect is a significant issue across all chatbots because it enables the spread of misinformation. In addition to that risk, the low accuracy scores should be enough to make you reconsider using ChatGPT for these types of prompts. —ZDNet

Post was last modified on 10 Aug 2023 2:43 pm

Share
Published by
Dennis G. Jerz
Tags: ai

Recent Posts

My virtual army of grey capsules is just barely smart enough to follow my avatar around. #unity3d

My virtual army of grey capsules is just barely smart enough to follow my avatar…

2 hours ago

Oops. I’m going to bed now. I’ll fix it tomorrow.

Oops. I’m going to bed now. I’ll fix it tomorrow.

21 hours ago

First steps towards reconstructing a 30-year-old simulation. Marking the locations of the traditional performance stations for the York Corpus Christi pageant. #medievalyork #mysteryplay #unity3d

First steps towards reconstructing a 30-year-old simulation. Marking the locations of the traditional performance stations…

2 days ago

Support the arts in your community! #augustwilson #radiogolf

Support the arts in your community! #augustwilson #radiogolf

1 week ago