OpenAI says it ignored the concerns of its expert testers when it rolled out an update to its flagship ChatGPT artificial intelligence model that made it excessively agreeable.<\/p>\n
The company released an update to its GPT\u20114o model<\/a> on April 25 that made it \u201cnoticeably more sycophantic,\u201d which it then rolled back three days later due to safety concerns, OpenAI said<\/a> in a May 2 postmortem blog post.<\/p>\n The ChatGPT maker said its new models undergo safety and behavior checks<\/a>, and its \u201cinternal experts spend significant time interacting with each new model before launch,\u201d meant to catch issues missed by other tests.<\/p>\n During the latest model\u2019s review process before it went public, OpenAI said that \u201csome expert testers had indicated that the model\u2019s behavior \u2018felt\u2019 slightly off\u201d but decided to launch \u201cdue to the positive signals from the users who tried out the model.\u201d<\/p>\n \u201cUnfortunately, this was the wrong call,\u201d the company admitted. \u201cThe qualitative assessments were hinting at something important, and we should\u2019ve paid closer attention. They were picking up on a blind spot in our other evals and metrics.\u201d<\/p>\n OpenAI CEO Sam Altman said on April 27 that it was working to roll back changes making ChatGPT too agreeable. Source: <\/em>Sam Altman<\/em><\/a><\/p>\n Broadly, text-based AI models<\/a> are trained by being rewarded for giving responses that are accurate or rated highly by their trainers. Some rewards are given a heavier weighting, impacting how the model responds.<\/p>\n OpenAI said introducing a user feedback reward signal weakened the model\u2019s \u201cprimary reward signal, which had been holding sycophancy in check,\u201d which tipped it toward being more obliging.<\/p>\n \u201cUser feedback in particular can sometimes favor more agreeable responses, likely amplifying the shift we saw,\u201d it added.<\/p>\n After the updated AI model rolled out, ChatGPT users<\/a> had complained online about its tendency to shower praise on any idea it was presented, no matter how bad, which led OpenAI to concede<\/a> in an April 29 blog post that it \u201cwas overly flattering or agreeable.\u201d<\/p>\n For example, one user told ChatGPT it wanted to start a business selling ice over the internet, which involved selling plain old water for customers to refreeze. <\/p>\n Source: <\/em>Tim Leckemby<\/em><\/a><\/p>\n In its latest postmortem, it said such behavior from its AI could pose a risk, especially concerning issues such as mental health.<\/p>\n \u201cPeople have started to use ChatGPT for deeply personal advice \u2014 something we didn\u2019t see as much even a year ago,\u201d OpenAI said. \u201cAs AI and society have co-evolved, it\u2019s become clear that we need to treat this use case with great care.\u201d<\/p>\nOpenAI is now checking for suck up answers<\/h2>\n