OpenAI, the company behind ChatGPT, has developed a tool that can tell if the large language model has generated a… Continue reading ChatGPT parent company, OpenAI, has ‘promising’ tool to detect AI content
The post ChatGPT parent company, OpenAI, has ‘promising’ tool to detect AI content appeared first on ReadWrite.
OpenAI, the company behind ChatGPT, has developed a tool that can tell if the large language model has generated a piece of text, but it is hesitant to release it due to negative customer responses.
The detection tool involves text watermarking – making slight changes to how ChatGPT selects its words in a way that can be detected with a high degree of accuracy with a tool that knows the watermark.
According to The Wall Street Journal, releasing the detection tool would be “just a matter of pressing a button”, but the results of a survey showed that nearly 30% of ChatGPT users would use the artificial intelligence (AI) tool less if it deployed watermarking when rival chatbots did not.
According to an OpenAI spokesperson, “The text watermarking method we’re developing is technically promising but has important risks we’re weighing while we research alternatives,” she said. “We believe the deliberate approach we’ve taken is necessary given the complexities involved and its likely impact on the broader ecosystem beyond OpenAI.”
The company has also updated a page on its website to elaborate on some of its reasoning. A key concern they have is the fact that the watermarking could “disproportionately impact some groups”, particularly non-native English speakers who use AI as a writing tool.
They also detailed their concerns about how easily the watermarking could be overcome, stating “While it has been highly accurate and even effective against localized tampering, such as paraphrasing, it is less robust against globalized tampering; like using translation systems, rewording with another generative model, or asking the model to insert a special character in between every word and then deleting that character – making it trivial to circumvention by bad actors.”
On the same page, OpenAI also mentions that their focus has been on developing tools to detect audiovisual content because images, audio, and videos “are widely considered to present higher levels of risk at this stage of capabilities of our models.”
According to The Wall Street Journal, a recent survey conducted by the Center for Democracy and Technology showed that 59% of middle school and high school teachers are sure some students are using AI in their work, up from 42% in the previous year, and debates about AI in education rage on.
One of the problems is the lack of effective AI detection tools – there are many out there but the most proficient ones are locked behind paywalls, and even these are not immune to false positives and other failures. There are some ways to tell if an AI wrote a piece of text but as the technology behind these LLMs becomes more sophisticated, it will become harder to detect.
As it becomes more clear to parties internal to OpenAI that the watermarking method will not affect the quality of ChatGPT’s output, internal pushback on withholding the tool increases. “Our ability to defend our lack of text watermarking is weak now that we know it doesn’t degrade outputs,” said employees who were involved in testing the tool, according to internal documents seen by The Wall Street Journal. “Without this, we risk credibility as responsible actors,” said a summary of a recent meeting on the issue of AI detection.
Featured image credit: generated by Ideogram
The post ChatGPT parent company, OpenAI, has ‘promising’ tool to detect AI content appeared first on ReadWrite.
