OpenAI has created a new AI tool called CriticGPT to help spot mistakes in ChatGPT‘s work, especially when it comes to writing computer code. This is important because as AI gets smarter, it can be hard for humans to notice when it makes errors.
Key Takeaways
CriticGPT is based on the same technology as ChatGPT but is trained differently. It learned by looking at examples where humans added mistakes to ChatGPT’s code on purpose. Then, people taught CriticGPT how to find and explain these errors.
The results are impressive. When people use CriticGPT to check ChatGPT’s code, they do a better job than those without CriticGPT’s help 60% of the time. This shows that AI can be a useful partner for humans in spotting AI mistakes.
One of the best things about CriticGPT is that it’s good at finding real problems without pointing out tiny, unimportant issues. It also doesn’t make up problems that aren’t there as often as ChatGPT does when it tries to check its own work.
In tests, people preferred CriticGPT’s feedback over ChatGPT’s self-checks 63% of the time when looking at naturally occurring mistakes. This means CriticGPT is better at finding and explaining real problems in the code.
OpenAI’s research also showed that it’s tricky to get people to agree on what makes good code or good feedback. However, when there were clear, specific errors to find, people were more likely to agree on whether the feedback was helpful.
The company plans to use tools like CriticGPT to help train future AI models. This could make AI systems more accurate and trustworthy over time.
However, CriticGPT isn’t perfect. It sometimes makes mistakes too, and there are still challenges in dealing with very long or complex tasks. OpenAI is working on improving these areas.
This new tool is an important step in making AI safer and more reliable. By using AI to check AI, we can catch more mistakes and understand them better. This could help make AI systems that are smarter and more trustworthy in the future.
