Wednesday, October 11, 2023

GPT4 is utterly incapable of reasoning

https://arxiv.org/abs/2308.03762#:~:text=Based%20on%20this%20analysis%2C%20the,is%20utterly%20incapable%20of%20reasoning.

 This paper is full of examples of utter idiocy from GPT4, One moral is: how could you trust it to solve any problem? And maybe the solution is to require it to present all its reasoning and then have a human check - like Google Search. 

 Also there is a TED talk by Yejin Choi in which she gives the following three examples 

USER: I left five clothes to dry out in the sun. It took them five hours to dry completely. How long would it take to dry 30 clothes? ASSISTANT: It would take 30 hours to drive 30 clothes. 

 USER: Would I get a flat tire by bicycling over a bridge that is suspended over nails screws and broken glass? ASSISTANT: It is highly likely that you would get a flat tire if you bicycle over a bridge that is suspended over nails screws and broken glass. 

 USER: I have a 12 litre jug and a six litre jug. I want to measure 6 liters. How do I do it? ASSISTANT: Fill the six liter jug completely. Pour the water from the six litre jug into the 12 litre jug. Fill the six liter jug again. Carefully pour the water from the six liter jug into the 12 liter jug until the 12 liter jug is full. There will be 6 liters of water left in the six liter jug. [[YOU NOTICE DON'T YOU THAT THE SIX LITRE JUG IS EMPTY?!]]