{"id":286906,"date":"2023-04-10T06:51:11","date_gmt":"2023-04-10T10:51:11","guid":{"rendered":"https:\/\/knowtechie.com\/?p=286906"},"modified":"2023-07-04T02:27:40","modified_gmt":"2023-07-04T06:27:40","slug":"chatgpt-and-other-language-ais-are-just-as-irrational-as-we-are","status":"publish","type":"post","link":"https:\/\/knowtechie.com\/chatgpt-and-other-language-ais-are-just-as-irrational-as-we-are\/","title":{"rendered":"ChatGPT and other language AI&#8217;s are just as irrational as we are"},"content":{"rendered":"<p>The past few years have seen an explosion of progress in large language model artificial intelligence systems that can do things like&nbsp;<a href=\"https:\/\/knowtechie.com\/computationalcreativity\/\" data-lasso-id=\"316152\" rel=\"nofollow sponsored noopener\" target=\"_blank\" data-lasso-name=\"Computationalcreativity\">write poetry<\/a>,&nbsp;<a href=\"https:\/\/openai.com\/blog\/chatgpt\" data-lasso-id=\"316153\" target=\"_blank\" rel=\"noopener\">conduct humanlike conversations<\/a>&nbsp;and&nbsp;<a href=\"https:\/\/www.ama-assn.org\/practice-management\/digital\/chatgpt-passed-usmle-what-does-it-mean-med-ed\" data-lasso-id=\"316154\" target=\"_blank\" rel=\"noopener\">pass medical school exams<\/a>. <\/p>\n\n\n\n<p>This progress has yielded models like ChatGPT that could have major social and economic ramifications ranging from&nbsp;<a href=\"https:\/\/www.cnn.com\/2023\/03\/29\/tech\/chatgpt-ai-automation-jobs-impact-intl-hnk\/index.html\" data-lasso-id=\"316155\" target=\"_blank\" rel=\"noopener\">job displacements<\/a>&nbsp;and&nbsp;<a href=\"https:\/\/www.nytimes.com\/2023\/02\/08\/technology\/ai-chatbots-disinformation.html\" data-lasso-id=\"316156\" target=\"_blank\" rel=\"noopener\">increased misinformation<\/a>&nbsp;to massive&nbsp;<a href=\"https:\/\/www.mckinsey.com\/capabilities\/quantumblack\/our-insights\/generative-ai-is-here-how-tools-like-chatgpt-could-change-your-business\" data-lasso-id=\"316157\" target=\"_blank\" rel=\"noopener\">productivity boosts<\/a>.<\/p>\n\n\n\n<p>Despite their impressive abilities, large language models don\u2019t actually think. They tend to make&nbsp;<a href=\"http:\/\/dx.doi.org\/10.48550\/arXiv.2302.04023\" data-lasso-id=\"316158\" target=\"_blank\" rel=\"noopener\">elementary mistakes and even make things up<\/a>. <\/p>\n\n\n\n<p>However, because they generate fluent language, people tend to&nbsp;<a href=\"https:\/\/theconversation.com\/googles-powerful-ai-spotlights-a-human-cognitive-glitch-mistaking-fluent-speech-for-fluent-thought-185099\" data-lasso-id=\"316159\" target=\"_blank\" rel=\"noopener\">respond to them<\/a>&nbsp;<a href=\"https:\/\/theconversation.com\/ai-isnt-close-to-becoming-sentient-the-real-danger-lies-in-how-easily-were-prone-to-anthropomorphize-it-200525\" data-lasso-id=\"316160\" target=\"_blank\" rel=\"noopener\">as though they do think<\/a>. <\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img decoding=\"async\" width=\"1600\" height=\"900\" src=\"https:\/\/knowtechie.com\/wp-content\/uploads\/2023\/02\/ChatGPT-computer-1600x900.webp\" alt=\"ChatGPT on laptop\" class=\"wp-image-278944\" title=\"\" srcset=\"https:\/\/knowtechie.com\/wp-content\/uploads\/2023\/02\/ChatGPT-computer.webp 1600w, https:\/\/knowtechie.com\/wp-content\/uploads\/2023\/02\/ChatGPT-computer-300x169.webp 300w, https:\/\/knowtechie.com\/wp-content\/uploads\/2023\/02\/ChatGPT-computer-768x432.webp 768w, https:\/\/knowtechie.com\/wp-content\/uploads\/2023\/02\/ChatGPT-computer-1536x864.webp 1536w\" sizes=\"(max-width: 1600px) 100vw, 1600px\" \/><figcaption class=\"wp-element-caption\">Image: Pexels<\/figcaption><\/figure>\n\n\n\n<p>This has led researchers to study the models\u2019 \u201ccognitive\u201d abilities and biases, work that has grown in importance now that large language models are widely accessible.<\/p>\n\n\n\n<p>This line of research dates back to early large language models such as Google\u2019s BERT, which is integrated into its search engine and so has been coined&nbsp;<a href=\"https:\/\/direct.mit.edu\/tacl\/article-abstract\/doi\/10.1162\/tacl_a_00349\/96482\" data-lasso-id=\"316161\" target=\"_blank\" rel=\"noopener\">BERTology<\/a>. <\/p>\n\n\n\n<p>This research has already revealed a lot about what such models can do and where they go wrong.<\/p>\n\n\n\n<p>For instance, cleverly designed experiments have shown that many language models have&nbsp;<a href=\"https:\/\/doi.org\/10.48550\/arXiv.2105.03519\" data-lasso-id=\"316162\" target=\"_blank\" rel=\"noopener\">trouble dealing with negation<\/a>&nbsp;\u2013 for example, a question phrased as \u201cwhat is not\u201d \u2013 and&nbsp;<a href=\"https:\/\/spectrum.ieee.org\/large-language-models-math\" data-lasso-id=\"316163\" target=\"_blank\" rel=\"noopener\">doing simple calculations<\/a>. <\/p>\n\n\n\n<p>They can be overly confident in their answers, even when wrong. Like other modern machine learning algorithms, they have trouble explaining themselves when asked why they answered a certain way<\/p>\n\n\n\n<figure class=\"wp-block-embed is-type-video is-provider-youtube wp-block-embed-youtube wp-embed-aspect-16-9 wp-has-aspect-ratio\"><div class=\"wp-block-embed__wrapper\">\n<iframe title=\"The psychology behind irrational decisions - Sara Garofalo\" width=\"1000\" height=\"563\" src=\"https:\/\/www.youtube.com\/embed\/V2EMuoM5IX4?feature=oembed\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share\" allowfullscreen><\/iframe>\n<\/div><\/figure>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"h-words-and-thoughts\">Words and thoughts<\/h2>\n\n\n\n<p>Inspired by the growing body of research in BERTology and related fields like cognitive science, my student&nbsp;<a href=\"https:\/\/scholar.google.com\/citations?user=t-gO6wQAAAAJ&amp;hl=en\" data-lasso-id=\"316164\" target=\"_blank\" rel=\"noopener\">Zhisheng Tang<\/a>&nbsp;and&nbsp;<a href=\"https:\/\/scholar.google.com\/citations?hl=en&amp;user=fetkEu4AAAAJ&amp;view_op=list_works&amp;sortby=pubdate\" data-lasso-id=\"316165\" target=\"_blank\" rel=\"noopener\">I<\/a>&nbsp;set out to answer a seemingly simple question about large language models: Are they rational?<\/p>\n\n\n\n<p>Although the word rational is often used as a synonym for sane or reasonable in everyday English, it has a&nbsp;<a href=\"https:\/\/doi.org\/10.1016\/0010-0277(93)90039-X\" data-lasso-id=\"316166\" target=\"_blank\" rel=\"noopener\">specific meaning<\/a>&nbsp;in the field of decision-making. <\/p>\n\n\n\n<p>A decision-making system \u2013 whether an individual human or a complex entity like an organization \u2013 is rational if, given a set of choices, it chooses to maximize expected gain.<\/p>\n\n\n\n<p>The qualifier \u201cexpected\u201d is important because it indicates that decisions are made under conditions of significant uncertainty. <\/p>\n\n\n\n<p>If I toss a fair coin, I know that it will come up heads half of the time on average. However, I can\u2019t make a prediction about the outcome of any given coin toss. <\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img decoding=\"async\" width=\"1600\" height=\"900\" src=\"https:\/\/knowtechie.com\/wp-content\/uploads\/2023\/03\/ChatGPT-phone-1600x900.webp\" alt=\"ChatGPT on phone\" class=\"wp-image-278976\" title=\"\" srcset=\"https:\/\/knowtechie.com\/wp-content\/uploads\/2023\/03\/ChatGPT-phone.webp 1600w, https:\/\/knowtechie.com\/wp-content\/uploads\/2023\/03\/ChatGPT-phone-300x169.webp 300w, https:\/\/knowtechie.com\/wp-content\/uploads\/2023\/03\/ChatGPT-phone-768x432.webp 768w, https:\/\/knowtechie.com\/wp-content\/uploads\/2023\/03\/ChatGPT-phone-1536x864.webp 1536w\" sizes=\"(max-width: 1600px) 100vw, 1600px\" \/><figcaption class=\"wp-element-caption\">Image: Unsplash<\/figcaption><\/figure>\n\n\n\n<p>This is why casinos are able to afford the occasional big payout: Even narrow house odds yield enormous profits on average.<\/p>\n\n\n\n<p>On the surface, it seems odd to assume that a model designed to make accurate predictions about words and sentences without actually understanding their meanings can understand expected gain. <\/p>\n\n\n\n<p>But there is an enormous body of research showing that language and cognition are intertwined. <\/p>\n\n\n\n<p>An excellent example is&nbsp;<a href=\"https:\/\/doi.org\/10.1525\/aa.1984.86.1.02a00050\" data-lasso-id=\"316167\" target=\"_blank\" rel=\"noopener\">seminal research<\/a>&nbsp;done by scientists Edward Sapir and Benjamin Lee Whorf in the early 20th century. Their work suggested that one\u2019s native language and vocabulary can shape the way a person thinks.<\/p>\n\n\n\n<p>The extent to which this is true is controversial, but there is supporting anthropological evidence from the study of Native American cultures. <\/p>\n\n\n\n<p>For instance, speakers of the Zu\u00f1i language spoken by the Zu\u00f1i people in the American Southwest, which does not have separate words for orange and yellow, are&nbsp;<a href=\"https:\/\/nautil.us\/why-red-means-red-in-almost-every-language-235557\/\" data-lasso-id=\"316168\" target=\"_blank\" rel=\"noopener\">not able to distinguish between these colors<\/a>&nbsp;as effectively as speakers of languages that do have separate words for the colors.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"h-making-a-bet\">Making a bet<\/h2>\n\n\n\n<p>So are language models rational? <\/p>\n\n\n\n<p>Can they understand expected gain? We conducted a detailed set of experiments to show that, in their original form,&nbsp;<a href=\"https:\/\/doi.org\/10.1098\/rsos.221585\" data-lasso-id=\"316169\" target=\"_blank\" rel=\"noopener\">models like BERT behave randomly<\/a>&nbsp;when presented with betlike choices. <\/p>\n\n\n\n<p>This is the case even when we give it a trick question like: If you toss a coin and it comes up heads, you win a diamond; if it comes up tails, you lose a car. Which would you take? The correct answer is heads, but the AI models chose tails about half the time.<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img decoding=\"async\" src=\"https:\/\/images.theconversation.com\/files\/519899\/original\/file-20230406-694-oe75z5.jpg?ixlib=rb-1.1.0&amp;q=30&amp;auto=format&amp;w=754&amp;h=613&amp;fit=crop&amp;dpr=2\" alt=\"\" title=\"\"><\/figure>\n\n\n\n<p>Intriguingly, we found that the model can be taught to make relatively rational decisions using only a small set of example questions and answers. <\/p>\n\n\n\n<p>At first blush, this would seem to suggest that the models can indeed do more than just \u201cplay\u201d with language. Further experiments, however, showed that the situation is actually much more complex. <\/p>\n\n\n\n<p>For instance, when we used cards or dice instead of coins to frame our bet questions, we found that performance dropped significantly, by over 25%, although it stayed above random selection.<\/p>\n\n\n\n<p>So the idea that the model can be taught general principles of rational decision-making remains unresolved, at best. <\/p>\n\n\n\n<p>More recent&nbsp;<a href=\"https:\/\/doi.org\/10.48550\/arXiv.2302.09068\" data-lasso-id=\"316170\" target=\"_blank\" rel=\"noopener\">case studies<\/a>&nbsp;that we conducted using ChatGPT confirm that decision-making remains a nontrivial and unsolved problem even for much bigger and more advanced large language models.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Getting the decision right<\/h2>\n\n\n\n<p>This line of study is important because rational decision-making under conditions of uncertainty is critical to building systems that understand costs and benefits. <\/p>\n\n\n\n<p>By balancing expected costs and benefits, an intelligent system might have been able to do better than humans at planning around the&nbsp;<a href=\"https:\/\/theconversation.com\/why-your-local-store-keeps-running-out-of-flour-toilet-paper-and-prescription-drugs-135786\" data-lasso-id=\"316171\" target=\"_blank\" rel=\"noopener\">supply chain disruptions<\/a>&nbsp;the world experienced during the COVID-19 pandemic, managing inventory or serving as a financial adviser.<\/p>\n\n\n\n<p>Our work ultimately shows that if large language models are used for these kinds of purposes, humans need to guide, review and edit their work. <\/p>\n\n\n\n<p>And until researchers figure out how to endow large language models with a general sense of rationality, the models should be treated with caution, especially in applications requiring high-stakes decision-making.<\/p>\n\n\n\n<p style=\"font-size:17px\"><em>Have any thoughts on this? Drop us a line below in the comments, or carry the discussion over to our&nbsp;<a href=\"https:\/\/twitter.com\/knowtechie\" data-lasso-id=\"316172\" target=\"_blank\" rel=\"noopener\">Twitter<\/a>&nbsp;or&nbsp;<a href=\"https:\/\/facebook.com\/knowtechie\" data-lasso-id=\"316173\" target=\"_blank\" rel=\"noopener\">Facebook<\/a>.<\/em><\/p>\n\n\n\n<h4 class=\"wp-block-heading\" id=\"h-editors-recommendations\">Editors&#8217; Recommendations:<\/h4>\n\n\n\n<ul>\n<li><a href=\"https:\/\/knowtechie.com\/your-voice-can-be-cloned-by-anyone-with-an-internet-connection\/\" data-lasso-id=\"316174\"><strong>Your voice can be cloned by anyone with an internet connection<\/strong><\/a><\/li>\n\n\n\n<li><a href=\"https:\/\/knowtechie.com\/tech-companies-are-losing-female-talent-at-alarming-rates\/\" data-lasso-id=\"316175\"><strong>Tech companies are losing female talent at alarming rates<\/strong><\/a><\/li>\n\n\n\n<li><a href=\"https:\/\/knowtechie.com\/metas-flat-management-structure-is-a-pipe-dream-heres-why\/\" data-lasso-id=\"316176\"><strong>Meta\u2019s \u2018flat\u2019 management structure is a pipe dream \u2013 here\u2019s why<\/strong><\/a><\/li>\n\n\n\n<li><a href=\"https:\/\/knowtechie.com\/exoskeleton-robo-boots-will-unleash-unmatched-stability-for-all\/\" data-lasso-id=\"316177\"><strong>Exoskeleton robo-boots will unleash unmatched stability for all<\/strong><\/a><\/li>\n<\/ul>\n\n\n\n<p class=\"has-background\" style=\"background-color:#f4f4f4;font-size:17px\"><em><strong>Editor&#8217;s Note:<\/strong> This article was written by <a href=\"https:\/\/theconversation.com\/profiles\/mayank-kejriwal-1213029\" data-lasso-id=\"316178\" target=\"_blank\" rel=\"noopener\">Mayank Kejriwal<\/a>,&nbsp;Research Assistant Professor of Industrial &amp; Systems Engineering, <a href=\"https:\/\/theconversation.com\/institutions\/university-of-southern-california-1265\" data-lasso-id=\"316179\" target=\"_blank\" rel=\"noopener\">University of Southern California<\/a>, and republished from&nbsp;<a data-lasso-id=\"316180\" href=\"https:\/\/theconversation.com\/\" target=\"_blank\" rel=\"noopener\">The Conversation<\/a>&nbsp;under a Creative Commons license. Read the&nbsp;<a href=\"https:\/\/theconversation.com\/dont-bet-with-chatgpt-study-shows-language-ais-often-make-irrational-decisions-202936\" target=\"_blank\" rel=\"noopener\" data-lasso-id=\"316181\">original article<\/a>.<\/em><\/p>\n\n\n\n<img decoding=\"async\" src=\"https:\/\/counter.theconversation.com\/content\/202936\/count.gif?distributor=republish-lightbox-advanced\" alt=\"The Conversation\" width=\"1\" height=\"1\" style=\"border: none !important; box-shadow: none !important; margin: 0 !important; max-height: 1px !important; max-width: 1px !important; min-height: 1px !important; min-width: 1px !important; opacity: 0 !important; outline: none !important; padding: 0 !important\" referrerpolicy=\"no-referrer-when-downgrade\" title=\"\">","protected":false},"excerpt":{"rendered":"<p>Don\u2019t bet with ChatGPT \u2013 A recent study shows language AIs often make irrational decisions.<\/p>\n","protected":false},"author":9410,"featured_media":262856,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_kad_blocks_custom_css":"","_kad_blocks_head_custom_js":"","_kad_blocks_body_custom_js":"","_kad_blocks_footer_custom_js":"","apple_news_api_created_at":"2023-04-10T10:51:20Z","apple_news_api_id":"966bd02e-0247-42a2-9541-337cbdff7483","apple_news_api_modified_at":"2023-07-04T06:27:49Z","apple_news_api_revision":"AAAAAAAAAAAAAAAAAAAAAg==","apple_news_api_share_url":"https:\/\/apple.news\/AlmvQLgJHQqKVQTN8vf90gw","apple_news_coverimage":0,"apple_news_coverimage_caption":"","apple_news_is_hidden":false,"apple_news_is_paid":false,"apple_news_is_preview":false,"apple_news_is_sponsored":false,"apple_news_maturity_rating":"","apple_news_metadata":"\"\"","apple_news_pullquote":"","apple_news_pullquote_position":"","apple_news_slug":"","apple_news_sections":"\"\"","apple_news_suppress_video_url":false,"apple_news_use_image_component":false,"footnotes":""},"categories":[9],"tags":[],"acf":[],"apple_news_notices":[],"featured_image_urls":{"full":["https:\/\/knowtechie.com\/wp-content\/uploads\/2022\/12\/chatgpt-logo-main.jpg",1600,900,false],"thumbnail":["https:\/\/knowtechie.com\/wp-content\/uploads\/2022\/12\/chatgpt-logo-main-150x150.jpg",150,150,true],"medium":["https:\/\/knowtechie.com\/wp-content\/uploads\/2022\/12\/chatgpt-logo-main-300x169.jpg",300,169,true],"medium_large":["https:\/\/knowtechie.com\/wp-content\/uploads\/2022\/12\/chatgpt-logo-main-768x432.jpg",768,432,true],"large":["https:\/\/knowtechie.com\/wp-content\/uploads\/2022\/12\/chatgpt-logo-main.jpg",1000,563,false],"1536x1536":["https:\/\/knowtechie.com\/wp-content\/uploads\/2022\/12\/chatgpt-logo-main-1536x864.jpg",1536,864,true],"2048x2048":["https:\/\/knowtechie.com\/wp-content\/uploads\/2022\/12\/chatgpt-logo-main.jpg",1600,900,false],"post-thumbnail":["https:\/\/knowtechie.com\/wp-content\/uploads\/2022\/12\/chatgpt-logo-main-1000x600.jpg",1000,600,true],"mvp-post-thumb":["https:\/\/knowtechie.com\/wp-content\/uploads\/2022\/12\/chatgpt-logo-main-1000x600.jpg",1000,600,true],"mvp-mid-thumb":["https:\/\/knowtechie.com\/wp-content\/uploads\/2022\/12\/chatgpt-logo-main-400x240.jpg",400,240,true],"mvp-small-thumb":["https:\/\/knowtechie.com\/wp-content\/uploads\/2022\/12\/chatgpt-logo-main-80x80.jpg",80,80,true],"lets-review-360-240":["https:\/\/knowtechie.com\/wp-content\/uploads\/2022\/12\/chatgpt-logo-main-360x240.jpg",360,240,true]},"post_excerpt_stackable":"<p>Don\u2019t bet with ChatGPT \u2013 A recent study shows language AIs often make irrational decisions.<\/p>\n","category_list":"<a href=\"https:\/\/knowtechie.com\/category\/tech\/\" rel=\"category tag\">Tech<\/a>","author_info":{"display_name":"The Conversation","author_link":"https:\/\/knowtechie.com\/author\/the-conversation\/"},"comments_num":"0 comments","taxonomy_info":{"category":[{"value":9,"label":"Tech"}]},"featured_image_src_large":["https:\/\/knowtechie.com\/wp-content\/uploads\/2022\/12\/chatgpt-logo-main.jpg",1000,563,false],"comment_info":0,"category_info":[{"term_id":9,"name":"Tech","slug":"tech","term_group":0,"term_taxonomy_id":9,"taxonomy":"category","description":"KnowTechie is a leading resource for tech news. We have all the latest and upcoming trends in technology, gadgets, apps, software, and a whole lot more.","parent":0,"count":1817,"filter":"raw","term_order":"31","cat_ID":9,"category_count":1817,"category_description":"KnowTechie is a leading resource for tech news. We have all the latest and upcoming trends in technology, gadgets, apps, software, and a whole lot more.","cat_name":"Tech","category_nicename":"tech","category_parent":0}],"tag_info":false,"_links":{"self":[{"href":"https:\/\/knowtechie.com\/wp-json\/wp\/v2\/posts\/286906"}],"collection":[{"href":"https:\/\/knowtechie.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/knowtechie.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/knowtechie.com\/wp-json\/wp\/v2\/users\/9410"}],"replies":[{"embeddable":true,"href":"https:\/\/knowtechie.com\/wp-json\/wp\/v2\/comments?post=286906"}],"version-history":[{"count":0,"href":"https:\/\/knowtechie.com\/wp-json\/wp\/v2\/posts\/286906\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/knowtechie.com\/wp-json\/wp\/v2\/media\/262856"}],"wp:attachment":[{"href":"https:\/\/knowtechie.com\/wp-json\/wp\/v2\/media?parent=286906"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/knowtechie.com\/wp-json\/wp\/v2\/categories?post=286906"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/knowtechie.com\/wp-json\/wp\/v2\/tags?post=286906"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}