Would you trust a recipe from ChatGPT?

My daughter really wanted to replicate the cookies she remembered from her childhood, the ones given to her when we shopped at Albertsons grocery store. So, she consulted ChatGPT.

I must add that both of my daughters love to bake and have made many treats for us over the years. It's not like they don't know how to bake. But one daughter was looking for a recipe for a specific cookie.

When she was asking ChatGPT about this cookie, she gave it feedback and tried to provide as much information as she could, questioning some advice and offering her own opinion here and there. When she thought she was finally given a good recipe, she headed out to the store to buy the special ingredients AI suggested. This recipe called for Crisco, butter, margarine, instant pudding, corn syrup, as well as eggs, flour, sugar, chocolate chips, salt, and baking soda. I'm not sure why ChatGPT thought the cookies needed pudding mix, corn syrup, or three types of fat.

She followed ChatGPT's recipe to the letter, using specific ingredients, refrigerating the dough, and doing everything ChatGPT called for. Here's how they turned out.

They spread a lot, were overdone on the edges, and were under-baked at the centers.

For reference, they should have looked something like this, from Albertsons' own website.

While my daughters have baked many batches of cookies, I must say that there are some things they just haven't thought through about recipes. When I saw how her batch turned out, I asked her for details from ChatGPT's recipe. I could tell right away that ChatGPT's recipe called for far too much sugar and fat, both ingredients that can cause cookies to spread. Like many women my age, I have many years of experience baking cookies and can kind of guess how a cookie will turn out based on the ratio of some of the ingredients. At the very least, I can think of "fixes" when a cookie doesn't turn out as I'd thought it would.

What I told my daughter was that when it comes to coming up with baking recipes, she's really better off consulting a seasoned baker instead of AI. AI might sometimes have a good recipe, but you can't count on it.

When AI was becoming a thing, about a year or so ago, I recall folks talking about it "hallucinating." That is, when AI didn't have an answer, instead of saying "I don't have the answer," it sometimes gave information based on nothing. I wonder how much of that was happening when my daughter asked for help coming up with a recipe for these cookies. I heard today that ChatGPT just released its version 5. Perhaps this improved version will address this issue of sometimes providing incorrect or not-quite-correct information. I will say that the recipe my daughter tried at least produced cookies and contained cookie ingredients. It just couldn't get a specific kind of cookie right. But my family still ate almost all of them. As my daughter said, "they're cookies. What's not to like. But they're not at all like the Albertsons' cookies. The taste and texture are way off." 

After the cookies were done baking my daughter went back to ChatGPT and explained the recipe it had given her and how the cookies looked and tasted. Funnily, ChatGPT told her the exact things I said, that excess sugar and fats caused the cookies to spread too much. Nice to know that when it comes to baking cookies, I'm as smart as AI.

So what kind of information is ChatGPT good for? I've used it for suggestions for movies for our family to watch. I can give parameters and ask it to recommend a movie list for me. It's good at making lists. However, it's up to me to then research every movie on the list to see if it would be right for our family. Like other lists or suggestions, I'd recommend researching anything AI says before acting on that information. It's a starting point, only, in my mind.

Anyway, my daughter will be sourcing recipes from a wider pool of information in the future.