← All threads

Another Great Demonstration of how AI doesnt "research" brands on Reddit

6 of Michael's comments in this thread · View thread on Reddit ↗

u/michaelnovati replied ·
You asked a leading question and got a bad result. This is a skll issue. AI will be as lazy as it can be. I'm an engineer and same with code. If I say "why is my code so good?" it will be garbage. I need to ask "audit my code and create a report of pros and cons and validate each one".

u/WebLinkr wrote (the comment Michael replied to):

How is it any different from any other question?

u/michaelnovati replied ·
Your question implied it was already decided that SEO was the best. So AI is just going to be super lazy. Second, if you don't enable web search, AI can't can new facts beyond its training window.

u/WebLinkr wrote (the comment Michael replied to):

>Second, if you don't enable web search, AI can't can new facts beyond its training window. I dont understand - I'm saying it got this number from doing a web search

u/michaelnovati replied ·
Sorry, I see that. It sourced that from GummySearch. Does that source say the wrong number?

u/WebLinkr wrote (the comment Michael replied to):

So this isn't a leading question, right? https://preview.redd.it/jv5pemxt351h1.png?width=1317&format=png&auto=webp&s=01e3f551642e6e1a72113832e539da38097a5f0f

u/michaelnovati replied ·
Asking if it's good is still slightly leading yeah. I have to explicitly ask for something like this: "audit r/SEO and produce a report evaluating the subreddit from different angles" like any use of adjectives that have positive or negative connotation can bias the LLM.

u/michaelnovati wrote (the comment Michael replied to):

Asking if it's good is still slightly leading yeah. I have to explicitly ask for something like this: "audit r/SEO and produce a report evaluating the subreddit from different angles" like any use of adjectives that have positive or negative connotation can bias the LLM.

u/michaelnovati replied ·
I would then ask more specific questions from the exhaustive report, like "analyze the content quality", "anaylze the community health and structure", etc.. LLMs shouldn't be making subjective judgements. They should be robots that collect and synthesize information.

u/WebLinkr wrote (the comment Michael replied to):

How is that different though? I wasn't assessing its capability of answering the question - I was demonstrating that it doesn't go to Reddit and do something - it doesnt kick off a group of agents that explore Reddit and assess it. Here's a result - most of the pages "cited" ar

u/michaelnovati replied ·
Yeah that's a great point, a lot of those sites suck and are just SEO garbage manipulation stuff. I would then ask it after, to evaluate the reliability of the sources.