Linux server admin, MySQL/TSQL database admin, Python programmer, Linux gaming enthusiast and a forever GM.

  • 6 Posts
  • 306 Comments
Joined 1 year ago
cake
Cake day: June 8th, 2023

help-circle




  • I’m happy with the Oxford definition: “the ability to acquire and apply knowledge and skills”.

    LLMs don’t have knowledge as they don’t actually understand anything. They are algorithmic response generators that apply scores to tokens, and spit out the highest scoring token considering all previous tokens.

    If asked to answer 10*5, they can’t reason through the math. They can only recognize 10, * and 5 as tokens in the training data that is usually followed by the 50 token. Thus, 50 is the highest scoring token, and is the answer it will choose. Things get more interesting when you ask questions that aren’t in the training data. If it has nothing more direct to copy from, it will regurgitate a sequence of tokens that sounds as close as possible to something in the training data: thus a hallucination.