[TowerTalk] ChatGPT

[David Gilbert]

Yup.

73,
Dave  AB7E


On 4/5/2023 4:20 PM, Christian Kennedy wrote:
>
> It's probably useful to recall what the "GPT" in ChatGPT stands for -- 
> Generative Pretrained Transformer; GPTs are a particularly effective  
> form of long-term memory that's useful for making predictions 
> regarding patterns.  ChatGPT has zero semantic knowledge; in essence 
> it's a text prediction engine that's been trained on a massive corpus 
> of text (apparently including code scraped off of github); what it 
> pukes out in response to your input text is, literally the 
> statistically most probable outcome based on the sequence of tokens 
> that it produced by shredding your input text.  Of course if that's 
> all it did the results would be both dry and deterministic, so GPTs 
> incorporate the notion of "temperature"; at any given point in 
> constructing an output string it has a list of candidate words based 
> on what it's already output, ranked by probability, but then it 
> perturbs the probability so it doesn't always produce the same text.  
> Because each output token affects the probability of what comes next, 
> the result is different (occasionally very different) output strings 
> for the same input.
>
> GPTs can be entertaining, and for narrow domains can be useful, but as 
> autocomplete systems with zero understanding of what they're spewing, 
> they can also be thought of as incredible bullshit generators.  Worse, 
> their predictions become part of their corpus, and competing GPT 
> systems have started scraping each other, resulting in a sort of 
> exponential bullshit amplification phenomena that has required some 
> tamping down.
>
> My suggesting is that you use it to get into the game, but always go 
> back to primary sources and do your own work.
>