Asklemmy

I didn't read this paper (I'll update this comment once I read it), but this sort of argument relying on emergent properties pops up from time to time, to claim that the bots are handling something further than tokens. It's weak for two reasons:

1. It boils down to appeal to ignorance - "we don't know, it's a blackbox system, so let's assume that some property (conceptualisation) is there, even if there are other ways to explain the phenomenon (output)".
2. Hallucinations themselves provide evidence against any sort of conceptualisation. Specially when the bot contradicts itself.

And note that the argument does not handle the lack of pragmatic purpose of the bot utterances. At all.

Specifically regarding games, what I think that is happening is that the bot is handling some logic based on the tokens themselves, in order to maximise the probability of a certain output (e.g. "you won"). That isn't even remotely surprising, and it doesn't require any sort of further abstraction to explain.

EDIT, from the paper:

If we think of a board as the “world,” then games provide us with an appealing experimental testbed to explore world representations of moderate complexity

This setting allows us to investigate world representations in a highly controlled context

Our next step is to look for world representations that might be used by the network

Othello makes a natural testbed for studying emergent world representations

To systematically determine if this world representation

Are you noticing the pattern? The researchers are taking for granted that the model will have some sort of "world representation", as an unspoken premise. With no h₀ like "no such thing".

And at the end of the day they proved that a chatbot can perform the same sort of logical operations that a "proper" game engine would.