That was a really excellent description. I work in the space in the engine underpinnings (ML compilers for custom AI chips) and that fleshed out what I had guessed but hadn’t had time to fully investigate.
Thank you for that very straightforward article. I'm glad I remember both my physics and chemistry. I think we need to take into account our inexorable and universal habit of anthropomorphizing everything that looks like us in a physical or relational way. This very act is what makes AI more dangerous. Take for example the AI companion avatar, robot, etc. The closer this looks like person to person interaction the more we will forget we are NOT talking to a person and the more we might possibly prefer this over real personal interactions. We could crash our population the minute we think the new Real Doll with AI is preferred over the messy relationship of actual skin on skin contact. Our AI needs to fall into the uncanny valley just long enough to remind us this is a tool, not a replacement for human interaction. We still follow the siren's song right onto the rocks of destruction.
I’d explained it in this way: It’s a neural network that learned to understand knowledge by reading a large part of the internet. It’s emergent behavior inside the neural net. It what happens in the brain of a baby. In the first months the eyes can see but the brains cannot. But the data will flow into the brain and due to the learning algorithm: it will start to understand the visual data over time. It’s emergent behavior. The net builds relationship to have a better estimate of the required output to minimize loss. Predicting the future requires intelligence
The rather fundamental problem with this "it's all just probabilities, and those words don't actually have any meaning to the model" take is that it can't explain this:
TL;DR: if you train a GPT model on a board game without explaining the rules (or even that it is a game) - just giving it valid moves and training to predict the next move - it turns out that this training process actually builds something inside the neural net that appears to be a representation of the game board; and twiddling bits in it actually changes what the model "thinks" the state of the game is. At this point, you could reasonably say that the trained model "understands" the game - what is understanding if not a mental model?
But if so, then why can't ChatGPT also have an internal model of the world based on its training data, above and beyond mere probabilities of words occurring next to each other? It would necessarily be a very simplified model, of course, since the real world is a lot more complex than Othello. But, however simple, it would still mean that "the model has a kind of internal, true/false orientation to the cat and different claims about its circumstances". And it would explain why it can actually perform tasks that require such modelling, which is something that requires a lot of "none of it is real, it just feels that way!" handwaving with a purely probabilistic approach.
I'm still trying to wrap my head around the difference between "training" the model, "fine-tuning" it, and then providing "RLHF". Are the differences just one of degree or point in the process?
That was a really excellent description. I work in the space in the engine underpinnings (ML compilers for custom AI chips) and that fleshed out what I had guessed but hadn’t had time to fully investigate.
The kids who all got trophys for last place now don't even want to write their own sentences. Is anyone really suprised?
Thank you for that very straightforward article. I'm glad I remember both my physics and chemistry. I think we need to take into account our inexorable and universal habit of anthropomorphizing everything that looks like us in a physical or relational way. This very act is what makes AI more dangerous. Take for example the AI companion avatar, robot, etc. The closer this looks like person to person interaction the more we will forget we are NOT talking to a person and the more we might possibly prefer this over real personal interactions. We could crash our population the minute we think the new Real Doll with AI is preferred over the messy relationship of actual skin on skin contact. Our AI needs to fall into the uncanny valley just long enough to remind us this is a tool, not a replacement for human interaction. We still follow the siren's song right onto the rocks of destruction.
I’d explained it in this way: It’s a neural network that learned to understand knowledge by reading a large part of the internet. It’s emergent behavior inside the neural net. It what happens in the brain of a baby. In the first months the eyes can see but the brains cannot. But the data will flow into the brain and due to the learning algorithm: it will start to understand the visual data over time. It’s emergent behavior. The net builds relationship to have a better estimate of the required output to minimize loss. Predicting the future requires intelligence
The rather fundamental problem with this "it's all just probabilities, and those words don't actually have any meaning to the model" take is that it can't explain this:
https://thegradient.pub/othello/
TL;DR: if you train a GPT model on a board game without explaining the rules (or even that it is a game) - just giving it valid moves and training to predict the next move - it turns out that this training process actually builds something inside the neural net that appears to be a representation of the game board; and twiddling bits in it actually changes what the model "thinks" the state of the game is. At this point, you could reasonably say that the trained model "understands" the game - what is understanding if not a mental model?
But if so, then why can't ChatGPT also have an internal model of the world based on its training data, above and beyond mere probabilities of words occurring next to each other? It would necessarily be a very simplified model, of course, since the real world is a lot more complex than Othello. But, however simple, it would still mean that "the model has a kind of internal, true/false orientation to the cat and different claims about its circumstances". And it would explain why it can actually perform tasks that require such modelling, which is something that requires a lot of "none of it is real, it just feels that way!" handwaving with a purely probabilistic approach.
I'm still trying to wrap my head around the difference between "training" the model, "fine-tuning" it, and then providing "RLHF". Are the differences just one of degree or point in the process?