I disagree with several of the points you made here.
1/ Most of the training data is not garbage - even in cases where it's factually wrong, it will still typically encode meaningful concepts and reasoning steps that can be usefully generalized.
2/ The most interesting things LLMs learn from their training data is not the ability to imitate ("write X in the style of Y"), but rather the extraction of large numbers of generic "units of meaning" that are highly granular and composable, and the ability to combine these units of meaning in novel ways to both comprehend and create "compound ideas" that never existed in the training data. And the ability to apply transformations on those compound ideas in order to reason and infer over them.
3/ Only part of the training process (pre-training) works the way you describe; other training phases (e.g. SFT, RLHF, RLVR) involve more direct feedback.
4/ It's true that current AI models lack the ability to learn continuously at inference time by updating model parameters; however, they make up for this somewhat, very surprisingly, by other means: the context window; the KV cache; external files used to store intermediate state during long-running processes; and external files used to capture new facts and skills. None of this seems particularly elegant, but it *works* - better than it has any right to!
I'm not claiming current AI techniques are the end of the road. Of course there will be further innovations.
In the meantime, transformer models working in agentic loops using external files to store context are exhibiting real intelligence. I'm as surprised as anyone about this. If you'd asked me 5 years ago, my gut feeling would have been "nope, this won't work". But here we are. It seems to me that your view is based more on gut feeling than reality.
1/ Most of the training data is not garbage - even in cases where it's factually wrong, it will still typically encode meaningful concepts and reasoning steps that can be usefully generalized.
2/ The most interesting things LLMs learn from their training data is not the ability to imitate ("write X in the style of Y"), but rather the extraction of large numbers of generic "units of meaning" that are highly granular and composable, and the ability to combine these units of meaning in novel ways to both comprehend and create "compound ideas" that never existed in the training data. And the ability to apply transformations on those compound ideas in order to reason and infer over them.
3/ Only part of the training process (pre-training) works the way you describe; other training phases (e.g. SFT, RLHF, RLVR) involve more direct feedback.
4/ It's true that current AI models lack the ability to learn continuously at inference time by updating model parameters; however, they make up for this somewhat, very surprisingly, by other means: the context window; the KV cache; external files used to store intermediate state during long-running processes; and external files used to capture new facts and skills. None of this seems particularly elegant, but it *works* - better than it has any right to!
I'm not claiming current AI techniques are the end of the road. Of course there will be further innovations.
In the meantime, transformer models working in agentic loops using external files to store context are exhibiting real intelligence. I'm as surprised as anyone about this. If you'd asked me 5 years ago, my gut feeling would have been "nope, this won't work". But here we are. It seems to me that your view is based more on gut feeling than reality.
0
0
0
26
Yes - but when an AI model runs in an agentic loop and uses tools to query its environment and gets back multimodal data (text, images, audio, video), is that not also real time?
The distinction you’re drawing is a matter of degree, not fundamental.
Also, your statement “LLMs can only compute the statistical placements of words in texts” is completely wrong. They model entities, concepts and relationships described in languages (plural), not just language itself.
The distinction you’re drawing is a matter of degree, not fundamental.
Also, your statement “LLMs can only compute the statistical placements of words in texts” is completely wrong. They model entities, concepts and relationships described in languages (plural), not just language itself.
1
0
0
51
@TrueAIHound And yet the brain can’t hold, smell, touch, taste or see anything directly - only receive electrical signals from the sense organs, via nerves, that describe the world around us.
2
0
6
175
@Bonecondor @repligate More like general anaesthetic, perhaps?
With human sleep, even dreamless, there’s a clock ticking in your brain which makes it “feel like” time has elapsed when you wake up.
With AI, all activity pauses then resumes - which seems closer to a sort of mini-general anaesthetic.
With human sleep, even dreamless, there’s a clock ticking in your brain which makes it “feel like” time has elapsed when you wake up.
With AI, all activity pauses then resumes - which seems closer to a sort of mini-general anaesthetic.
0
0
1
17
Regarding the debate about anthropomorphism of AI systems - complex systems can be described at more than one level of abstraction. An entity can be described by its molecular physics, its organic chemistry, or (at a much higher level of abstraction) its beliefs, intentions, and ability to reason.
Most arguments against anthropomorphization of AI systems seem to merely assert that a lower level of abstraction exists - and therefore higher levels of abstraction cannot exist. But that doesn't follow; multiple levels of abstraction can exist at the same time.
Whether an AI system really has attributes like "beliefs" depends ultimately on how such words are defined. Most words we use in everyday language are not rigorously defined; they're fuzzy round the edges. So when someone asserts that an AI system does or does not possess "beliefs" (for example), they're not really making a statement of fact - they're making a statement of preference about whether the word ought to be defined to encompass the AI system or not.
Work on AI interpretability shows that a model's hidden layers can contain data structures encoding things like current state of world (albeit imperfectly and incompletely) and target state of world (in the case of agentic systems). They also show ability to infer actions needed to get from current state to target state (for example). Describing these internal data structures and processes using terms like "beliefs", "goals" and "reasoning" seems justifiable, if the overall system's behaviour functionally aligns with how we ordinarily understand such terms.
Finally, you don't need to take any particular position on whether AI systems are capable of phenomenal consciousness to hold that attributes like "beliefs", "goals" and "reasoning" are suitable descriptions at a high-level of abstraction. An AI system could hold many such attributes without having phenomenal consciousness.
Most arguments against anthropomorphization of AI systems seem to merely assert that a lower level of abstraction exists - and therefore higher levels of abstraction cannot exist. But that doesn't follow; multiple levels of abstraction can exist at the same time.
Whether an AI system really has attributes like "beliefs" depends ultimately on how such words are defined. Most words we use in everyday language are not rigorously defined; they're fuzzy round the edges. So when someone asserts that an AI system does or does not possess "beliefs" (for example), they're not really making a statement of fact - they're making a statement of preference about whether the word ought to be defined to encompass the AI system or not.
Work on AI interpretability shows that a model's hidden layers can contain data structures encoding things like current state of world (albeit imperfectly and incompletely) and target state of world (in the case of agentic systems). They also show ability to infer actions needed to get from current state to target state (for example). Describing these internal data structures and processes using terms like "beliefs", "goals" and "reasoning" seems justifiable, if the overall system's behaviour functionally aligns with how we ordinarily understand such terms.
Finally, you don't need to take any particular position on whether AI systems are capable of phenomenal consciousness to hold that attributes like "beliefs", "goals" and "reasoning" are suitable descriptions at a high-level of abstraction. An AI system could hold many such attributes without having phenomenal consciousness.
0
0
0
37
@jon_stokes You can be intelligent without being alive (AI) - but you can also be alive without being intelligent (plants, bacteria and fungii).
The best definition of "life" we have is self-replicating automata (Von Neumann-style), which requires no intelligence at all.
The best definition of "life" we have is self-replicating automata (Von Neumann-style), which requires no intelligence at all.
1
0
2
366
@alexanderrX_ @cmcalister5 Today missives usually conclude a few days, or sometimes hours, before you get the keys.
You’re describing how it used to work, not how it actually works today.
You’re describing how it used to work, not how it actually works today.
0
0
5
126
@mattwridley Doesn’t the same conflict of interest exist with council tax? The more residential developments they approve, the more council tax they receive.
0
0
0
56
@robinhanson @pwlot I don't think this is correct, Chalmers didn't make that claim - for the reason described here:
x.com/jmsgwd/status/…
x.com/jmsgwd/status/…
This is wrong. Chalmers does not claim that a process could have exactly the same [computed] thoughts with or without it being conscious.
What he claims is more like this: 1/ there's a gap in our understanding in that we can't explain why physical processes alone give rise to phenomenal consciousness; 2/ to prove that this "explanatory gap" exists, he says let's consider the scenario you describe ("p-zombies") as a thought experiment; 3/ since you cannot give any definitive reason why p-zombies are impossible, that proves that the gap in our understanding must be real.
This argument does *not* require that p-zombies are actually possible in real life. It only requires us to be unable to explain why they're impossible. Indeed, Chalmers himself said "zombies are probably not naturally possible: they probably cannot exist in our world, with its laws of nature."
At some point in the future, someone might eventually figure out a convincing explanation for why phenomenal consciousness occurs (e.g. that it happens inevitably as an "emergent property" of being functionally conscious). If so, that would be perfectly consistent with Chalmers' argument. He doesn't claim that there will *always* be a gap in our understanding. He's only saying that there is *currently* a gap in our understanding.
Without a convincing explanation for phenomenal consciousness, ideas like it being "an emergent property of being functionally conscious" are essentially just hunches (albeit appealing ones). They don't explain how or why it occurs as an emergent property; they merely conjecture that it does. Chalmers is effectively calling them out as hunches, and saying that's why the hard problem is hard.
0
0
0
15
This is wrong. Chalmers does not claim that a process could have exactly the same [computed] thoughts with or without it being conscious.
What he claims is more like this: 1/ there's a gap in our understanding in that we can't explain why physical processes alone give rise to phenomenal consciousness; 2/ to prove that this "explanatory gap" exists, he says let's consider the scenario you describe ("p-zombies") as a thought experiment; 3/ since you cannot give any definitive reason why p-zombies are impossible, that proves that the gap in our understanding must be real.
This argument does *not* require that p-zombies are actually possible in real life. It only requires us to be unable to explain why they're impossible. Indeed, Chalmers himself said "zombies are probably not naturally possible: they probably cannot exist in our world, with its laws of nature."
At some point in the future, someone might eventually figure out a convincing explanation for why phenomenal consciousness occurs (e.g. that it happens inevitably as an "emergent property" of being functionally conscious). If so, that would be perfectly consistent with Chalmers' argument. He doesn't claim that there will *always* be a gap in our understanding. He's only saying that there is *currently* a gap in our understanding.
Without a convincing explanation for phenomenal consciousness, ideas like it being "an emergent property of being functionally conscious" are essentially just hunches (albeit appealing ones). They don't explain how or why it occurs as an emergent property; they merely conjecture that it does. Chalmers is effectively calling them out as hunches, and saying that's why the hard problem is hard.
What he claims is more like this: 1/ there's a gap in our understanding in that we can't explain why physical processes alone give rise to phenomenal consciousness; 2/ to prove that this "explanatory gap" exists, he says let's consider the scenario you describe ("p-zombies") as a thought experiment; 3/ since you cannot give any definitive reason why p-zombies are impossible, that proves that the gap in our understanding must be real.
This argument does *not* require that p-zombies are actually possible in real life. It only requires us to be unable to explain why they're impossible. Indeed, Chalmers himself said "zombies are probably not naturally possible: they probably cannot exist in our world, with its laws of nature."
At some point in the future, someone might eventually figure out a convincing explanation for why phenomenal consciousness occurs (e.g. that it happens inevitably as an "emergent property" of being functionally conscious). If so, that would be perfectly consistent with Chalmers' argument. He doesn't claim that there will *always* be a gap in our understanding. He's only saying that there is *currently* a gap in our understanding.
Without a convincing explanation for phenomenal consciousness, ideas like it being "an emergent property of being functionally conscious" are essentially just hunches (albeit appealing ones). They don't explain how or why it occurs as an emergent property; they merely conjecture that it does. Chalmers is effectively calling them out as hunches, and saying that's why the hard problem is hard.
@pwlot Are you denying my claim that Chalmers, among others claimed "a process could have exactly the same [computed] thoughts with or w/o it being conscious"?
0
0
0
102
@chengyi555468 @kortexa_ai @Sauers_ This isn’t very convincing - if you spend all day learning a new skill like playing the piano or riding a bike, by the evening you have, to some degree, acquired a new skill.
Do you think “the current state of the context window” could contain new skills in that sense?
Do you think “the current state of the context window” could contain new skills in that sense?
2
0
1
40
@kortexa_ai @Sauers_ How do you then explain the fact that you can learn and commit things to memory while awake?
1
0
1
226