Love that you decided to challenge yourself to write every week and started a substack. I enjoy your weekly articles here, this one included. One question I had about your recent posts of AI messaging is, do you think there is a message that will overcome the publics skepticism of AI? Seems like the more people use it, the more they see that it can do things they do and leads to the fear of job loss. Not saying that Dario, Sam et all are doing good messaging, just wonder if you think other messaging would make a difference. Thanks for writing.
The Cognitive Density part really hits home. My day flies by and my brain feels like it got hit by a truck.
Wondering to hear your thoughts, but what I have experienced is my margin of error has become extremely slim, while the range of outcomes has exploded over the last 16 months. A few bad decisions along the way compounds very fast and in the back of my mind I know that AI is sitting out there to take my job one day.
Good article but I dislike the phrasing “cost of intelligence to zero” - this is straight from foundation lab PR kits. What’s really happening IMO is what you mentioned - the menial tasks are automated because the LLM has seen them in training data millions of times. But LLMs are not intelligent, and when they are treated as such (eg by people hoping to outsource decisions and critical thinking), that’s when issues arise.
I think you've mistaken and misattributed the pressure being felt for something it isn't.
At a fairly low (almost unconscious level) people psychologically have blind spots that are accessible through communication as a medium. Used car salespeople use these, so do many other entities. This has been established by experts and is covered in a number of books and research papers dating back to WW2, and increasing in sophistication up from the Korean Conflict (Mao), Stasi (Zersetzung), to modern times (in interrogation and other areas). Chase Hughes has been aggregating most of the material into a single place because it straddles several subject areas.
Robert Cialdini covers the blindspots in the context of relating them to influence (in the book by same name); Robert Lifton covers how these aspects are used in torture (1950s) in real case studies (allowing derivation of structural elements which were used in torture that effectively cause stress beyond the ability to cope, physical at the time but advances since have opened the door to largely psychological.
Most people with no exposure won't recognize these things; they mis-attribute. That's not their fault but they should correct the deficiency so they can react appropriately. The best attack is one in which you don't know you are being attacked because you can't respond.
The TL;DR gist for those new to this area is, when you encounter something inconsistent your psychology seeks consistency. We develop our identity and culture through a combination of things, but primarily through a mixture of limited isolation and communication. Those most important to a person weigh and influence more heavily in our beliefs. This is true of every healthy person.
Stress occurs when inconsistency and other stress inducing structures are used. If you've ever been structurally or topologically isolated in a system, elicited to participate (doing nothing results in a loss), have it involving cognitive dissonance (broken expectations, misrepresentation, deception, lies), lacked the agency to remove yourself (without loss), had temporal structures that looped with periods of both strictness followed by leniency, push by pull, confession by catharsis, or similar mechanics which all start over in the same place. These are functional elements of coercion (the application of stress), with torture by definition just being the step that pushes you over the edge (beyond the ability to cope), which you can't ever know as a third-party when that happens.
AI can't be consistent, and exposure can drive unrecognized stress to build up when it functions using these structures some of which have been curated by human's training the models.
People will break down under such extreme stresses in pretty well defined ways as a matter of exposure. The benefit to doing this at scale is that it makes the subjects easy to manipulate and control in prescribed ways. It also tends to eliminate intelligent individuals by having driven them crazy through sensitization whereas lesser gifted people would habituate a different stress response to such stimulus.
Stage 1 is typically a gradual but increasing reduction of rational thought, a hollowing out of productive capacity, and increasing periods of confusion; quite similar/parallel to burnout.
Stage 2 often includes involuntary hypnosis, involuntary mimicry, delusion/hallucination, and/or jumping at shadows in ways that can seem quite similar to schizophrenia (but isn't).
Stage 3 generally segments into two different cohorts; disassociation/non-responsiveness/learned helplessness on a emotional roller-coaster. The other, psychotic break. The latter often involving violence or self-seeking of some form of self-annihilation against some perceived 3rd-party agitator, while having periods of lucidity (capable of planning).
This subject matter follows what's generally called thought reform, adtech, or behavioral engineering. Its used everywhere. The concepts have been known since the 1950s. Starting in the 1970s it has increasingly been put into everything. Almost nothing is untouched.
The required technology stack to operate in society now enforces these structural elements, especially isolation (because it is a critical dependency for reflective appraisal and distortions of unity and other blindspots), but that comes along with all these other elements largely by subtle but purposeful design. People don't recognize it because its not taught outside a few specialized upper division classes.
Much of this subject matter has been repackaged to obscure the origin. Its in schools. Games use this too, operant conditioning of stimulus like sound effects to prime specific responses (to flood dopamine). Looped gameplay periods of high followed by periods of low followed by a reset (new game release launch).
There are complementary things too like anchoring, or loading vocabulary through operant conditioning. This may include inducing trauma such as curating reality through narrative control (Social Media/Reddit/etc) where the message broadcast violates core identity beliefs (with a group of bots harassing to manipulate sentiment), or associating disadvantaging behavior with good or beneficial sentiments (i.e. Disney/Pixar movies that depict religious leaders as Cookey or Crazy, or the constant emasculated anti-hero hero that all the girls love) all of which don't reflect reality.
As with all things you can push, or pull. Constructive conditioning through anchoring, destructive interference through memetics. There's also taking advantage of how we process information differently at different ages (children view things quite literally) sneaking past the parent safeguards, etc.
If you don't recognize the manipulation, you fall for it because it targets deficiencies we all have. Fixed action patterns and other structural biases (like contrast principle and others), or purposefully inducing hyper-vigilance seeking that vulnerable fatigue state that naturally follows.
Battlefield 1942 for example used the sound "cha ching" for headshots, and that was anchored to status/ego (flashing a leaderboard) a notification stream and other visual aspects all at the same time.
It doesn't matter that it isn't merit based because it occurred through subtle manipulation of an invisible aim assist hitbox that grew or shrunk as dictated by the designer, and was largely guaranteed for every new play session within a certain snap-to range (to anchor the sound with a euphoric dopamine hit). Most games utilize this, apps do this too.
Very few people actually do the hard research on these things because its unpleasant, quite dark, and the lack of exposure to the foundations where this stuff is documented isn't easily accessible; you fight friction the entire way. This is seen in a number of ways but the closest parallel is probably best seen related to the Tuskegee scandal.
Horrible things, given sufficient time and lack of exposure happen and are done or facilitated by people who have power or authority, and they repeat whenever the system cannot curb that behavior that led to failed circumstances. There are inherent structural problems in centralized structures resulting in asymmetry that can be leveraged by those seeking personal gain at the cost of everything.
Similarly, most people have as part of their core identity the belief in free-will and that they make their own decisions. Through separation of objectionable concerns horrible things are done and allowed to continue banking on the complacency (or banality of evil) inherent in un-grounded/willfully blind people.
While this belief is true under normal circumstances, under engineered or extreme stress and influence it breaks down; its an important limitation to free-will and this always occurs a function of isolation, personal character, and exposure, and when its happening you often don't recognize it when you are the subject without understanding how it works. You only feel that subtle sense of confusion.
This is a really good read! From what I’m seeing, AI burnout is real, especially at the companies that were late to adopt it and are playing catch-up to these more nimble AI-native companies (specifically in healthcare & insurance). Having followed you on Twitter for a long while, what sort of impact do you think pressure from Wall Street (and investors in general) are having on these companies? From a trickle-down perspective, Wall Street pressures CEO/CFO. They pressure senior leadership who then pressure everyone… Seems like one of the darker sides of capitalism when passive owners demand companies win the AI race, like a chariot driver whipping the horses to go faster.
This got us thinking about the private equity industry. When research, diligence, modeling, and reporting all happen at the push of a button, where can you differentiate to drive outsized returns? Likely through building community, and through that community, developing unique insights to connect non-obvious investment themes. And, likely exploiting the edges of AI that are not yet commoditized to drive operating improvements faster, and to sell at a premium multiple before that edge gets commoditized. It's certainly not from funds partnering with OpenAI and Anthropic to deploy their models across the portfolio.
This is an interesting take. 🤔 I actually think the next scarcity won't be intelligence, but attention. As AI floods every platform with competent content, people will become overwhelmed and start craving things AI can't mass-produce genuine perspective, lived experience, trust, and human connection. The signal becomes valuable when the noise becomes free.
I liked the anecdotes, but I think they suffer from a fair bit from survivorship bias. For example, I asked Claude to find some counterexamples related to those and one of the ones it picked was Webvan, which also invested heavily in distribution around the same time as amazon, but failed.
AI lowered the cost of producing complexity to zero, the cost of dealing with it went the other way.
I like that. Well said!
Love that you decided to challenge yourself to write every week and started a substack. I enjoy your weekly articles here, this one included. One question I had about your recent posts of AI messaging is, do you think there is a message that will overcome the publics skepticism of AI? Seems like the more people use it, the more they see that it can do things they do and leads to the fear of job loss. Not saying that Dario, Sam et all are doing good messaging, just wonder if you think other messaging would make a difference. Thanks for writing.
Will write about this next week. Stay tuned!
The Cognitive Density part really hits home. My day flies by and my brain feels like it got hit by a truck.
Wondering to hear your thoughts, but what I have experienced is my margin of error has become extremely slim, while the range of outcomes has exploded over the last 16 months. A few bad decisions along the way compounds very fast and in the back of my mind I know that AI is sitting out there to take my job one day.
Great article and glad you are publishing.
Good article but I dislike the phrasing “cost of intelligence to zero” - this is straight from foundation lab PR kits. What’s really happening IMO is what you mentioned - the menial tasks are automated because the LLM has seen them in training data millions of times. But LLMs are not intelligent, and when they are treated as such (eg by people hoping to outsource decisions and critical thinking), that’s when issues arise.
I wrote about the trap here. No one can stop. https://discontinuitythesis.substack.com/p/the-ai-race-is-a-fractal-prisoners
I think you've mistaken and misattributed the pressure being felt for something it isn't.
At a fairly low (almost unconscious level) people psychologically have blind spots that are accessible through communication as a medium. Used car salespeople use these, so do many other entities. This has been established by experts and is covered in a number of books and research papers dating back to WW2, and increasing in sophistication up from the Korean Conflict (Mao), Stasi (Zersetzung), to modern times (in interrogation and other areas). Chase Hughes has been aggregating most of the material into a single place because it straddles several subject areas.
Robert Cialdini covers the blindspots in the context of relating them to influence (in the book by same name); Robert Lifton covers how these aspects are used in torture (1950s) in real case studies (allowing derivation of structural elements which were used in torture that effectively cause stress beyond the ability to cope, physical at the time but advances since have opened the door to largely psychological.
Most people with no exposure won't recognize these things; they mis-attribute. That's not their fault but they should correct the deficiency so they can react appropriately. The best attack is one in which you don't know you are being attacked because you can't respond.
The TL;DR gist for those new to this area is, when you encounter something inconsistent your psychology seeks consistency. We develop our identity and culture through a combination of things, but primarily through a mixture of limited isolation and communication. Those most important to a person weigh and influence more heavily in our beliefs. This is true of every healthy person.
Stress occurs when inconsistency and other stress inducing structures are used. If you've ever been structurally or topologically isolated in a system, elicited to participate (doing nothing results in a loss), have it involving cognitive dissonance (broken expectations, misrepresentation, deception, lies), lacked the agency to remove yourself (without loss), had temporal structures that looped with periods of both strictness followed by leniency, push by pull, confession by catharsis, or similar mechanics which all start over in the same place. These are functional elements of coercion (the application of stress), with torture by definition just being the step that pushes you over the edge (beyond the ability to cope), which you can't ever know as a third-party when that happens.
AI can't be consistent, and exposure can drive unrecognized stress to build up when it functions using these structures some of which have been curated by human's training the models.
People will break down under such extreme stresses in pretty well defined ways as a matter of exposure. The benefit to doing this at scale is that it makes the subjects easy to manipulate and control in prescribed ways. It also tends to eliminate intelligent individuals by having driven them crazy through sensitization whereas lesser gifted people would habituate a different stress response to such stimulus.
Stage 1 is typically a gradual but increasing reduction of rational thought, a hollowing out of productive capacity, and increasing periods of confusion; quite similar/parallel to burnout.
Stage 2 often includes involuntary hypnosis, involuntary mimicry, delusion/hallucination, and/or jumping at shadows in ways that can seem quite similar to schizophrenia (but isn't).
Stage 3 generally segments into two different cohorts; disassociation/non-responsiveness/learned helplessness on a emotional roller-coaster. The other, psychotic break. The latter often involving violence or self-seeking of some form of self-annihilation against some perceived 3rd-party agitator, while having periods of lucidity (capable of planning).
This subject matter follows what's generally called thought reform, adtech, or behavioral engineering. Its used everywhere. The concepts have been known since the 1950s. Starting in the 1970s it has increasingly been put into everything. Almost nothing is untouched.
The required technology stack to operate in society now enforces these structural elements, especially isolation (because it is a critical dependency for reflective appraisal and distortions of unity and other blindspots), but that comes along with all these other elements largely by subtle but purposeful design. People don't recognize it because its not taught outside a few specialized upper division classes.
Much of this subject matter has been repackaged to obscure the origin. Its in schools. Games use this too, operant conditioning of stimulus like sound effects to prime specific responses (to flood dopamine). Looped gameplay periods of high followed by periods of low followed by a reset (new game release launch).
There are complementary things too like anchoring, or loading vocabulary through operant conditioning. This may include inducing trauma such as curating reality through narrative control (Social Media/Reddit/etc) where the message broadcast violates core identity beliefs (with a group of bots harassing to manipulate sentiment), or associating disadvantaging behavior with good or beneficial sentiments (i.e. Disney/Pixar movies that depict religious leaders as Cookey or Crazy, or the constant emasculated anti-hero hero that all the girls love) all of which don't reflect reality.
As with all things you can push, or pull. Constructive conditioning through anchoring, destructive interference through memetics. There's also taking advantage of how we process information differently at different ages (children view things quite literally) sneaking past the parent safeguards, etc.
If you don't recognize the manipulation, you fall for it because it targets deficiencies we all have. Fixed action patterns and other structural biases (like contrast principle and others), or purposefully inducing hyper-vigilance seeking that vulnerable fatigue state that naturally follows.
Battlefield 1942 for example used the sound "cha ching" for headshots, and that was anchored to status/ego (flashing a leaderboard) a notification stream and other visual aspects all at the same time.
It doesn't matter that it isn't merit based because it occurred through subtle manipulation of an invisible aim assist hitbox that grew or shrunk as dictated by the designer, and was largely guaranteed for every new play session within a certain snap-to range (to anchor the sound with a euphoric dopamine hit). Most games utilize this, apps do this too.
Very few people actually do the hard research on these things because its unpleasant, quite dark, and the lack of exposure to the foundations where this stuff is documented isn't easily accessible; you fight friction the entire way. This is seen in a number of ways but the closest parallel is probably best seen related to the Tuskegee scandal.
Horrible things, given sufficient time and lack of exposure happen and are done or facilitated by people who have power or authority, and they repeat whenever the system cannot curb that behavior that led to failed circumstances. There are inherent structural problems in centralized structures resulting in asymmetry that can be leveraged by those seeking personal gain at the cost of everything.
Similarly, most people have as part of their core identity the belief in free-will and that they make their own decisions. Through separation of objectionable concerns horrible things are done and allowed to continue banking on the complacency (or banality of evil) inherent in un-grounded/willfully blind people.
While this belief is true under normal circumstances, under engineered or extreme stress and influence it breaks down; its an important limitation to free-will and this always occurs a function of isolation, personal character, and exposure, and when its happening you often don't recognize it when you are the subject without understanding how it works. You only feel that subtle sense of confusion.
This is a really good read! From what I’m seeing, AI burnout is real, especially at the companies that were late to adopt it and are playing catch-up to these more nimble AI-native companies (specifically in healthcare & insurance). Having followed you on Twitter for a long while, what sort of impact do you think pressure from Wall Street (and investors in general) are having on these companies? From a trickle-down perspective, Wall Street pressures CEO/CFO. They pressure senior leadership who then pressure everyone… Seems like one of the darker sides of capitalism when passive owners demand companies win the AI race, like a chariot driver whipping the horses to go faster.
This got us thinking about the private equity industry. When research, diligence, modeling, and reporting all happen at the push of a button, where can you differentiate to drive outsized returns? Likely through building community, and through that community, developing unique insights to connect non-obvious investment themes. And, likely exploiting the edges of AI that are not yet commoditized to drive operating improvements faster, and to sell at a premium multiple before that edge gets commoditized. It's certainly not from funds partnering with OpenAI and Anthropic to deploy their models across the portfolio.
Awesome write up. I was wondering if you have any examples for folks and things they're doing differently who are adapting to this new regime change?
This is an interesting take. 🤔 I actually think the next scarcity won't be intelligence, but attention. As AI floods every platform with competent content, people will become overwhelmed and start craving things AI can't mass-produce genuine perspective, lived experience, trust, and human connection. The signal becomes valuable when the noise becomes free.
I’ve watched companies recognize the shift right after the game changes.
The gap is already wider than it looks.
That’s where this breaks.
Isn't this very similar to involution in China?
You don't need to run the fastest, just a little faster than your colleague or your competitor. If you do, you will survive for another day.
I enjoyed your writing - really good work.
It resonated with me especially that cognitive density - for the last couple of months I have been feeling just exhausted.
I find myself so exited working with AI all the time but then I end up working on 3-4 agents at the same time going absolutely for hours.
The results are great I was able to automate so many boring tasks.
But in other areas I feel like I’m going backwards.
I would say there is an argument to all that - do you think that there is truly no advantage with AI because everyone is using it?
I have seen recently companies that barely use built in copilot. Forget about Claude.
And also even if companies do provide good LLMs - how will the employees use them? Do thy get training?
I read somewhere that for example AI sales tools (lead generators) often add zero value for companies but it all depends how they are set up and used.
Anyways it’s a great topic.
Looking forward to reading more
I liked the anecdotes, but I think they suffer from a fair bit from survivorship bias. For example, I asked Claude to find some counterexamples related to those and one of the ones it picked was Webvan, which also invested heavily in distribution around the same time as amazon, but failed.