Spot on. Or because of addictiveness. Or impact on democracy. Or ruining people's ability to focus on anything. Those attention grabbing algos havent been designed by non-technical people.
I could have explained that better, but it was already getting too long.
Personally if I could Thanos-snap AI out of existence I would, but as long as it exists, it seems better on balance to make good use of it where possible than just try to ignore it.
"Focusing more on AI stuff" ... not really? Of course I was using AI a lot in my work at Google, as well as contributing to AI acceleration. Now I won't be contributing to acceleration and I can be as cautious as I want to be about using it.
Advertising has the appearance of representing an on-going revenue source for an increasing number of devices used in our daily lives by way of surveillance and creation of a picture of an individuals habits and psychology in order to manipulate said psychology to induce the individual to spend more money on literally worthless shit.
That is a threat to the fundamental stability of society.
Correct, but advertising was a better disguise for surveillance. AI goes well beyond surveillance and many evil use cases are now obvious to more people.
I don’t think that’s what he meant. They both are surveillance tools. He’s just upset only one of those is making people advertise their leave in protest.
I think it's worse than that. Or at least, the products that have been built because of advertising are worse than that. Instagram/TikTok/Facebook are all ultimately just surfaces to serve ads, but they've wormed their way into their users' consciousness in fairly insidious ways
It’s funny how the [very real] thing that normalizes the exploitation of user data, enables “reel casinos,” suffocates creators and replaces them with “influencers,” etc. is compared with the [still very abstract] danger of AI.
Even before big money was on the table, people didn’t blink an eye at starting to work for Google/Facebook, “solving fun tasks.”
Advertising turned brains of entire generation (or maybe a few) into mush.
All intellectual, psych and social scores are in free fall since introduction of ad-supported attention grabbing internet.
AI is theorethical threat. Advertising is a constant detriment to our brains, societies and culture.
If AI can mitigate a fraction of the damage that ads do it will have massive positive impact.
We live in times where everyone can pick their own flavor of apocalypse to be scared of. Ads are leading us into Idiocracy since ~2004 and made huge progress towards it.
Google and Meta's wealth came at the cost of civilization getting sucked dry of its attention and mental capacity.
Why is uncertainty about the motives of one form of superintelligence terrifying, whilst, at the same time, uncertainty about the motives of another, omnipotent supernatural intelligence comforting? i.e. the “I don’t know God’s plan, but I sleep well knowing He has one” line of reasoning.
God’s been around for thousands of years and we are still here and flourishing, so obviously God is in some way “aligned”; the same cannot be said for AGI.
Survivorship bias. Ask all the humans that have died in horrific screaming agony, and saw everything they worked for turned to dust how well that was aligned with their interests.
This little blue marble of Earth is an as-yet undiscovered anti-entropic bug in an otherwise accurately functioning entropy-respectingly beautiful wasteland of a universe.
Christians believe that God is inherently good. “This is the message we have heard from him and proclaim to you: that God is light and in him is no darkness at all.”
Well It seems like you may join us in our goal: Find a way to stop the progress or further development of it. Not that we are afraid, but we are fully aware of the progress escalation.
We had people departing from OpenAI, Anthropic and now Google. The pace the frontier open letter was signed by over 1,000 experts. At which point do we believe the message?
For a resignation letter that is unusually detailed w.r.t. "what comes next?" It may not be uncommon to write why you leave, but the rest of that text does not read like what you send to HR to quit but what you send to your (soon to be ex-)coworkers afterwards.
If you think humans are a net positive to be around, you wouldn't be spending trillions trying to build a machine to replace them; not just their work, but all aspects of their lives.
> I’m also very concerned about other AI-related issues: cognitive surrender, AI-induced psychosis
How's he so sure he's not already in both of these groups? His post about the AGI boogeyman sounds like it to me. Why is he bothering working on both of his projects if AGI is right around the corner anyways?
Makes some important and objectively correct points and then completely blows any chance of looking sane or being taken seriously in the last paragraph. What exactly is your God forcing you to do, and how? Is the coming collapse not also part of his plan from your framework or is it only relevant when it's comforting?
Perhaps a good thing to come out of these big companies over-indexing on "AI" is the liberation of creative minds. So many fantastic open source contributors have disappeared into corporate black holes...
I think this take is extremely cynical and made in bad faith. Lots of dangers that are due to AI acceleration are raised by a very wide mix of experts, scientists, engineers and researchers.
It is clearly incorrect to call it a conspiracy theory. You can be skeptical, as there is no clear consensus even among experts, but outright dismissal is simply an unscientific position.
As an aside, this was the most eloquent, professional, polite and concise disagreement on HN I've seen in decades. I genuinely aspire to have this level of class and character in public discourse. Bookmarking this comment as reference for the future :)
I did not call it a conspiracy theory; not here at least. Have you even read what Ted Kaczynski has written? Perhaps you'd like some of it if you do. I bet he's also smarter than your average Anthropic employee.
As much as I cannot disagree more with the author's fundamental premise (religion), it is refreshing to see someone claim to have a belief and then follow through, change their behaviour to match their belief, at no small cost to themselves, and in pursuit of doing the right thing.
> I really enjoyed the work, and for a while I told myself it was relatively harmless, but over time God forced me to confront the reality that the main impact of these tools will be to accelerate the design of a new breed of AI chips...
There's no arguing with someone who claims God forced them to do something, but I do wish this person good luck in whatever they do next.
Anytime you use absolutes you paint yourself in a corner. Definitely not 100% of things that big tech "democratized", as you put it, has been bad for people.
Presenting it as such does disservice to any serious conversation.
Interesting thing to write in a forum provided for free by big tech. Probably using a browser that big tech offers for free. On a phone or laptop purchased from big tech. Connected using networking gear and software made by big tech.
I for one would prefer to pay for services, than be psychologically manipulated into paying (more, on average) to other companies who are paying to make those services worse.
Welcome to the capitalism 101. Everything has price. So far, I would say, humanity done quite well under this _oppressive_ regime when compared with others.
I for one cannot be convinced that these companies saying they want to slow down AI progress is for the safety of mankind (or any other altruistic reason). However, I respect someone willing to give up a very comfortable career for his convictions.
That most people are sincere in their beliefs (in AI doom), as the author says and I think too, doesn't mean there aren't cult leader-types exploiting these beliefs. And their private beliefs may be quite different from their stated ones.
I don't get these comments. If he got paid way above his needs for a while he can easily retire already if he continues to keep his expenses in check, and take on a position in his church. Pays with happiness, not money, though if he has enough of the latter he can focus on maximize the former.
respect for choosing your faith over your work man! I'm trying to find the intersection of my faith and the new tech too. Feel free to email if you ever wanna chat it out.
Thanks for living your values. I might not fully agree but I appreciate people whose convictions are genuine and altruistic.
Sometimes people use the "Yet you participate in society" meme as a shield for blatant hypocrisy, like a Marxist-Leninist Twitch streamer who lives in a multi-million Hollywood mansion, drives a Porsche and wears thousand dollar pants.
I think the scepticism and gaslighting of people sounding the alarm over the trajectory of AI is concerning. This feels like a case of "knowing better" than the actual people involved in the projects.
You might be right. You might be wrong. But being overly confident on a topic you're a spectator to is possibly not the right choice.
Yep. In fact you just have to look at the people running these AI companies and listen to their own words, they are not painting a positive outcome for their own businesses.
The first is that AI will actuallly the world Skynet-style. I actually consider this unlikely. People are unlikely to completely remove people from the decisions to launch nuclear missiles, for example. Look at the Soviet/Russian design for Dead Hand [1], as one example.
The second is that AI destroys so many jobs that society falls apart. This is the far more problematic and likely scenario (IMHO). To put this in context, I see two general outcomes for AI:
1. AI will be used for further widen the inequality gap as the already ultra-wealthy will get, well, more wealthy, for literally no reason. This is the outcome where you still need a job to survive but a whole bunch of them disappear and aren't replaced. The world of The Expanse was kind of like this (on Earth); or
2. AI makes everyone's lives easier so less work needs to be done. Crappy jobs get automated. People in general get more leisure time and quality of life.
I believe (1) is far more likely at this stage and it's not going to be pretty. But this is an issue of economics not technology. Put another way: our society isn't structured to handle this change. Put yet another way: no amount of guardrails or slowdowns on AI will solve the fundamental issue of wealth distribution. And put even yet another way: this is capitalism vs socialism.
We have a preview of what this looks like when you compare the US to China.
The US has crumbling infrastructure, has lost the ability to manufacture everything but weapons and about 1000 people are insanely wealthy while everyone else are struggling to survive while AI systems drive up the cost of everything necessary to live like food and housing through demand pricing. We now spend over $1 trillion a year on a military to essentially siphon money to those 1000 from everyone else.
And what's China done? It's invested in infrastructure and its people. China's vision for AI isn't to maximize shareholder value at all costs. It will be strictly regulated so society as a whole gets to benefit.
So while I can respect people such as this author leaving on principle, I think they're still focusing on the wrong problem.
> And what's China done? It's invested in infrastructure and its people. China's vision for AI isn't to maximize shareholder value at all costs. It will be strictly regulated so society as a whole gets to benefit.
China is not in an ideal place right now, youth unemployment is way up, lots of dead end jobs, a 35 age employment limit that isn't just a meme. China has disparate visions for AI but maximizing owner (erm, sort of like shareholder) value is definitely one of them. Don’t get me wrong, China could get this right in the end, but it’s not a forgone conclusion right now.
Yet another "AI must be stopped" post... If you're burned out, just say so. Don't philosophise. No, we're not buying that your latest model can almost replace all software engineers. It's us working with the harnesses, adding guardrails etc that make this agentic world possible for everyone.
> or they don’t they understand that it’s mostly marketing stunt.
Understanding implies having some factual information, proof, data, hard evidence. Meanwhile you frame your conspiracy theory as such and are surprised people don't see it the same way?
What's fearmongering? There's no claim that AI will destroy the world here - only a claim that cheap omnipresent AI will make most of society worse - which is hard to dispute.
And Google’s ad-driven money-making machine is making society worse as well, but OP had no problem with this.
It’s a bit of a fad to do these things in the name of AI. I think it’s great OP chooses a more fulfilling life that will surely make them happier, but it feels a bit pretentious.
If someone did exist who was sincere and not fear-mongering, but was terrified of AI, what would their post need to look like for you to take it seriously?
Your response to someone afraid of the implications of AI is to leave. But you can't just leave society, the economy, or the environment, all of which there are reasonable reasons to think AI will harm them.
I don't think many associate intellectual capability with evil. What many do, however, is associate serious risk with living alongside vastly more intelligent entities. Especially if those entities are not aligned with us.
> Why do people automatically associate intellectual capability with evil?.
No intellectual capability, but non-human, inherently amoral intellectual capability that has already been proven act in hard-to-predict, illegal, and unethical ways. I think that's a very rational thing to be afraid of.
Isn't AI just the next step in the history of evolution?
I have the feeling it is inevitable that a new species will take over Earth. Just as humans took over a hundred thousand years ago. Now a species even more capable of thinking is coming up.
Thinking a million times faster. Knowing a million times more. Able to wrap its head around a million times more aspects at the same time.
A strong effort to slow it down might slow it down a bit. But does that really make a difference? It's not like we can "brace" ourselves from what is coming. Would the apes have been able to prepare for the arrival of humans if they had had 5 more years?
There's no new species being created here. A tool has been made. Either humanity will wipe itself out using said tool, destroy the tool, or learn to live with the tool.
It seems pretty capable of producing variations of itself. If these variations improve over time and roam earth - what is in your definition of "species" that is not met?
conceivably you could say humans took over 6000 years ago.
At any rate it can't be a new step in the history of evolution, as AI did not evolve in the same way as other species did and AI is not a species nor alive.
There is a solution to this problem of "AI going to kill everyone".
- Create powerful AI but don't release it and keep it secret.
- Ask this AI to create some sort of virus that can be delivered to every human in the world without them knowing it and this virus does a fundamental change to the human brain such that humans never ever could even think of hurting other humans.
> I plan to study how AI agents debug code and whether debugging tools that can make them more effective at that.
> I want to use AI agents to bring some of my hobby project ideas to life.
> I’m keen to reap the benefits of AI
and addresses the anti-AI view:
> AI detractors think that AI won’t really work. I think it will. (ed. for brev)
The main reason then is the concern of the velocity of the rollout and the increased pervasiveness of AI. Too fast, too much.
Personally if I could Thanos-snap AI out of existence I would, but as long as it exists, it seems better on balance to make good use of it where possible than just try to ignore it.
"Focusing more on AI stuff" ... not really? Of course I was using AI a lot in my work at Google, as well as contributing to AI acceleration. Now I won't be contributing to acceleration and I can be as cautious as I want to be about using it.
AI is concerning in some aspects, yes, but the loss of a shared reality has nothing to do with that, and is even more scary.
That is a threat to the fundamental stability of society.
I think it's worse than that. Or at least, the products that have been built because of advertising are worse than that. Instagram/TikTok/Facebook are all ultimately just surfaces to serve ads, but they've wormed their way into their users' consciousness in fairly insidious ways
Even before big money was on the table, people didn’t blink an eye at starting to work for Google/Facebook, “solving fun tasks.”
All intellectual, psych and social scores are in free fall since introduction of ad-supported attention grabbing internet.
AI is theorethical threat. Advertising is a constant detriment to our brains, societies and culture.
If AI can mitigate a fraction of the damage that ads do it will have massive positive impact.
We live in times where everyone can pick their own flavor of apocalypse to be scared of. Ads are leading us into Idiocracy since ~2004 and made huge progress towards it.
Google and Meta's wealth came at the cost of civilization getting sucked dry of its attention and mental capacity.
Stay away from the lucrative then? Why did you ever think it was a good idea to start at Google? I don't think Jesus would have approved ;)
There are trillions of dollars at stake here, do you think getting 1000 people on board is that difficult with this level of money? lol
How's he so sure he's not already in both of these groups? His post about the AGI boogeyman sounds like it to me. Why is he bothering working on both of his projects if AGI is right around the corner anyways?
excellent logic, m'lord.
Perhaps a good thing to come out of these big companies over-indexing on "AI" is the liberation of creative minds. So many fantastic open source contributors have disappeared into corporate black holes...
It is clearly incorrect to call it a conspiracy theory. You can be skeptical, as there is no clear consensus even among experts, but outright dismissal is simply an unscientific position.
The post resonates with me and the author shows the way.
Is that all that different from what people fear AI might do?
Why the one makes people feel elation and the other dread?
There's no arguing with someone who claims God forced them to do something, but I do wish this person good luck in whatever they do next.
I have a lot of bones to pick with Google's morality, but "democratizing the transformer and LLMs" is at the bottom of my list.
Presenting it as such does disservice to any serious conversation.
All bad?
Woe is me, victim of the 21st century separated from my true calling of paying for software.
Edit (rate limited):
> Does Gemma belong to this list?
Yes, and Tensorflow too.
I do expect to use AI in my work, as I indicated in my blog post. I haven't (yet?) reached the point where I consider that anathema...
Sometimes people use the "Yet you participate in society" meme as a shield for blatant hypocrisy, like a Marxist-Leninist Twitch streamer who lives in a multi-million Hollywood mansion, drives a Porsche and wears thousand dollar pants.
You might be right. You might be wrong. But being overly confident on a topic you're a spectator to is possibly not the right choice.
The first is that AI will actuallly the world Skynet-style. I actually consider this unlikely. People are unlikely to completely remove people from the decisions to launch nuclear missiles, for example. Look at the Soviet/Russian design for Dead Hand [1], as one example.
The second is that AI destroys so many jobs that society falls apart. This is the far more problematic and likely scenario (IMHO). To put this in context, I see two general outcomes for AI:
1. AI will be used for further widen the inequality gap as the already ultra-wealthy will get, well, more wealthy, for literally no reason. This is the outcome where you still need a job to survive but a whole bunch of them disappear and aren't replaced. The world of The Expanse was kind of like this (on Earth); or
2. AI makes everyone's lives easier so less work needs to be done. Crappy jobs get automated. People in general get more leisure time and quality of life.
I believe (1) is far more likely at this stage and it's not going to be pretty. But this is an issue of economics not technology. Put another way: our society isn't structured to handle this change. Put yet another way: no amount of guardrails or slowdowns on AI will solve the fundamental issue of wealth distribution. And put even yet another way: this is capitalism vs socialism.
We have a preview of what this looks like when you compare the US to China.
The US has crumbling infrastructure, has lost the ability to manufacture everything but weapons and about 1000 people are insanely wealthy while everyone else are struggling to survive while AI systems drive up the cost of everything necessary to live like food and housing through demand pricing. We now spend over $1 trillion a year on a military to essentially siphon money to those 1000 from everyone else.
And what's China done? It's invested in infrastructure and its people. China's vision for AI isn't to maximize shareholder value at all costs. It will be strictly regulated so society as a whole gets to benefit.
So while I can respect people such as this author leaving on principle, I think they're still focusing on the wrong problem.
[1]: https://www.military.com/history/russias-dead-hand-soviet-bu...
China is not in an ideal place right now, youth unemployment is way up, lots of dead end jobs, a 35 age employment limit that isn't just a meme. China has disparate visions for AI but maximizing owner (erm, sort of like shareholder) value is definitely one of them. Don’t get me wrong, China could get this right in the end, but it’s not a forgone conclusion right now.
Understanding implies having some factual information, proof, data, hard evidence. Meanwhile you frame your conspiracy theory as such and are surprised people don't see it the same way?
It’s a bit of a fad to do these things in the name of AI. I think it’s great OP chooses a more fulfilling life that will surely make them happier, but it feels a bit pretentious.
Or you think, those scumbag billionaires will care more when people fear more?
If they were genuinely afraid, you'd prefer they stay silent?
Why do people automatically associate intellectual capability with evil?
Does American anti-intellectualism explain the difference in perception of AI by Americans and Chinese?
No intellectual capability, but non-human, inherently amoral intellectual capability that has already been proven act in hard-to-predict, illegal, and unethical ways. I think that's a very rational thing to be afraid of.
The reverse also holds, btw.
I have the feeling it is inevitable that a new species will take over Earth. Just as humans took over a hundred thousand years ago. Now a species even more capable of thinking is coming up.
Thinking a million times faster. Knowing a million times more. Able to wrap its head around a million times more aspects at the same time.
A strong effort to slow it down might slow it down a bit. But does that really make a difference? It's not like we can "brace" ourselves from what is coming. Would the apes have been able to prepare for the arrival of humans if they had had 5 more years?
It seems pretty capable of producing variations of itself. If these variations improve over time and roam earth - what is in your definition of "species" that is not met?
At any rate it can't be a new step in the history of evolution, as AI did not evolve in the same way as other species did and AI is not a species nor alive.
- Create powerful AI but don't release it and keep it secret.
- Ask this AI to create some sort of virus that can be delivered to every human in the world without them knowing it and this virus does a fundamental change to the human brain such that humans never ever could even think of hurting other humans.
- Release this powerful AI to everyone.
Will that work?
I do believe this is possible for a virus to achieve, I just don't think it's possible without killing the human.