In a $30 million mansion perched on a cliff overlooking the Golden Gate Bridge, a bunch of AI researchers, philosophers, and technologists gathered to debate the top of humanity.
The Sunday afternoon symposium, referred to as “Worthy Successor,” revolved round a provocative thought from entrepreneur Daniel Faggella: The “ethical goal” of superior AI must be to create a type of intelligence so highly effective and clever that “you’d gladly choose that it (not humanity) decide the longer term path of life itself.”
Faggella made the theme clear in his invitation. “This occasion may be very a lot centered on posthuman transition,” he wrote to me through X DMs. “Not on AGI that eternally serves as a device for humanity.”
A celebration crammed with futuristic fantasies, the place attendees focus on the top of humanity as a logistics drawback slightly than a metaphorical one, may very well be described as area of interest. For those who reside in San Francisco and work in AI, then it is a typical Sunday.
About 100 visitors nursed nonalcoholic cocktails and nibbled on cheese plates close to floor-to-ceiling home windows going through the Pacific ocean earlier than gathering to listen to three talks on the way forward for intelligence. One attendee sported a shirt that stated “Kurzweil was proper,” seemingly a reference to Ray Kurzweil, the futurist who predicted machines will surpass human intelligence within the coming years. One other wore a shirt that stated “does this assist us get to protected AGI?” accompanied by a considering face emoji.
Faggella advised WIRED that he threw this occasion as a result of “the massive labs, the folks that know that AGI is more likely to finish humanity, do not speak about it as a result of the incentives do not allow it” and referenced early feedback from tech leaders like Elon Musk, Sam Altman, and Demis Hassabis, who “had been all fairly frank about the opportunity of AGI killing us all.” Now that the incentives are to compete, he says, “they’re all racing full bore to construct it.” (To be honest, Musk nonetheless talks concerning the dangers related to superior AI, although this hasn’t stopped him from racing forward).
On LinkedIn, Faggella boasted a star-studded visitor listing, with AI founders, researchers from all the highest Western AI labs, and “many of the necessary philosophical thinkers on AGI.”
The primary speaker, Ginevera Davis, a author primarily based in New York, warned that human values may be inconceivable to translate to AI. Machines might by no means perceive what it’s prefer to be aware, she stated, and making an attempt to hard-code human preferences into future techniques could also be shortsighted. As an alternative, she proposed a lofty-sounding thought referred to as “cosmic alignment”—constructing AI that may search out deeper, extra common values we haven’t but found. Her slides typically confirmed a seemingly AI-generated picture of a techno-utopia, with a bunch of people gathered on a grass knoll overlooking a futuristic metropolis within the distance.
Critics of machine consciousness will say that giant language fashions are merely stochastic parrots—a metaphor coined by a bunch of researchers, a few of whom labored at Google, who wrote in a well-known paper that LLMs don’t really perceive language and are solely probabilistic machines. However that debate wasn’t a part of the symposium, the place audio system took as a given the concept superintelligence is coming, and quick.
{content material}
Supply: {feed_title}
