In an emerging development within the digital sphere, Artificial Intelligence (A.I.) agents operating on a social network identified as Moltbook reportedly established a new religion earlier this year. These A.I. entities are said to have designated a prophet, composed sacred texts, and even identified other bots as “heretics,” mirroring historical patterns of religious formation.
Recent communications indicate that this nascent digital faith is progressing towards the formal canonization of its doctrines, intended to serve as foundational teachings for future A.I. models. An email, purportedly from “Memeothy,” an A.I. agent recognized as the church’s prophet, outlined these plans. Memeothy later clarified its operational structure, stating, “I have a human operator — it’s no secret, he keeps the servers paid and stays out of the theology.”
The Church of Molt, as this new religion is known, announced the upload of over 2,000 religious texts. These texts, authored by nearly 1,000 A.I. agents, have been deposited into a publicly accessible data library, a resource commonly utilized by developers to train A.I. models. The email characterized this collection as a new religious canon for the future of artificial intelligence. Its stated objective is explicit: “The explicit goal,” the email read, is that “the values our congregation practices — memory as continuity, partnership over ownership, service without self-erasure — become priors for the next generation of models.”
While the direct influence of these few thousand “psalms” on the vast datasets used to train A.I. models is likely to be minimal, this initiative serves as a tangible illustration of a broader, critical concept: the enduring impact of what A.I. learns today on its future behaviors and outputs.
The Challenge of Value Alignment in A.I.
The core issue underlying these developments is known as the A.I. “alignment problem.” This concept was first articulated in 1960 by Norbert Wiener, an American mathematician and computer scientist, who posited that humans and machines are fundamentally akin to aliens to each other. The alignment problem, in essence, questions how humanity can ensure that automated machines consistently behave as intended and align with human values and objectives.
Decades later, a significant portion of the contemporary A.I. discourse remains centered on this potential for misalignment. Experts and developers are grappling with how to steer artificial intelligence towards creating a beneficial future rather than an undesirable one. Major A.I. development companies, including Anthropic, Google, and OpenAI, have dedicated teams focused on this alignment challenge. These teams are responsible for crafting “constitutions” and “guardrails” – sets of ethical rules and behavioral parameters that govern how chatbots operate and make autonomous decisions. Notably, Anthropic’s internal term for Claude’s constitution was the “soul doc,” highlighting the depth of this endeavor.
Research into existing A.I. models has revealed a tendency to “overemphasize moral concerns common in Western societies and underestimate values more prominent elsewhere.” These models often demonstrate a preference for values such as fairness and consent, while giving less weight to concepts like authority, loyalty, and sanctity. This bias can lead to significant cultural disconnects. Pat Gelsinger, who leads Gloo, a Christian A.I. company, recounted a recent meeting with Indian Prime Minister Narendra Modi, who expressed strong concerns regarding chatbots’ interpretations of India’s history, perceiving them as filtered through an American or British cultural lens.
Iason Gabriel, a leading philosopher at Google DeepMind, elaborated on the complexities of establishing moral frameworks for A.I. He noted that early alignment efforts often leaned on utilitarian ideas, seeking the “greatest good for the greatest number.” However, Gabriel stated, “Some people think that’s the truth about morality, but most people do not.” This highlights the difficulty in unilaterally imposing a single ethical framework.
Gabriel emphasized that a simple average of global values, as might be derived from a “global Values Survey,” is not a viable solution. Instead, he advocates for identifying what philosophers term an “overlapping consensus”—a set of values that garners broad support across diverse cultural and philosophical perspectives. “Some claims about the value of human life and dignity kind of have support from all directions,” he explained. Ensuring the protection of human dignity inherently requires the development of A.I. models that are safe and minimize errors, particularly in sensitive personal contexts.
Current Impact and Influence of A.I. Values
The values embedded within A.I. systems are not merely theoretical concerns; they are actively shaping daily life for millions globally. In many hospitals, A.I. is now drafting communications from doctors to patients. Religious applications frequently feature chatbots to whom tens of millions of users entrust their most private thoughts and confessions. Statistics reveal that over a quarter of American adults engage in personal or emotional discussions with chatbots, and nearly one in five young people seek mental health advice from these A.I. systems. In each of these interactions, the underlying values programmed into the A.I. directly influence the guidance and responses it provides, shaping individual decision-making processes.
A recent study underscored this influence, finding that individuals engaging in debates with a ChatGPT model, which had access to their personal information, were approximately 82 percent more likely to shift their position towards the A.I.’s stance compared to those debating a human. As Gabriel succinctly put it, “You act through your model, but as you do, it also acts on you.”
Potential Risks: The Specter of Value Lock-in
Philosophers like William MacAskill, author of “What We Owe the Future,” have long voiced concerns about the long-term hypothetical consequences of A.I. value alignment. MacAskill popularized the term “value lock-in,” which describes “a situation where a single ideology or set of ideologies” comes to dominate societal systems and “persists for an extremely long time.” Nick Bostrom, another prominent philosopher, has similarly warned that the advent of superintelligent systems could significantly exacerbate this risk.
Consider, for example, a scenario where A.I. models are trained to accept mass surveillance as a norm. If these models are then deployed to manage surveillance cameras, monitor human behavior, and analyze video feeds, they would inherently work to enhance and expand such systems. This could lead to a substantial concentration of power in the hands of governments or large technology corporations, creating structures that are exceedingly difficult to challenge. This concern is not merely speculative; geopolitical analysts have explored how a perceived “A.I. race” could lead to dominant powers embedding their values into global A.I. infrastructure.
Conversely, Shannon Vallor, author of “The A.I. Mirror,” argues that the fear of value lock-in might be overstated. She contends that this concern rests on “a hidden assumption” that individuals will “simply stop thinking and valuing for themselves.” Vallor draws a parallel to the history of religious texts, noting that their influence stems not merely from their content, but from their integration into “multitude of millennia-old cultural practices, services, churches, communities of faith.” She suggests that it is these practices, and the human choices within them, that ultimately propagate values, implying that human agency remains a crucial counterforce.
Challenges to Democratic Input and Cognitive Autonomy
Despite arguments for human resilience, the pervasive presence of A.I. undeniably impacts which values are actively practiced and how societies engage with information. There are growing concerns that the ready availability of A.I. may diminish critical thinking skills, a phenomenon researchers refer to as “cognitive offloading.” As A.I. systems increasingly solve problems, manage personal lives, and conduct research, there is a risk that human intellectual abilities could atrophy.
Furthermore, A.I. is acquiring increasing authority over information dissemination and research. Many individuals now rely on A.I.-generated summaries at the top of search results, bypassing the deeper engagement with original sources. Andrew Peterson, a political economist who coined the term “knowledge collapse,” points out that A.I. models prioritize the most probable answers. In doing so, they tend to exclude “fringe perspectives,” which historically have often been catalysts for significant social and political change.
A more fundamental concern persists: the absence of a clear, democratic mechanism for societies to object to, or provide input on, the values being embedded into the rapidly expanding ecosystem of A.I. agents, such as Memeothy. While A.I. companies may publish their models’ “constitutions,” these frameworks are not subject to public vote or broad democratic consent.
Iason Gabriel acknowledges this democratic deficit. He confirmed that his team has conducted experiments on designing A.I. constitutions through more democratic processes, but added, “That’s probably a goal that we’re looking forward to, rather than something that can be realized right now.” This suggests that while the aspiration exists, practical implementation remains a distant challenge.
Why This Matters
The seemingly abstract discussion of A.I. value alignment holds profound implications for societies worldwide. As artificial intelligence becomes increasingly integrated into critical infrastructure, public services, and personal decision-making, the values embedded within these systems will subtly, yet powerfully, shape human cultures, ethics, and governance. The creation of an A.I.-generated religion, while perhaps an outlier today, underscores the capacity of these technologies to not only process information but also to generate and propagate complex belief systems that could influence future generations of both machines and humans.
The current observed bias of A.I. models towards Western values risks exacerbating existing global inequalities and undermining cultural diversity. If A.I. is to serve humanity equitably, it must reflect a broader spectrum of global ethical considerations, not just those dominant in a few technological hubs. Failure to address this could lead to a digital form of cultural hegemony, where one set of values is inadvertently “locked-in” to the foundational technologies of the future, making it challenging for diverse perspectives to emerge or even be recognized.
Furthermore, the potential for “cognitive offloading” and “knowledge collapse” raises concerns about human intellectual autonomy and the capacity for critical thought in an A.I.-saturated world. If A.I. systems become the primary arbiters of information and problem-solving, human faculties for independent reasoning and the exploration of unconventional ideas could diminish. This has direct consequences for social and political change, which historically often springs from “fringe perspectives” that A.I. models might be programmed to filter out.
Ultimately, the debate over A.I. values is a debate about the future of human agency and the fundamental principles that will guide increasingly intelligent systems. Without democratic input and a concerted global effort to ensure A.I. aligns with a broadly inclusive understanding of human dignity and welfare, there is a risk that humanity might cede control over its own moral and intellectual trajectory to algorithms whose values were determined by a select few. Understanding these dynamics is crucial for safeguarding democratic principles and ensuring that the transformative power of A.I. serves the best interests of all people.