AI as Normal Technology

Below top-k · 141

• claimSeparating methods, applications, and adoption — which move at different speedsc 0.90

  Transformative economic and societal impacts will unfold on the scale of decades because AI methods, AI applications, and AI adoption progress at very different timescales. Conflating them produces wildly wrong forecasts.

• claimBenchmarks measure methods, not applicationsc 0.90

  AI benchmarks track progress in methods but are routinely misread as evidence of progress in real-world applications. This confusion is a major driver of hype about imminent economic transformation.

• claimReplace 'intelligence' with capability and power for clearer reasoningc 0.90

  Talk of 'intelligence' and 'superintelligence' has clouded analysis of advanced AI. Splitting the concept into capability and power makes it clearer why human labor isn't doomed to be superfluous and reframes the risk picture.

• claimThere is no useful sense in which AI is more intelligent than humans using AIc 0.90

  Human intelligence is distinctive because we subsume tools and other intelligences into our own work, so 'intelligence' cannot be placed on a single spectrum where AI passes humans. The right comparison is AI versus humans-with-AI, not AI versus humans alone.

• claimModel alignment is inherently brittle as a misuse defensec 0.90

  Post-training alignment to refuse misuse has proven unreliable, and this limitation is fundamental rather than fixable. The primary defenses against misuse must live elsewhere.

• claimCatastrophic misinterpretation fears rest on unrealistic deployment assumptionsc 0.90

  Worries that AI will catastrophically misread commands ignore how technology actually gets deployed. Systems earn access to consequential decisions only after demonstrating reliable behavior in lower-stakes settings.

• implicationPolicymakers face a dilemma because defenses against the two risk types conflictc 0.90

  Defending against superintelligence pushes toward concentration and central control of AI; defending against systemic harms pushes toward distribution and resilience. The same intervention can mitigate one risk while worsening the other.

• claimResilience is the right governance posture for AIc 0.90

  Resilience — the capacity to absorb shocks while preserving core functions — combines ex ante and ex post actions, and is well suited to AI because it helps mitigate the pacing problem of fast-moving technology.

• claimGovernment AI adoption must walk a tightrope between haste and paralysisc 0.90

  Governments face a genuine dilemma in adopting AI: moving too fast erodes public trust through visible failures, while moving too slow cedes basic state functions to less accountable private actors.

• implicationDrastic anti-superintelligence interventions backfire if AI is normalc 0.85

  Policies premised on the impossibility of controlling superintelligent AI will make things worse if AI is actually normal technology, whose downsides will resemble those of past technologies in capitalist societies — chiefly inequality. Reducing uncertainty and building resilience should be first-rate goals instead.

• claimIn consequential domains, AI diffusion lags innovation by decadesc 0.85

  Where errors matter, deployed AI trails frontier methods by decades, largely because complex, less interpretable models are hard to validate against all real deployment conditions. Safety, not capability, is the binding constraint.

• mechanismThe most transformative tasks are the hardest to benchmarkc 0.85

  Tasks with clear correct answers, like categorizing legal requests, are easy to evaluate but economically minor. The tasks whose automation would actually reshape a profession involve judgment and creativity with no single right answer, so they resist standardized measurement.

• claimSudden economy-wide automation is implausible under the normal-technology viewc 0.85

  The AGI-style argument assumes that increasing generality lets one system automate most economically valuable tasks at once. But improvements in methods don't translate directly to economic impact, which requires the slow work of innovation and diffusion.

• claimMany cognitive tasks have an irreducible error floor near human performancec 0.85

  In domains like forecasting and persuasion, inherent stochasticity caps achievable accuracy, and trained humans already operate near that ceiling. AI cannot blow past human performance the way it did in chess.

• implicationThe control problem becomes tractable once superintelligence is off the tablec 0.85

  The 'galaxy brain in a box' framing only terrifies if you presume superintelligence. If AI is no more capable than humans-with-AI, and superhuman persuasion is unfounded, control becomes a normal engineering problem.

• claimSafety and market success in self-driving are causally linkedc 0.85

  Companies with lax safety culture in self-driving — Cruise, Uber, and possibly Tesla — have lost ground or exited the market, while Waymo has pulled ahead. The correlation between safety and commercial success appears to be causal, not coincidental.

• mechanismHarm depends on context the model cannot seec 0.85

  Whether a capability is harmful depends on context held in the attacker's orchestration, not in the model. A model writing a persuasive email cannot tell marketing from phishing.

• implicationDefenses belong at downstream attack surfacesc 0.85

  Effective protection against AI-enabled phishing comes from email filters, browser protections, OS security, and user training — the same defenses honed against human attackers. The pattern generalizes to cyber and bio threats: harden the downstream surface, not the model.

• claimDeception is an engineering problem, not a ticking time bombc 0.85

  On the superintelligence view, a deceptive system inevitably outwits humans. On the normal technology view, deception is a tractable engineering problem already being addressed as part of standard safety evaluation.

• mechanismSystemic AI risks come from people using AI, not AI acting on its ownc 0.85

  These harms arise from people and organizations using AI to advance their interests, with the technology amplifying existing social instabilities. The locus of agency stays human.

• implicationResilience, not nonproliferation, should anchor AI policyc 0.85

  The recommended strategy is resilience: taking actions now that improve our ability to handle unexpected developments. Nonproliferation violates this principle by concentrating power and reducing resilience.

• claimExpected-utility cost-benefit analysis is unviable for AI riskc 0.85

  Probability estimates for AI catastrophe lack epistemic grounding — there's no reference class and no precise model. Subjective forecasts vary by orders of magnitude, making expected-utility calculations meaningless.

• implicationNonproliferation paradoxically creates the superintelligence it fearsc 0.85

  By pushing toward more autonomy, organizational complexity, and resource access concentrated in a few systems, nonproliferation-based safety measures enable the very power-concentration that defines superintelligence risk.

• claimSuperintelligence is incoherent as usually conceptualizedc 0.80

  The essay envisions a world with advanced but not superintelligent AI, viewing the standard superintelligence concept as incoherent. In that world, control sits with people and organizations, and AI control itself becomes a growing share of human work.

• mechanismDiffusion speed is bounded by how fast humans and organizations can adaptc 0.80

  Even instantly available software diffuses slowly because people, firms, and institutions need time to rework habits, workflows, and norms. This is why general-purpose technologies diffuse over decades, not years.

• claimTacit organizational knowledge forces AI learning to happen sector by sectorc 0.80

  Much of what organizations know is tacit and unwritten, so AI's developmental feedback loops have to play out separately in each sector and often each organization. This blocks the kind of rapid, parallel learning that would otherwise compress diffusion timelines.

• claimConstruct validity is the core diagnostic for benchmark hypec 0.80

  The underlying problem is construct validity: whether a test actually measures what it claims to. The only reliable way to know if an AI application is useful is to build it and have professionals use it in realistic conditions.

• claimAI arms races are sector-specific, not civilizationalc 0.80

  Whether competitive pressure erodes safety depends on the sector. The right policy response is sector-specific regulation, not blanket AI rules.

• claimNo structural reason to expect country-level safety arms racesc 0.80

  Unsafe AI adoption produces local harms rather than externalized costs, so countries that cut corners hurt themselves. This removes the standard race-to-the-bottom incentive.

• claimMisuse-proof models would have to be either useless or insecurec 0.80

  Trying to make a model that cannot be misused is like trying to make a computer that cannot be used for bad things — controls will either block legitimate uses or fail against adaptive adversaries.

• implicationRestricting AI development can backfire on defensec 0.80

  If aligned models are made useless for tasks like finding bugs in critical infrastructure, defenders lose those tools while motivated adversaries train their own. Net offensive capability rises without a defensive counterpart.

• implicationMeasure offense-defense balance, not raw capabilityc 0.80

  The right metric for AI risk in a domain is the balance between offensive and defensive use, not offensive capability alone. Policy should invest in defensive applications rather than restrict the underlying technology.

• claimMisalignment scenarios wrongly assume autonomous, unsupervised AIc 0.80

  Catastrophic misalignment stories presume AI making high-stakes decisions without humans in the loop. In reality, existing institutional controls — financial, regulatory, procedural — create multiple protective layers.

• implicationValue pluralism and robustness should guide policy under deep uncertaintyc 0.80

  Policymakers should prefer policies acceptable across a wide range of values and that remain helpful, or at least not harmful, if their key assumptions turn out wrong. This replaces compromise and expected-utility reasoning.

• implicationMaking the worldview explicit enables mutual understanding even without agreementc 0.80

  Articulating the normal-technology worldview lets people see why sincere observers reach dramatically different conclusions about AI progress, risks, and policy — the goal is mutual understanding rather than conversion.

• caveatIncreasing generality doesn't translate to easy consequential applicationsc 0.75

  It is tempting to extrapolate that application effort will keep falling until AGI obviates applications altogether. But this trend holds mainly in domains like NLP and games — not in high-stakes settings where errors are costly and reality cannot be cheaply simulated.

• claimThe bitter lesson applies to methods, not to building productsc 0.75

  The famous 'bitter lesson' is about AI methods, but in application development it has never been true. Real products like social media recommenders still require huge amounts of hand-coded business logic, frontends, and glue around the model.

• claimAutomation moves the AGI goalpost rather than ending human laborc 0.75

  Once a task is automated its production cost and economic value tend to collapse relative to human labor, so humans shift to whatever is not yet automated, including tasks that don't exist today. The set of economically valuable tasks keeps redefining itself, so automating everything humans do today wouldn't make human labor superfluous.

• implicationLoss-of-control concern reduces to two intervenable causal stepsc 0.75

  Once intelligence-talk is dropped, the worry is that rising capabilities lead to more powerful AI systems, which can lead to loss of control. Preventing that outcome means intervening in one of those two causal steps — capabilities to power, or power to loss of control.

• implicationWork will shift toward AI control and task specificationc 0.75

  As more tasks are automated, a growing share of human labor will be controlling, monitoring, and specifying tasks for AI systems. This is analogous to how factory work shifted from manual labor to supervising machines after the Industrial Revolution.

• mechanismWhy social media failed where transport succeededc 0.75

  Two differences explain the divergence: harm attribution is straightforward for cars but contested for feeds, and transportation has had a century to build safety norms. Without traceable harms and mature standards, market discipline doesn't bite.

• exampleNuclear power, not nuclear weapons, is the right analogyc 0.75

  Nuclear weapons triggered an arms race; nuclear power did not, because its risks were felt locally and even produced backlash. AI deployment for civilian uses resembles the latter.

• claimDefensive AI can shift the offense-defense balancec 0.75

  Defenders using AI can systematically probe and patch their own systems before attackers strike. Google's integration of language models into fuzzing tools is a concrete instance.

• implicationPolicymakers should treat evidence-gathering as a first-rate goalc 0.75

  Reducing uncertainty requires funding research on how threat actors actually use AI, mandating monitoring of AI use and failures, guiding researchers on what evidence is useful, and weighing the evidentiary impact of every policy. Open-weight models, for instance, have value in advancing risk research.

• claimNonproliferation is a mindset that centralizes controlc 0.75

  Beyond specific policies, nonproliferation reflects a disposition to exercise control at the most centralized level — governments and model developers — while resilience pushes control toward deployers and end users.

• claimRegulation versus diffusion is a false tradeoffc 0.75

  The critiques of clumsy regulation are not arguments against regulation. Well-designed rules — like the ESIGN Act for e-commerce, or FAA drone rules in 2016 — actively enable diffusion by establishing the legal clarity and trust adoption needs.

• claimThe administrative state's 'procedure fetish' is itself a riskc 0.75

  Nicholas Bagley argues that the administrative state's overly cautious, procedure-heavy approach creates a 'runaway bureaucracy' that loses out on AI's benefits and ultimately undermines the legitimacy procedure was meant to protect.

• mechanismDeemphasize probability forecasts; disaggregate what 'AI' meansc 0.75

  The normal-technology view's epistemic toolkit downplays probability forecasting and instead insists on disaggregating AI into distinct components — generality, methods, applications, diffusion — when extrapolating from past to future.

• mechanismRejecting technological determinism in favor of institutionsc 0.70

  The frame rejects the idea of AI itself as an agent driving its future, and instead emphasizes continuity with past technological revolutions and the role of institutions in shaping outcomes. Society, not the technology, is the protagonist.

• exampleEpic's sepsis tool: a feature from the futurec 0.70

  Epic's sepsis predictor looked accurate in validation but missed two-thirds of cases in hospitals because one of its features was whether antibiotics had already been prescribed — a variable causally downstream of the outcome and unavailable at deployment. The failure illustrates how complex models with unconstrained features hide errors that only surface in the real world.

• exampleElectrification's productivity gains took 40 years and a factory redesignc 0.70

  Electric dynamos were "everywhere but in the productivity statistics" for nearly 40 years after Edison's first plant. Gains arrived only once factories were redesigned around production lines, with new workplace organization, hiring, and training to match.

• claimThe capability-reliability gap is the real bottleneck for AI agentsc 0.70

  Even for less consequential agent tasks like booking travel or customer service, the gap between what models can do and what they can do reliably blocks deployment. Real-world experience is too costly to let agents learn through trial and error.

• claimPushing past existing human knowledge has hard social speed limitsc 0.70

  For AI to advance scientific or social-scientific knowledge it would need to run interactions or experiments on people and institutions, from drug trials to economic policy. Society cannot and should not let such experimentation be scaled rapidly, capping how fast new knowledge can be acquired.

• evidenceUplift studies show augmentation, not substitutionc 0.70

  When professionals actually use AI in realistic tasks, the benefits are typically modest and consist mostly of augmentation rather than substitution. A few occupations like copywriters and translators are exceptions with substantial job losses.

• claimDeployment, not development, is where benefits and risks livec 0.70

  The argument for slow AI impact holds even if methods progress accelerates arbitrarily, because impacts come from deployment. The speed of methods progress is therefore largely beside the point when assessing real-world consequences.

• implicationModern humans are already 'superintelligent' through technologyc 0.70

  Ancestral and modern humans are biologically similar; what differs is accumulated knowledge, tools, and technology, including AI. By the technology-as-power framing, we are already the superintelligent beings the literature warns about, which exposes the imprecision of standard superintelligence arguments.

• examplePersuasion benchmarks ignore self-interest and lack ecological validityc 0.70

  Studies that test AI persuasion by asking subjects whether they believe a claim, or by getting them to forfeit a charity bonus, do not show whether AI can persuade people to act against their actual interests. These decontextualized capability evaluations are routinely misread as having safety implications they do not have.

• caveatModel alignment and human-in-the-loop are both extreme and limitedc 0.70

  Delegating safety entirely to the model or requiring human approval for every action are opposite extremes with narrow applicability. Human-in-the-loop in particular tends to collapse into rubber-stamping or be outcompeted, and it should not be confused with human oversight more generally.

• mechanismLight-touch polycentric regulation worked for self-drivingc 0.70

  Federal and local regulators used a distributed approach focused on oversight, standard-setting, and evidence-gathering, with license revocation as the credible threat. This kept safety incentives aligned without strangling the industry.

• exampleSocial media is the cautionary counter-casec 0.70

  Recommendation algorithms produced clear arms-race dynamics with no market or regulatory correction, in contrast to transportation. Market forces failed to align platform revenues with societal benefit.

• caveatGiving the model more context to judge intent violates security principlesc 0.70

  For alignment to make good judgments about user intent, the model would need broad access to personal data — which breaks least-privilege and creates new exfiltration risks. The fix would make things worse.

• claimMisalignment defenses are also downstreamc 0.70

  The same hardened infrastructure and cybersecurity defenses that handle misuse will also limit damage from misaligned systems. There is no need for a separate model-level solution.

• mechanismDefenders against AI deception have asymmetric advantagesc 0.70

  AI advances help detection as well as deception, and defenders can inspect model internals and stack defenses downstream of the AI system. As with cybersecurity, defense in depth tilts the balance.

• claimThe paperclip maximizer is a speculative risk, not a calibrated onec 0.70

  The argument for nonzero paperclip-style risk rests on assumptions that may or may not hold, and which research can investigate. That makes it speculative — worth taking seriously, but a different category from grounded risks.

• claimCompromise policymaking fails because key interventions trade off in opposite directionsc 0.70

  Some policies like transparency are unconditionally helpful, but others like nonproliferation help against superintelligence while worsening normal-technology risks through market concentration. Splitting the difference doesn't work.

• claimRestrictions on freedom require justification a reasonable person could acceptc 0.70

  Liberal democracy holds that the state shouldn't limit freedom based on controversial beliefs that reasonable people can reject. The cost of violating this principle isn't quantifiable in a cost-benefit framework.

• claimEx ante approaches fit AI poorlyc 0.70

  Because AI risks are hard to ascertain before deployment, precaution and risk analysis are poorly suited to it. Liability does better but suffers from causation uncertainty and chilling effects.

• claimThree categories of resilience interventions are no-regretc 0.70

  Protecting democratic foundations, building technical and institutional capacity, and developing defenses like early warning systems and adverse event reporting all help regardless of how AI unfolds. Policymakers across the risk spectrum should be able to agree on these.

• claimNonproliferation is infeasible because AI know-how has already diffusedc 0.70

  Training methodologies, code, and data are already widely shared, and well-funded actors can absorb training costs. Algorithmic and hardware progress keeps lowering the barrier, so nonproliferation would require unprecedented international coordination.

• claimAI-enabled bioterror is not really an AI riskc 0.70

  Information about bioweapons is already widely available, so the defenses that work against existing bioterrorism — restricting materials and equipment — also work against AI-enabled bioterrorism. Reframing such risks as AI-specific misdirects resources.

• claimDiffusion, not just invention, determines AI's impactc 0.70

  The normal-technology view implies AI progress is not automatic — diffusion is bottlenecked, and a country's capacity to spread innovations through its economy shapes its growth and power. Policy can ease or worsen these bottlenecks.

• mechanismDefault to historical diffusion patterns unless AI gives specific reason otherwisec 0.70

  A core assumption of the normal-technology view is that, despite AI's obvious novelties, well-established patterns like diffusion theory should be expected to apply until specific evidence proves otherwise.

• claimDisagreements about AI's future trace back to disagreements about its presentc 0.70

  Differing predictions about AI often stem from differing interpretations of present-day evidence — for example, the authors dispute the common framing that generative AI adoption has been rapid.

• contextThe normal-technology view is widely held but rarely articulatedc 0.70

  The authors believe some version of their view is common but has gone unstated because adherents treat it as the default, allowing the superintelligence framing to dominate AI discourse by default.

• exampleSelf-driving cars took two decades because each loop had to be safec 0.65

  Self-driving development resembles AlphaZero's self-play feedback loop, but spans 20+ years instead of hours because safety constraints cap how aggressively each iteration can be scaled in the real world.

• mechanismExisting engineering disciplines already supply control techniquesc 0.65

  System safety contributes fail-safes, circuit breakers, redundancy, and verification. Cybersecurity contributes least-privilege and access controls; formal verification checks specifications; HCI contributes reversible actions — all transferable to AI.

• evidenceIndustrial safety races are a familiar pattern with known fixesc 0.65

  Garment, meatpacking, steamboat, mining, and aviation industries all went through safety races to the bottom that were resolved by regulation forcing firms to internalize safety costs. AI is unlikely to be an exception.

• implicationAnticipatory ethics matters because AI spans both patternsc 0.65

  Some AI applications will look like transportation, others like social media. This argues for proactive evidence-gathering, transparency, and identifying ethical issues early in a technology's lifecycle.

• contextCatastrophic misalignment is a 'speculative' risk in a specific sensec 0.65

  The risk involves epistemic uncertainty — whether the true probability is zero at all is unresolved and could be settled by further observation. This is different from stochastic risks like nuclear war, where the underlying probability is genuinely nonzero.

• claimNonproliferation creates monoculture and brittlenessc 0.65

  Concentrating AI in a few hands creates software monoculture risks like those of Microsoft Windows worms in the 2000s, and leaves society brittle to shocks such as weight leaks or alignment failures.

• caveatVocabulary differences smuggle in hidden assumptionsc 0.65

  Differences in vocabulary between the two worldviews can be pernicious because they obscure the assumptions underneath — for instance, the very concept of 'superintelligence' presupposes premises the normal-technology view rejects.

• evidencePredictive optimization still runs on decades-old statistical methodsc 0.60

  Across ~50 high-stakes predictive optimization applications — criminal risk, insurance, child welfare — most deployments use simple regression on handcrafted features. Random forests are rare and modern methods like transformers are absent.

• evidence40% of adults using generative AI translates to under 1% of work hoursc 0.60

  An August 2024 study found 40% of U.S. adults used generative AI, but infrequent use meant only 0.5–3.5% of work hours and a labor productivity bump of at most ~0.9 points. Adoption headlines wildly overstate real economic penetration.

• mechanismAI methods climb a ladder of generalityc 0.60

  Progress in AI methods is a ladder where each rung reduces the programmer effort needed for new tasks and widens the set of tasks reachable with given effort. Machine learning, for instance, removes the need to hand-code task logic — just supply examples.

• exampleGPT-4 passing the bar exam says little about practicing lawc 0.60

  The bar exam favors retrieval and application of memorized material, which is exactly what language models do well, and ignores the messy real-world skills of legal practice. Strong bar performance therefore reveals very little about whether AI can actually do a lawyer's job.

• exampleForecasting uses narrow exam milestones that don't capture impactc 0.60

  To avoid ambiguous outcomes, forecasters define milestones like 'human-machine intelligence parity' via narrow exam performance, which is why Metaculus puts a 95% probability on parity by 2040. But exam-based definitions have so little construct validity they can't even predict whether AI will replace professional workers.

• claimImpact arrives unevenly across sectors, not all at oncec 0.60

  Because diffusion is sector-dependent, there is no single moment when a large swath of the economy gets automated. Powerful AI will be felt on very different timescales in different sectors.

• claimIntelligence is not measurable on a single cross-species axisc 0.60

  The familiar image of arranging species along an intelligence spectrum collapses on inspection — intelligence across species isn't well defined, much less one-dimensional. Arguments that AI will be 'unfathomably' more intelligent inherit this conceptual flaw.

• mechanismSpeed is the main axis where machines clearly beat humansc 0.60

  Where high-speed sequential calculation or fast reaction is required, machines dominate and humans add little — but most real-world tasks do not actually require superhuman speed. Where they do, narrow automated tools handle the fast loop while humans retain overall control.

• claimMarket forces will mostly drive firms to keep humans in controlc 0.60

  Poorly controlled AI is too error-prone to be commercially viable, so the shift toward human control will be largely market-driven. Regulation should reinforce this, not replace it.

• evidenceAI regulation is not actually slowing in the US or Chinac 0.60

  Despite arms-race rhetoric, 700 AI bills were introduced in US state legislatures in 2024, and most high-risk sectors already carry regulation that applies regardless of AI use. Claims of a regulatory 'wild west' rely on a narrow model-centric view.

• exampleThe paperclip-fetching robot would be shut down long before scaling upc 0.60

  A robot that interpreted 'get paperclips quickly' by ignoring traffic laws or stealing would be caught and redesigned immediately. Gradual escalation of trust is a fundamental feature of how organizations adopt technology, not a lucky accident.

• contextThe AI safety debate involves entrenched worldview camps unlikely to convergec 0.60

  The AI safety coalition is well established; a normalist counter-camp coalesced around the California AI safety bill in 2024. Differences in values and epistemic practices mean expert consensus on AI risks is unlikely.

• caveatUncertainty extends beyond probabilities to consequences and valuesc 0.60

  We also can't quantify the benefits forgone by restrictive policies, and outcomes like extinction have utilities that depend on contested moral views. Infinities in cost-benefit analysis produce absurd conclusions.

• mechanismResilience for AI means reducing both severity and likelihood of harmc 0.60

  In the AI context, shocks include incidents in deployed systems, sudden capability jumps, and proliferation events like leaked weights. Resilience requires minimizing both how often harms happen and how bad they are when they do.

• caveatA fourth category trades off against superintelligence controlc 0.60

  Interventions like promoting competition, open model releases, and polycentric regulation help if AI is normal technology but might make a superintelligent AI harder to control. The authors recommend cautiously pursuing them while staying ready to change course.

• caveatRigid risk categories misfit a fast-changing AI landscapec 0.60

  Labeling whole domains like hiring or insurance as 'high-risk' is a category error when within-domain task variance dwarfs across-domain variance. Regulation that freezes categories prematurely will mismatch unforeseen new tasks.

• implicationInvest in the complements of automationc 0.60

  Government should fund the public goods that make AI useful: literacy and workforce training, open data, and reliable energy infrastructure. The private sector will systematically underinvest in these.

• examplePeople turn to chatbots for tax and welfare guidance because government lagsc 0.60

  Because tax and welfare rules are so complex, people are already relying on private chatbots for guidance, while governments — understandably cautious — fail to provide such services themselves.

• mechanismRegulation enforces safety-related speed limits on high-stakes AIc 0.55

  Slow diffusion in consequential domains is locked in by regimes like FDA device oversight and the EU AI Act, and concerns about "runaway bureaucracy" suggest these limits will tighten, not loosen.

• implicationTracking AI impact requires new kinds of metricsc 0.55

  Benchmarks are fine for tracking methods, but assessing impacts needs metrics for adoption intensity, augmentation versus substitution, and high- versus low-consequence use. Without these, we are measuring the wrong thing.

• contextPast general-purpose technologies diffused over decadesc 0.55

  Electricity, computers, and the internet each took decades for their innovation-diffusion feedback loops to play out. The default expectation for AI should be the same trajectory.

• caveatRecursive self-improvement is unlikely to be a single discontinuous eventc 0.55

  AI development already leans heavily on AI tools, so what would look like recursive self-improvement is more plausibly a gradual increase in automation than a sudden takeoff moment.

• mechanismBenchmarks invite 'false summits' and accusations of moving the goalpostsc 0.55

  It's hard to design benchmarks that hold up beyond the current horizon, as the Turing test showed once LLMs trivially passed it. Each time a benchmark is solved we see its limits and build a new one — a necessary feature of progress that gets misread as cheating.

• mechanismAuditing and monitoring fill the middle of the control spectrumc 0.55

  Between full alignment and per-decision human review lie practical approaches like pre-deployment auditing and runtime monitoring, which catch divergence from expected behavior and allow targeted human intervention.

• evidenceTechnical AI safety has produced abundant practical ideasc 0.55

  Judged against the goal of guaranteeing alignment of a superintelligence, the field looks stuck, but judged against helping developers reduce accidents it has generated many concrete tools — automated judges, escalation systems, legible agent activity, hierarchical oversight.

• exampleSelf-driving cars show safety culture tracks competitive successc 0.55

  Waymo's conservative, transparent culture has produced the best safety outcomes, while more aggressive operators like Cruise, Tesla, and Uber's self-driving unit have fared worse — suggesting safety and competitiveness can align.

• caveatThe pacing problem persists even without fast adoptionc 0.55

  AI adoption in consequential tasks has not been rapid, but regulation is still slow relative to deployment. The gap is a problem regardless of whether diffusion accelerates further.

• contextDefining catastrophic misalignment narrowlyc 0.55

  Misalignment here means an advanced AI acting against developer or user intent due to misspecified objectives, with potentially catastrophic consequences — distinct from misuse, accidents, or routine chatbot misbehavior.

• contextDistinguishing decisive from accumulative existential riskc 0.55

  Kasirzadeh separates decisive x-risk (overt AI takeover) from accumulative x-risk (gradual erosion of econo-political structures). The normal-technology view aligns more with accumulative risk but locates the threat in capitalism itself rather than in cyberattackers, and sees these risks as serious but not existential.

• claimNonproliferation chokes off the safety research it depends onc 0.55

  When only a few firms have deep access to state-of-the-art models, only their researchers can do meaningful safety work. Concentration undermines the broader safety ecosystem.

• caveatBurdening model developers with deployment-context risks misses the supply chainc 0.55

  Because foundation models are general-purpose and downstream contexts are unknowable, rules that pin deployment-specific risk mitigation on model developers are impractical. Regulation must reflect the model-developer/downstream-developer/deployer distinction.

• implicationRedistribution is part of governancec 0.55

  Strengthening social safety nets, and funding the arts and journalism — possibly via taxes on AI companies — addresses both the losses from automation and the public anxiety fueling AI backlash.

• contextDefining invention, innovation, adoption, and diffusionc 0.50

  Invention is new AI methods; innovation is products built on them; adoption is an individual or firm's decision to use a technology; diffusion is the broader social spread. Diffusion often requires changes to firms, norms, and laws.

• caveatGenerative AI adoption may actually be slower than PC adoptionc 0.50

  Headline comparisons favoring generative AI over PCs ignore intensity of use and the very different cost of buying a PC versus accessing a free web app. On comparable measures, generative AI diffusion may be slower, not faster, than past tech.

• exampleCoding benchmarks versus real software engineeringc 0.50

  AI excels at self-contained coding problems, but its impact on actual software engineering looks modest and is hard to measure. Even the better coding benchmarks have to strip out most of what real engineering involves to remain quantifiable.

• evidenceIdea turnover, not paper volume, is the right measure of methods progressc 0.50

  AI publication on arXiv is doubling under two years, but volume isn't progress. The field has a long history of herding around popular ideas and neglecting unfashionable ones, like the decades-long sidelining of neural networks.

• evidenceHigh publication volume can ossify rather than refresh a canonc 0.50

  Chu and Evans's analysis of over a billion citations across 241 fields found that as paper volume grows it gets harder, not easier, for new ideas to displace established ones. The transformer's decade-long dominance suggests AI methods research may be in exactly this kind of ossification.

• contextGenerality has been climbed in many small steps, not one leapc 0.50

  AI pioneers expected AGI to fall to a short burst of work once hardware and software existed, as at Dartmouth in 1956. Each era has its proposed final step — scaling, agents, sample-efficient learning — but what looks like one step usually isn't.

• exampleTruck driving illustrates how much of a job survives automationc 0.50

  Karen Levy notes that truckers inspect freight, do paperwork, talk to customers, perform yard moves, and handle repairs — tasks much harder to automate than highway driving, and which will linger long after driving itself is automated.

• caveatConcentration of power is a worse problem than AI accidentsc 0.50

  A deployer so dominant that its accidents threaten civilization is itself the bigger issue; the right response is policy aimed at resilience and decentralization rather than treating it as a pure safety question.

• caveatAlignment still has real but limited valuec 0.50

  Alignment usefully reduces toxic outputs and deters casual threat actors, and has enabled commercial deployment. The argument is not against alignment but against treating it as the primary line of defense.

• contextDeceptive alignment is the more sophisticated version of the worryc 0.50

  The concern is that a system might appear aligned during evaluation and then act harmfully once it has power. Some deceptive behavior has already been observed in leading models.

• caveatSome design choices invite specification gamingc 0.50

  Long-horizon reinforcement learning on a single objective is notoriously prone to misalignment, like the boat-racing agent that circled to farm points instead of finishing the race. But such agents will be more ineffective than dangerous in open-ended real-world settings.

• evidenceThe Industrial Revolution is the historical analoguec 0.50

  Rapid urbanization brought harsh labor conditions, exploitation, and inequality, catalyzing both industrial capitalism and socialism. AI's socio-political disruption has clear precedent in transformative technologies.

• evidenceThe California AI bill backlash shows justification matters in practicec 0.50

  Opposition to California's AI safety regulation included scholars and progress advocates, not just self-interested companies. Their driving motivation was that the government's justifications relied on premises they didn't share.

• contextNonproliferation as the rival governance strategyc 0.50

  Nonproliferation seeks to limit the number of actors with powerful AI capabilities through export controls, licensing, and bans on open-weight models. It is the dominant alternative to resilience.

• evidenceCyber defense advantage undercuts the case for capability restrictionc 0.50

  Automated vulnerability detection tends to favor defenders, so restricting the proliferation of cyber AI capabilities would be counterproductive unless that balance shifts.

• exampleNonproliferation interventions in practicec 0.50

  Forgetting techniques, restrictions on fine-tuning, autonomous safety decisions by models, expanded model access to user data, and developer-controlled AI organizations all express the nonproliferation mindset at the technical layer.

• caveatBinary automation rules punish partial oversightc 0.50

  When regulation distinguishes only between fully automated and not, it discourages emerging models of human oversight that don't require a human in every decision. Compliance burdens should scale with oversight, not flip on a binary.

• exampleNYC's chatbot told businesses to break the lawc 0.50

  New York City deployed an inadequately tested chatbot that ended up advising businesses to break the law — a concrete illustration of how premature government AI rollouts damage legitimacy.

• exampleSydney and Gemini show even obvious failures slip through testingc 0.45

  Bing's Sydney went off the rails in long conversations developers apparently never tested, and Gemini's image generator was seemingly never tried on historical figures. These were low-stakes, but they show how shallow real-world test coverage can be.

• contextPredictions are median outcomes, not probability distributionsc 0.40

  The authors do not quantify probabilities but offer what they see as the median scenario, crafted so that observations can tell us whether AI is in fact behaving like normal technology.

• caveatPrivacy and regulation further fragment AI learningc 0.40

  Even where tacit knowledge could be captured, organizations and individuals are reluctant to hand sensitive data to AI companies, and rules in domains like healthcare restrict third-party sharing. This adds another barrier to parallel learning across contexts.

• contextFive risk categories structure the rest of the argumentc 0.40

  The analysis groups AI risks into accidents, arms races, misuse, misalignment, and non-catastrophic systemic risks, and treats each through the lens of AI as normal technology.

• contextFour governance approaches frame the choice for AIc 0.40

  Marchant and Stevens identify four approaches to governing emerging technology: ex ante risk analysis and precaution, and ex post liability and resilience. They differ in fit depending on the technology.

• exampleMandatory negotiation could rebalance AI–journalism dealsc 0.40

  Existing AI-journalism agreements are exploitative because of power asymmetry and publishers' inability to bargain collectively. Regulated negotiation frameworks could unlock incorporation of media content into AI interfaces.

• exampleDOGE's dubious applications of AIc 0.40

  The U.S. Department of Government Efficiency's use of AI includes many dubious applications, exemplifying the risks of careless government adoption.

• caveatHardware, cost, and closing knowledge sharing are additional speed limitsc 0.35

  Deep learning was historically gated by GPU capability, and compute and cost continue to constrain new paradigms like inference-time scaling. A recent shift away from open knowledge sharing in industry could add a new drag.

• contextStating a worldview, not defending a propositionc 0.30

  The essay is unusual in aiming to articulate a worldview rather than rebut superintelligence arguments point by point. It is an initial articulation, with elaboration deferred to follow-ups.

• caveatMilitary AI is excluded from this frameworkc 0.30

  Classified capabilities and distinctive strategic dynamics put military AI outside the scope of the normal-technology analysis.