Atomic Razor

AI Is for...

As Bill Gibson opined "...the street finds its own uses for things." As we know the diffusion of both video recorders and the internet (indeed, earlier of motion picture technology) was driven by certain common human desires. So, it will inevitably be with A(G)I. This might not be something that DeepMind or Anthropic want to discuss, but it's going to be a major driver in the adoption of the tecxhnology. Infinite personalisation means that this is going to be huge. And, yes, there's lots of ethical implications of many different sorts. (Note that OnlyFans generated ~$6.5 billion in total revenues in 2023.)

Furthermore, just as people aren't going to stop consulting Dr Google, they are not going to want to stop talking to their agentic shrinks. We might prefer to spend a weekly 50-minute hour with a meatspace counsellor, but what about the other 167 hours and 10 minutes. Or just when you are having a crisis at 3 AM on Wednesday morning? 

People aren't going to want to stop "cheating" on their homework. If you're a company and you're dropping $200/month on your favourite FM, you don't need to be getting much value out of it for it to be worth $2400/year. In Seattle, to get an assistant who is in any way useful is probably going to cost you at least $75/hour in fully allocated costs, so if your AI is providing 3 hours a month of value you are ahead of the game. A programmer on $150k/year is probably costing you anything up to $450k fully absorbed, so a 10% increase in productivity would be worth about $45k or $4k/month. A 10% increase is hardly Freed Brooks's 10x silver bullet, but doesn't seem totally unreasonable. I know from my personal experience last year that for the kind of proof of concept work I was doing I was probably able to do it in half the time it would normally take and this wasn't vibes coding, just using the FM instead of Stack Overflow. It's going to depend very much on the kinds of projects you are working on, but it's not as though I am the only person to have ever done discovery-based programming, so I wonder what it would be like now.

So, even leaving aside people who say they never use AI (you don't have Waze on your iPhone?), there do seem a number of domains that are likely to continue being explored even if we never do get the Drop-In Replacement Worker or the Data (Centre|Country) of (Super)Geniuses. 

"Doctor Who" and "Doctor Who and the Silurians"

What is the name of the second story of the Pertwee era? The FoAK article is called "Doctor Who and the Silurians", but there's a section stating

This was the only time the name "Doctor Who" was used in the title of a serial on-screen (although Episode 5 of The Chase was titled "The Death of Doctor Who" on-screen and at the end of The Gunfighters, the caption read "Next Week Doctor Who and the Savages"). Although it was common in production paperwork to prefix "Doctor Who and..." to the story title at the time, the prefix was usually dropped when the director ordered the titles from the captioning department for transmission. However, this was not done for this particular story.^[2]

The reasons why this happened are not entirely clear. Director Timothy Combe states that he was presented with a story called Doctor Who and the Silurians and that it was always intended that the serial go out with that name. However, as Doctor Who historian Andrew Pixley points out, this was Combe's first serial as a full director and there was effectively no producer at this time, as noted above. In addition, the rehearsal scripts for the serial simply have The Silurians as the title. Pixley theorises that Combe was unaware of the standard production practice and gave the order to the captioning department for the "proper" title, as he believed it to be at the time.

Whatever the case, production paperwork from this point on stopped the practice of adding the prefix, perhaps as a measure to prevent the "mistake" from happening again.

It seems fairly clear that if Combe had been under adult supervision, the story would have been called "The Silurians". It's possible that Malcolm Hulke's original scripts called it "Doctor Who and the Silurians", but the rehearsal scripts didn't. But I mean it says "Doctor Who and the Silurians" right up there on the screen and the screen is surely the canonical source of ground truth for television. That's surely what the estimable Mr Cooray Smith would say, indeed, does say in a recent essay. But the title is tautologous. No-one doubts that the show is called Doctor Who. So, what's the point of calling the story "Doctor Who and the Silurians"? We already know it's Doctor Who! Well, it could be to emphasise that the story is focussed on the relationship between the Silurians and Doctor Who. There's a couple of issues there. The first is that it isn't about either the relationship between the Doctor, the character, and the Silurians or about the relationship between the show, Doctor Who, and the Silurians. We might see some DW fans in a 2025 episode, but it was a bit too meta for 1970, although we do see the Doctor stalking off the set at the end of the next story, "The Ambassadors of Death". It's not quite the end of Gangsters, but who knows it was an influence. But, of course, my second point is, who is this Doctor Who, if it's not the same of the show?

Episode 5 of "The Chase" is called "The Death of Doctor Who". Pertwee is credited as Doctor Who. The scripts probably called him DOCTOR WHO. "Knock, knock!" "Who's there?" "How did you know?"/"Knock, knock!" "Who's there?" "The Doctor." "Doctor who?" "How did you know?" The annuals call him Doctor Who., the comics probably did. In the films, Peter Cushing is called Doctor Who. And yet, and yet... The Doctor never calls himself Doctor Who (OK, the Doctor disguises himself as Doktor von Wer in "The Highlanders"; note that he is ennobled a couple of years before "The War Games" revealed he was a Time Lord). No-one except for WOTAN in "The War Machines" calls the Doctor Doctor Who (OK, Missy maybe when she's messing around). And that's quite a remarkable bit of continuity. From "An Unearthly Child" to "The Reality War", for more than 62 years, the Doctor is always, in multiverse, the Doctor, the definite article you might say.

 

The Hundred Million-Dollar Question

"Meta is reportedly investing nearly $15 billion in the data-labeling firm Scale AI and taking a 49% stake in the startup, while also bringing on CEO Alexandr Wang to help lead a new “superintelligence” lab within the company."

More eyebrow raising

"Zuckerberg's hands-on involvement is unprecedented. He's reorganized Meta's Menlo Park office so the new team sits near him and is offering compensation packages between seven and nine figures (whoa) to poach talent from OpenAI and Google."

(!)

And people wonder why I am depressed! I should have done that MSc in AI in 2004-5 rather than the MBA. The problem is that I couldn't have got a loan to do an AI MSc whereas I could an MBA. Plus , in 2005, there weren't many actual AI jobs around and the chances of getting one even with a MSc in AI would have been pretty slim. So, having an MSc in AI would benefit me (a bit), but not really then. Although it would have been fun to do. I have done lots of Coursera courses and some Udacity stuff, but I don't know far it gets you. I have the NVIDIA Jetson Orin Nano now, so I can do some real stuff that I post to GitHub. I should have done that months ago. The DGX Spark will have 15x the grunt.

But who's getting $100M/year? These aren't tech billionaires as such. Get a first in CompSci at Cambridge (2012), do a PhD at the UCL Gatsby Unit, join DeepMind (2016), transfer to Google Brain on an L-1B (2018), get an O-1 visa, join OpenAI (2020), spent the last 5 years doing cutting edge research. That would make you about 36 now (assuming you had a year off to intern at Microsoft Research in Cambridge). I'm 57 and I was never that good at maths.

So, you've probably been leading a team at OpenAI (or equivalent company) for half a decade or so. You are probably as much a manager as a researcher. Your presence presumably helps to recruit and retain others, most of whom are probably going to be 10 years younger and on seven figure salaries. Still, this is utterly remarkable in its way. The economic justification is that you are only a handful of $100M salaries to the team leaders, most people will be on closer to $1M with a tranche at $10M or so. But that's not chump change. Even if you are paying the thick end of a billion dollars a year in salaries (and x3 that for the compute), it's still worth it you can get to ASI in 2 or 3 years. In terms of proto-AGI, there's clearly a lot of scope for different models to catch up with what other models already do better. ASI, in principle, could eat everyone's lunch, the old AGI/SL4 list idea of a singleton or monolith, that you can only have ultraintelligence because ve will simply out-compete any rivals. I don't know about that, but then I guess you drone strike the opposition's air-gapped data centre or penetrate using more traditional humint methods so that Meta pwns OpenAI and the US pwns China (or vice versa) and Zuckerburg ends up caesar at least until he pwned by his own ASI. 

But is a $100M enough? You can't take it with you. In the long run we're all dead, but that long run might be quite a short one right now. The default assumption is that 2035 will look like 2025 only more so. But 2025 looks nothing like 2015 in a lot of ways (Brexit, Tr*mp I, electric cars, COVID, mRNA, Ukraine, proto-A(G)I, Gaza, Tr*mp II, the end of the post-WWII/post-Cold War dispensation, the Return of History), so I am not sure at all that that works anymore. If you really wanted to survive until 2035, you would probably want to make some pretty extreme precautions and how much are they going to cost and how much are they going to have to pay you to live in a nuclear submarine for the next few years?

But, of course, this was what I wanted. We are getting it, it seems. And there is nothing I can do, but be a bystander as we hurtle into history. And there's probably nothing I could ever have done. Even if I had done a PhD in AI in the late 1980s/early 1990s, I just wasn't good enough at maths and research to make it as an academic researcher. I wasn't going to be Murray Shanahan even if I could have written his 2015 book. There's the whole futurologist angle, which is what I said I wanted for so many years and this blog is, I suppose, an actual suppose, an actual instantiation, over 21 years, of that ambition. It's not nothing, but I wonder if there ever were a time that I could have leveraged it into something more. It seems hard now to imagine that it's not later than we think, but perhaps 90 will be the New 50.

The Fundamental Interconnectedness of All Things

I am reading the Hugo nominated novels and one of them is T. Kingfisher's A Sorceress Comes to Call, which is a fantasy mashup of a murder mystery (though we know who did it) and a Regency romance. Obviously, these are channelling Jane Austen at some leve, although, of course, as the FoAK opines

These derive not so much from the 19th-century contemporary works of Jane Austen, but rather from Georgette Heyer[.]

And, of course, all the writers who come after Heyer, although it's not as Austen isn't a direct influence and think of all the sequels other hands and retellings of Austen. But, anyway, this got be thinking about who influenced Austen. Obviously, there were a lot fewer books in those days and I got to thinking back to the days when I was reading my way through the canons of English literature. Samuel Richardson, of course (not read him), but Sterne, Tom Jones, the Gothic novels, probably Defoe and Swift, Pilgrim's Progress.  But, of course, there are lots of books and writers that are forgotten now. So, I asked Gemini, who wrote me a convincing enough response.

The Literary Architects: Writers Who Shaped Jane Austen's Genius

Jane Austen, a titan of the English novel, was not an artist in a vacuum. Her celebrated works, known for their sharp wit, social commentary, and enduring romantic plots, were built on the foundations laid by a diverse group of 18th-century writers. Austen was a voracious reader, and her novels are a testament to her engagement with the literary trends and titans of her time. Key among her influences were the pioneers of the novel form, particularly Samuel Richardson and Henry Fielding, the sharp social observations of fellow female novelists like Fanny Burney and Maria Edgeworth, and even the Gothic romps of Ann Radcliffe, which she famously satirized.

The Forefathers of the Novel: Samuel Richardson's mastery of the epistolary novel, which tells a story through letters, profoundly shaped Austen's narrative technique. His exploration of the inner lives and moral dilemmas of his characters, particularly in "Pamela" and "Clarissa," provided a model for Austen's own deep dives into the consciousness of her heroines. While Austen moved beyond the purely epistolary form, the focus on character psychology and the slow unfolding of relationships through correspondence are hallmarks of her work. In contrast, the robust, satirical, and socially panoramic novels of Henry Fielding, author of "Tom Jones," offered a different kind of blueprint. His use of an intrusive, witty narrator and his broad social critiques can be seen as precursors to the narrative voice and satirical eye that Austen would perfect.

The Sisterhood of the Pen: Perhaps the most direct and personal influences on Austen were the women who were her contemporaries and immediate predecessors. Fanny Burney, in particular, is often cited as a crucial figure. Her novels, such as "Evelina" and "Cecilia," were immensely popular and explored the challenges faced by young women navigating the complexities of society and the marriage market. It is from the final chapter of "Cecilia" that Austen is believed to have drawn the title for her most famous work, "Pride and Prejudice." Burney's talent for capturing realistic dialogue and her focus on the domestic sphere as a site of significant drama undoubtedly resonated with Austen.

Maria Edgeworth, an Irish novelist, was another contemporary whose work Austen admired. Edgeworth's novels, often set in her native Ireland, were lauded for their social realism and moral purpose. Austen even mentioned Edgeworth by name in "Northanger Abbey," a clear nod to her respected contemporary.

The Gothic Interlude: While she is famous for her realistic and grounded novels, Jane Austen was also a keen observer of the sensationalist Gothic fiction that was wildly popular in her youth. Ann Radcliffe's "The Mysteries of Udolpho" is famously parodied in "Northanger Abbey," where the naive Catherine Morland's expectations of life are hilariously shaped by her consumption of such thrilling tales. However, Austen's satire is not merely dismissive; it demonstrates a thorough understanding of the genre's conventions and an appreciation for its power to captivate readers.

The Bard and the Satirists: Beyond the realm of the novel, the influence of William Shakespeare is evident in the intricate plotting and witty banter of Austen's works. Her characters often quote or allude to Shakespearean plays, and the comedic and romantic structures of her novels owe a debt to his enduring dramatic forms. Furthermore, the broader satirical tradition of the 18th century, exemplified by writers like William Cowper and the playwrights Richard Brinsley Sheridan and William Congreve, helped to hone Austen's own sharp satirical edge. Their critiques of societal follies and affectations provided a fertile ground from which her own incisive social commentary could grow.

In conclusion, Jane Austen's genius was not born in isolation. It was nurtured and sharpened by her deep engagement with the literary landscape of her time. From the foundational structures of the novel laid by Richardson and Fielding to the shared concerns of fellow female writers and the popular trends she both admired and satirized, Austen's work is a rich tapestry woven from the threads of her literary predecessors and contemporaries.

Anyhoo, I was looking at the daily BookBub bargsain book email and there was a link to a cheap copy of Printer's Error: Irreverent Stories of Books History. Of course, I love books about books and the first chapter is called "How to Forge a Rare Book", so that was a must. I noticed that one of the writers, Rebecca Romney, had a recent book out called  Jane Austen's Bookshelf: A Rare Book Collector's Quest to Find the Women Writers Who Shaped a Legend, which sounds fascinating, but I will wait till I can get it as a bargain, bit it was another example of a resonance in the multiverse given that I had been thinking about exactly that topic the day or so before. 

Anyhoo, I was in PCC and M was getting some stuff and I was looking at the magazines (why don't supermarkets sell lots like in the UK?) and there was a copy of Psychology Today and there was some story about "7 Surprising Differences Between Men and Women", although I don't the article had 7, which seemed odd, or that they were that surprising, but there was also a Q&A profile of Rebecca Romney. Well, I had never heard of her until that morning and here she is talking about the importance of a particular copy of a Ursula K. Le Guin novel to her. So, she's a fan as well as book dealer/collector,. (See her Bluesky where's she's reposting Locus). But it turns out that I had come across Rebecca before because was (is?) the book expert on Pawn Stars, which I used to watch when I lived in Preston and worked in the dining room in front of the telly! This is exactly the kind of thing that I want to discuss in Little, Big. There's always more and all things are fundamental interconnected.

All Human Knowledge

My first PC in October 1991 had a 44 MB hard disk. If we think the average book has perhaps a 100,000 words that's about 500,000 characters or 500 kB. But there's formatting and pictures, but also lots of ways to compress text and images. (Remember DOS 6?) So, perhaps 0.5 MB per book, although I have always tended to round it up to 1 MB. My 1991 PC then could store 44 books, say. By January 1996, I had a a CD-ROM and a few hundred MB on my hard disk. A CD-ROM is 650 MB, let's say 1000 books. I have a lot more than on my British Fire. Gemini informs me that

[T]he amount of data a Blu-ray disc can store ranges from 25 GB to 128 GB, depending on its specific format and number of layers.

That's between at least 25,000 and 200,000 books or a small research library. I mean I wonder how long your Blu-ray player and quadruple-layer BDXL disc might last. That would capture a fair fraction of human knowledge. My NVIDIA Jetson Orin Nano has a 64 GB SD card and a 1 TB SSD. 1 TB is 1-2 million books. University of Washington libraries have about 9.8 million items, but there's probably a lot of duplication there or in the British Library or Library of Congress at getting on for 200 million items. The English language Wikipedia passed 7 million articles a few weeks ago ("The English Wikipedia has reached 7,000,000 articles with Operators and Things, a 1958 book about a woman's journey with schizophrenia, created by User:Therapyisgood on 2025-05-28 at 02:26 UTC").

An image estimating the size of a printed version of Wikipedia as of December 2023, ignoring images. Source: https://en.wikipedia.org/wiki/Wikipedia:Size_of_Wikipedia#/media/File:Size_of_English_Wikipedia_(22.14_GB).svg

As of 2025-06-01, there are 4,902,636,156 words on about 63 million pages. 5 billion words is about 5000 books, which is a lot fewer than I have and there's a lot of redundancy in there. 

In early 2015, Michael Mandiberg published the English Wikipedia in 7473 volumes of 700 pages each via Lulu, an online e-book and print self-publishing platform, distributor, and retailer.^[13]

This page looks very interesting. The whole of English Wikipedia would fit into about 25 GB, sans images, 2.5% of my Jetson SSD. Of course, as the All Human Knowledge page acknowledges, the English Wikipedia doesn't contain anything like all of human knowledge and wisdom (and there's a lot of blather in there), although it does contain a fair sample of it. You could probably get a reasonable representation of all human knowledge into a few gigabytes, which is a trivial amount of storage in 2025. An FM could assimilate non-English language Wikipedia too. I wonder how more knowledge you would get from all of that, probably a factor of 2-10x. So, training a FM on the whole of Wikipedia might be plausible with my NVIDIA DGX Spark Grace Blackwell AI supercomputer. In The Mundanomicon, the Eschaton is instantiated through training on the volumes housed in the 1914 Harris Library in Preston. Of course, that's mere fiction, but it would be interesting to train an Edwardian bot on the 1911 Encyclopedia Britannica. That could be another project. There are so many. But my basic point is that whereas in 1991, you would struggle to store a reasonable summary of all human knowledge on your home computer, that's certainly not the case in 2025., nor arguably for 30 years given the EB on CD-ROM. But what can we do now with all this accumulated knowledge?

When was AGI?

Jasmine Sun has a really interesting Substack about AGI and other topics and a few weeks published a post that got me thinking: when was AGI? She's talking to an SF AI researcher who says

I use the classic definition of AGI, instead of moving the goalposts. The “G” in AGI is about generality. There are two types of AI: narrow and general. We used to train models for specific tasks. For every task, you’d build a huge, specific data set and train a model just for that task. That changed with the alpha version of Claude I was using in the summer of 2022. It could do any task you asked it to do. It was fucking crazy. It didn't care whether it had specific training data. It was general while prior models were narrow.

I have often suggested that FMs are proto-AGI. But if we were to go back to me in 2004 or 1986 2022 ChatGPT or June 2025 Gemini 2.5 Pro and it would have seemed like AGI. Yes, they get stuff wrong, but they are robust in that you can interact with them in natural language, which they understand, and reply in natural language and do lots of useful things. I do some work for Outlier.ai, which is linked to Scale.com, trying to stump the model, but it is next to impossible. Certainly, it would have no problem getting a starred first in physics at Oxford in 1989 (special dispensation for the lab work; obviously, it would do, unlike me, the theoretical physics paper, the graveyard of many a first, although we have humanoid robots now). Although FMs are much capable in summer 2025 than summer 2022, you could argue that they passed the AGI threshold by the time of ChatGPT's release in November 2022. Probably then, in terms of what researchers in frontier labs had access to, AGI was achieved some time in 2021.

For certain values of AGI, of course. It's not just that we don't want to say we have achieved AGI for marketing reasons, it's also that we can see that there's plenty of road to go in logarithmic space. Sir Demis doesn't think AGI has arrived yet and that's because we don't, for instance, have the data (centre|country) of (idiot savant) (super)geniuses or the drop-in remote worker. A solution to the Riemann Hypothesis or the discovery of a room temperature superconductor would actually be examples of ASI. It's not as though we don't think people are GIs despite most people not even knowing or caring what those things are. I use Gemini 2.5 Pro and it feels like talking to a person, but it's not like I have personalised amanuensis quite yet.  Gemini is still a disembodied intelligence Out There. Within a year or two, if we are spared, we will have AGI best friends that are indisputably people and then it is going to be hard to deny we have reached AGI. And a year or two, after that? I guess we are probably all going to get killed in a grey goo incident, but who knows? Perhaps we might get something expectedly unexpectedly cool like a Theory of Everything. It would be something. Even more so it it enabled FTL... 

The Ear of the Beholder

Someone with the handle musicologyman recently opined on Bluesky

‪musicologyman‬ ‪@musicologyman.bsky.social‬

I’m confident that generative AI will never produce a symphony worth hearing. The result will undoubtedly be technically assured, but aesthetically and historically bankrupt.

and

‪musicologyman‬ ‪@musicologyman.bsky.social‬

Writing a great piece of music requires life experience & a sensitivity to so many aspects of an individual’s social, historical, and psychological situation that it would be virtually impossible for an AI to replicate.

"[V]irtually" there is doing a lot of heavy lifting, but is it true? Beauty certainly exists in nature. The tree I can see out of window of the library or my study is beautiful, contrasted today against the pure blue sky. In the library, the high windows frame the tree.

 

So, found beauty is a thing. A sunset, a flower, the sky at night, a pebble, the Mandelbrot set, a Game of Life pattern, a lava lamp, the swirling streamers of Jupiter, a galaxy. Some of those we weren't evolved to find beautiful and some of them require a great deal of craft to find. As Del asked do we like fractals because we like trees or do we like trees because we like fractals? But there's no denying that beauty can be both found and generated by various kinds of systems.

 

No-one is also going to deny that bird music can be beautiful too. A lot of music is samey. If you are writing a 100 symphonies, you are probably going to repeat stuff. After Beethoven, there's more pressure to be original, but a lot of music sounds like other music. As we have said, there's have been generative music systems since the C18th at least.

 

musicologyman allows that an AI's symphony would be "technically assured", which seems fair, but that it would be "aesthetically and historically bankrupt". "Historically bankrupt" seems a strong claim. After all, the AI would be trained on all (well, a lot of) the symphonies ever written and would develop its own taste. After all, where do our tastes come from? "[A]esthetically [...] bankrupt" seems a more reasonable claim, but might not be. "Writing a great piece of music requires life experience & a sensitivity to so many aspects of an individual’s social, historical, and psychological situation", but the problem with that kind of claim is that we can't quantify a person's "life experience" or their  "sensitivity" to the aspects of their "social, historical, and psychological situation". Haydn wrote a lot more symphonies that Beethoven and under a very different mode of production. But they are both great composers. We might prefer Beethoven or feel that he's able to be his troubled artistic genius self. Haydn wouldn't have been Haydn if he had come after Beethoven. 

 

You can't weight a person's soul. Note that composition is something that is taught. It's not like writing, something you are just supposed to be able to do it. We might be suspicious of the creative writing M(F)A, but not of studying composition. But, yes, a teacher might feel their pupil needs more life experience that a 22, their work is gauche and jejune and that they need a few unhappy love affairs to be able to fulfill their talent. But that isn't something you can quantify. You might be a better composer at 32 or 42, but I'm 57 and there's only so far you can go. Ultimately, like blind-tasting wine, we might feel that it is cheating because in the way the big Californians are flattered in the tests might be the same for AI-generated music. "If I had known this was AI, I wouldn't have liked it as much." I mean it's not as though anyone is writing great symphonies right now as it is. My hope has always been that AI can open up new vistas, whether in music or writing or art or mathematics or science, as it did with go. Let's see what it can do. Brian Eno sure to approve. 

 

 

 

Is A(G|S)I Denial the COVID/Climate Denial of the Left?

People are losing their shit over the Seattle Worldcon using AI to vet people. I am not really how much people should be being vetted, historically whoever was organising the programme would just assign people they knew or trusted to appropriate items. That seemed to work fine for decades. I guess there are a lot more programme items these days that at, say, Conspiracy in 1987, which probably had three programme streams plus stuff. I think it was the one that had the Third Programme (boom, boom!). Definitely had a film programme, which you don't seem to get these days. Paul and I went to see The Right Stuff. I mean given that a lot of fans are (a) profoundly irksome; (b) right-wing, it's hard to know what you are vetting for. Some people might be persona non grata, for some reason (Dave McCarthy), but what about the puppies, say? I suppose you could argue they launched a direct attack on the Hugos, but then they are hardly the only ones in recent years. There's the case of people being fraudulent and you might want to avoid having worst enemies on the same panel. A difficulty in 2025 is that there are several polarising issues about which things could kick off.

One of those is AI. So, people want the convention to pledge not to use "LLMs". Of course, it would be extremely silly to specify "LLM" specifically. Or even AI. As David out, how would you even know you weren't using AI? And that's going to be even more true in 2 years and then in four. It's like saying in the early 1990s, don't use email or the web. But the real problem is that a lot of dislike AI. Of course, I am booster. But we are not going to have a Banksian post-scarcity Culture-style society without A(G|S)I or even Fully Automated Luxury Space Communism. So, AI is Bad because it is stealing jobs or stealing my job or corrupting the youth or being developed by rapacious high tech companies that are destroying the world or is merely a stochastic parrot that can't do anything useful and is forever necessarily prone to hallucinations or can't do basic algebra or work out how many "r"s there are in "strawberry" or based on stolen IP or memorising the internet or is environmentally damaging or will go the way of the Zuck's metaverse in another few weeks or months or enable deep fakes or make relationships with other humans even more difficult for a lot of vulnerable people or end up destroying the planet through creating mirror life or gray goo or weaponised microbes.

The thing is that a lot of those things are actually true. The world is in a very bad state and a lot of the companies pumping resource into AI or also responsible for contributing to the awfulness of the world. AI enables some definitely bad things as well as some potentially good ones. A lot of people don't have access to the latest frontier models and don't realise how powerful they have become. For a lot of people, they don't need or want to use them right now, so they seem extraneous. Like the web for most people in the mid-1990s. The A(G|S)I Derangement Syndrome arises though when opposition to AI becomes a ideological shibboleth, just as COVID or climate denial is for many on the right. But we should mark the world of Thomas Pynchon from 1984, which I mist have quoted before 

If our world survives, the next great challenge to watch out for will come - you heard it here first - when the curves of research and development in artificial intelligence, molecular biology and robotics all converge. Oboy. It will be amazing and unpredictable, and even the biggest of brass, let us devoutly hope, are going to be caught flat-footed. It is certainly something for all good Luddites to look forward to if, God willing, we should live so long. 

It is 2025 now and Thomas Pynchon is still alive about to publish a new novel and we have finally arrived at an epoch at which the curves of research and development in artificial intelligence, molecular biology and robotics are all about to converge. It will be amazing and unpredictable, and let us devoutly hope even the biggest of brass are going to be caught flat-footed. Because if we want something like our Banksian inheritance, the Left is going to have seize what might be its Last, Best Chance. It will be no easy ride, but the alternatives are all too easily imaginably unimaginable.

 

The Inhumanoids

The Inhumanoids, I have probably said before, was the name of a science fiction tv series I wanted to create in a John Menzies notebook in Junior 4, in 1976-7. like most of my grand plans came to naught. Still, a pretty nifty name. OF course, someone else came up with it: "Inhumanoids is the title of an animated series and the name of a Hasbro toy property that were both released in 1986." I was there a decade earlier when I was 8. There's also this book from 2017. Great minds or not.

Anyhoo, this has nothing to do with what I actually wanted to talk about, which is that I was watching CNBC this morning and the estimable Carl Quintanilla (I assume you all follow him on Bluesky), where his takes are definitely spicier than on the broadcast channel) was taking about the technoeconomic tensions between China and the US. He mentioned China's lead in robots including "humanoids". Yes, he used the word humanoids on CNBC. It's probably not the first time and it certainly won't be the last, but it something that we have reached an epoch at which one can casually, in a mainstream medium, refer to China as having a lead in humanoid robots. As we have said, from CES and other sources, it's pretty clear than humanoid robots are going to be a thing. The clunk Boston Dynamics robots are long since a thing of the past. Of course, they have to be able to do useful stuff and it's probably going to be a while before we have one in our home has a personal butler, but I look forward to my first sighting of one in the wild.   

The Urgency of Interpretability

Dario Amodei has an interesting article on the urgency of interpretability. Of course, interpretability is nothing new. It's a classic classic AI problem. You can't just use a neural net blackbox to make mortgage decisions because you have to be able to justify them. This is one of the AI ethics that the kinds of researchers who get to get funded by the Turing Institute work on. I seem to recall it being discussed at workshop I went to at Imperial over a decade ago. In those days, you probably couldn't have build a neural net-based system even if you had wanted to, so you probably had handwired logic. Amodei is talking though about we can get interpret the actions of frontier models. Well, a big advantage over human minds is that we can look inside. Of course, what we see is mostly large matrices. But people, well, some people, a few people, have spent a lot of effort looking into this question,. There's certainly things we can do. But as, AI 27 suggested recently, this might only get us so far.   

Of course, interpretability and alignment aren't necessarily the safe. You could have an aligned, but non-interpretable A(G|S)I, one that, like the Eschaton, moved in mysterious ways. Equally, you could have non-aligned, but interpretable A(G|S)I, one that was perfectly open about not caring much, if at all, for humans. But here's the thing. There was a point, I guess, in the mid-2000s, when Yudkowsy realised that (a) he wasn't a good enough programmer to program the seed AGI; (b) even if were, there would be no way to ensure that it would stay aligned. He realised that his grift was better serviced running a website or writing sophomoric essays or Harry Potter fanfic. MIRI did do some work on 

No publications since 2021, which given the current situation is noteworthy (there's a couple of November 2024 preprints). Their homepage declares

The AI industry is racing toward a precipice.

The default consequence of the creation of artificial superintelligence (ASI) is human extinction.

Our survival depends on delaying the creation of ASI, as soon as we can, for as long as necessary.

The general feeling is that the doomsters and the accelerationists (Altman, certainly, Amodeii, presumably) lost. I am not sure where that leaves the grift. I guess you can buy ESY's books on Amazon. But I wonder what his plan is. Does he actually believe this stuff? I mean I do. Gemini 2.5 is a better physicist than I was at 27. It's also a better virologist. That's not a combo we have had before. I except an extinction-level event in the next years, whether deliberate or accident, state or non-state, viral, bacterial, fungal, grey goo, nanotech, mirror life.

Here's a picture of ESY with Altman, but I don't know when this was taken.

Because the thing is that, as hinted at by AI 27, you are probably going to have to have a trust network. So, you trust an AI that can interpret an AGI that can interpret an ASI. But it's like trying to explain the thinking of a Fields Medal-winning mathematician to the person on the Clapham omnibus. Maybe, the ASI is so good at interpreting verself that ve can make verself understood. But it has always seemed clear to me and I think Geoff Hinton and, it seems, ESY that no dynamic cybernetic system driven buy complex constantly evolving metagoals (and metametagoals) can ever be truly trusted. As with a person, you have just got to hope they don't unexpected go off the rails. Most people don't most of the time, but that's because humans are powerfully influenced by other humans and societal norms. That won't apply in the same to weakly god-like entities. You just can't know for sure how something is going to react in all circumstances and forever.   

Perhaps we need that Butlerian Jihad after all

(Only kidding!)

Rational Agent

Interesting article by Jasmine Sun on her Substack (I know, I know), an interview with "an AI researcher, founder, and community-builder". Two things stand out.

Firstly,

What does “AGI” mean to you?

I use the classic definition of AGI, instead of moving the goalposts. The “G” in AGI is about generality. There are two types of AI: narrow and general. We used to train models for specific tasks. For every task, you’d build a huge, specific data set and train a model just for that task. That changed with the alpha version of Claude I was using in the summer of 2022. It could do any task you asked it to do. It was fucking crazy. It didn't care whether it had specific training data. It was general while prior models were narrow.

When you say “moving the goalposts,” what do you mean?

There are ideological and corporate reasons why it’s important for some people that we never achieve AGI. The reality is that AGI is a coordination tool, rather than a specific technology or moment. If you look at the history of the AGI concept, it’s the organizing principle behind the more ambitious—you might say religious—side of the field.

Which is interesting because I have sometimes suggested that what we have is a kind of proto-AGI because generative AI has such a wide range of capabilities. As Dario Amodei points out in a recent essay, FMs are, effectively, "grown". That's why they are robust in contrast to traditional software systems, which are brittle. Bit, of course, we don't tend to think of FMs as robust because we concentrate on what they can't do (robustly). From the perspective of 2002, 2022 Claude would have seemed like AGI. but we don't think of it as AGI because it can't do some of the things we would reasonably expect AGI to do. Which is why I think he's wrong that just claiming AGI hasn't arrived is about keeping the hype cycle going. Claiming AGI has arrived would boost the hype cycle, but also inevitably lead to a lot of flak. WE aren't there yet, but we might be in a couple of years. Gemini 2.5 Pro is very powerful and it's only going to get more powerful.

Secondly, 

            Also, I talk to this group of clones of myself all the time on Telegram. I have 14 years of journals, about 30,000 pages, and I put all that writing into a system that ideates with me.

Do you have clones of mentors or friends, like an imaginary board of advisors?

I never got into that. But I cloned an ex-girlfriend of mine once. Her clone was really, really her.

Leaving aside the ethics of cloning his ex-girlfriend (indeed), how is this done? Is this a system he created himself or is this something off the shelf? I can't find anything very obvious, but, of course, you could do finetuning on top of an existing FM with RAG or whatever based on your own data. In my case, that would be my blog. So, I guess this is something that might be relatively easy to build. Not I could that with my Jetson Orin Nano Super Developer Kit. It's an intriguing possibility. 

Divergent Trajectories: A Comparative Analysis of Artificial Intelligence and Drexlerian Molecular Nanotechnology

I asked Gemini Deep Research with 2.5 Pro why A(G)I has advanced so much more successfully over the last 30 years compared to True Drexlerian Molecular Nanotechnology. This was inspired by the sad death of Damien Broderick recently and finding that the revised edition of The Spike was just $2.99 on Amazon US. I read the original version back in February 1999, where I first came across Eliezer S. Yudkowsky. The second edition contains significant few stuff from the late 1990s  and very early 2000s, so worth a punt. However, the reference to an article in the August 1995 edition of Wired might well be in the original. "Wired asked several nanotech experts to estimate when we will reap the rewards of their research."

Respondent	Molecular Assembler	Nanocomputer	Cell Repair	Commercial Product	Nanotech Law
Robert R. Birge	2005	2040	2030	2002	1998
Donald W. Brenner	2025	2040	2035	2000	2036
K. Eric Drexler	2015	2017	2018	2015	2015
J. Storrs Hall	2010	2010	2050	2005	1995
Richard E. Smalley	2000	2100	2010	2000	2000
Bottom Line	2011	2041	2029	2004	2009

I had always thought of Smalley as a critic of MNT, but perhaps I was wrong. He died in the 2000s sadly, but the others are still alive. Obviously, Donald W. Brenner was the one to got closest. He's still active, although not seemingly in research directly related to MNT, but I don't think he's going to be finally unveiling his molecular assembler this year. Someone might; we still have 8 months to go after all! There are obviously lots of commercial products that at some level are nanotechnology, although not really TDMNT, and I suspect that's what Brenner was thinking of in saying 2000. I suspect we will have nanocomputers by 2041. A lot of depends on whether we get the Amodei data centre/country of (super)geniuses and 50-100 years of bio/phystech progress in 5-10 years. Cell repair by 2029 seems optimistic by 2029, but Brenner suggested 2035, which could be on the money still. A molecular assembler by 2030? I am not going to bet on it, but then we didn't really expect to have AGI by 2025, yet we kind of do. 

The Gemini report included this useful table of comparative analysis of AI and Drexlerian MNT development.

Factor	Artificial Intelligence (AI)	Drexlerian Molecular Nanotechnology (MNT)
Core Domain	Information Processing	Matter Manipulation
Fundamental Challenge	Algorithm/Logic/Learning	Physical Fabrication/Control
Development Path	Incremental, Paradigm Shifts	Foundational Breakthroughs Needed
Key Enablers	Algorithms (ML/DL), Data, Computation (Moore's Law, GPUs)	Precise Positional Control, Mechanosynthesis (Theoretical/Nascent)
Hardware Dependence	Leveraged existing/advancing hardware	Required invention of core hardware (assemblers)
Simulation/Testing	Highly effective, rapid feedback	Useful but limited, physical realization is key
Tangible Early Outputs	Yes (games, chatbots, expert systems)	Limited (STM images, theoretical designs)
Funding Cycles	Boom/Bust (Winters) but Resilient	Initial interest, then marginalization/niche funding
Key Bottlenecks Overcome	Combinatorial explosion (via heuristics/ML), knowledge acquisition (via ML), computational limits (via hardware)	Feasibility critiques (fat/sticky fingers), thermal noise, control complexity (largely unresolved for universal assemblers)

Back in 1995, Drexler said he "believes that nanocomputers could finally provide the horsepower needed for artificial intelligence." That seemed to have turned out to be the case (I know, I know, but Gemini 2.5 Pro is better than me at 57 at physics,; it's also better than me at 27 at physics and people really should try the latest frontier models; o3 might not know how many "r"s there are in "strawberry" or be able to do simple algebra, but Gemini does; I suspect o3 also does, but if ask it an "easy" question it doesn't get answered by the full model; one thing that's clear is that different FMs have different capabilities and they are advancing rapidly). The chart very neatly lays out how we got to AGI before TDNMT. In summary, self-supervised training on large datasets of transformer architecture neural nets with backprop and gradient descent plus RL(HF) and various other tweaks like mixture of experts. There's still a lot of low hanging fruit and opportunities to combine different design philosophies, which is why I expect progress to continue and that we might get to something very like AGI by 2027 or so. Of course then, in the ideal world, we can use AGI to solve all the other problems. And one of the first and most fundamental problems to solve is TDMNT. Recall that Drexler's original 1981 nanotech paper was on protein engineering. Consider AlphaFold and the existence proof of the biological cell. Might we finally achieve the dream of the nanomachine that is formed from various proteins that fold into the engineered shapesof machine's components? It's going to be a crazy ride. Stay tuned.  

All That Is Solid Melts into Air II

Asked Gemini Deep Research with 2.5 Pro for a list of the top 10 largest global companies by market capitalization in April 1975 wondering if would do better than Gemini 2.5 Pro on its own. It certainly gave me a nicely formatted table and these companies are substantially the same as the previous list. Royal Dutch Shell and British Petroleum make the list, so we get five of the sisters (weren't there seven?). 

Table 1: Estimated Top 10 Largest Global Companies by Market Capitalization (Circa April 1975)

 

Estimated Rank	Company	Primary Industry	Headquarters Country	Estimated Market Cap (USD)	Supporting Evidence Notes
1	IBM	Technology / Computers	USA	N/A	Top 3 US Market Cap YE 1975 6, Top 10 Revenue 4, Nifty Fifty Leader 15
2	AT&T	Telecommunications	USA	N/A	Top 3 US Market Cap YE 1975 6, Dominant Utility 9
3	Exxon (Standard Oil of NJ)	Oil & Gas	USA	N/A	Top 3 US Market Cap YE 1975 6, #1 Revenue 4, Major Oil Player 8
4	General Motors	Automotive / Industrial	USA	N/A	#2 Revenue 4, Historical Giant 9, Likely Top 10 Market Cap
5	Royal Dutch Shell	Oil & Gas	UK / Netherlands	N/A	Major Global Oil Company 17, Leading Non-US Contender
6	General Electric	Industrial Conglomerate	USA	N/A	Top 10 Revenue 4, Historical Giant 6, Likely Top 10 Market Cap
7	Texaco	Oil & Gas	USA	N/A	Top 5 Revenue 4, Major Oil Player 8, Likely Top 10 Market Cap
8	Mobil	Oil & Gas	USA	N/A	Top 5 Revenue 4, Major Oil Player 8, Likely Top 10 Market Cap
9	British Petroleum (BP)	Oil & Gas	UK	N/A	Major Global Oil Company 16, Noted Strong Stock Performance 16, Likely Top 10
10	Eastman Kodak	Chemicals / Photography	USA	N/A	Historically Significant 9, High Profits 4, Plausible Top 10 Contender over lower-margin Industrials

The oil companies all still exist though Mobil merged with Exxon and Texaco is now Chevron. There's company called AT&T, but just a Baby Bell rebranded. The actually interesting bit of AT&T that we might care about and helped invent the future is, perhaps a little bizarrely (or not), owned by Nokia (which is also Alcatel, Lucent and Siemens telecoms). General Motors still produces not very good cars, but not so much in Europe if at all. No-one's saying what's good for GM is good for America and vice versa any more. In 1975, Rex Malik, who I must have come across on The Computer Programme, published And tomorrow ... the world?: Inside IBM, which I have a copy of and must read when I get it here because I suspect it's a fascinating account of IBM at its peak. In a way, AT&T invented more of the future that IBM did, we all hacve devices running Unix, but who is running OS/360 (or, indeed, z/OS; yes some people, but I have never come across any IBM hardware in the wild, except for someone who had an IBM workstation at DERA Holton Heath that ran AIX - we did briefly use a 5250 timesheet application when I first went to Ericsson)?

General Electric was dissolved into several parts in 2024, but several of the parts are still substantial and use the GE brand. But it a good example of a company that was absolutely driven into the ground by neoliberal ideology. I wonder what the return on a share of GE over the last 50 years has been. At times, probably good. Of course, in the long run, it's only cash that matters. 

Of course, the real loser here is Eastman Kodak. About 4000 work for the residual company. How the mighty are fallen. Kodak did do other stuff. I seem to recall an O'Reilly book recommending Kodak CD-ROMs. But it's difficult to pull off such profound pivots. The chemical photography business went from a mainstay of technological society to little more than a hobbyist niche in just a decade or so. They were mugged by history. Got to hope that the rest of us aren't.

 

All That Is Solid Melts into Air

The ten (actually eleven) largest companies in the world as of April 2025:

• Apple (AAPL): ~ $3.0 - $3.4 Trillion (Technology - USA)  
   
• Microsoft (MSFT): ~ $2.9 - $3.2 Trillion (Technology - USA)
• Nvidia (NVDA): ~ $2.7 - $3.2 Trillion (Technology/Semiconductors - USA)
• Amazon (AMZN): ~ $1.9 - $2.4 Trillion (E-commerce/Cloud Computing - USA)
• Alphabet (GOOG/GOOGL): ~ $1.9 - $2.3 Trillion (Technology - USA)
• Saudi Aramco (2222.SR): ~ $1.7 - $1.8 Trillion (Oil & Gas - Saudi Arabia)
• Meta Platforms (META): ~ $1.4 - $1.8 Trillion (Social Media/Technology - USA)
• Berkshire Hathaway (BRK.A/BRK.B): ~ $1.0 - $1.1 Trillion (Investment/Finance - USA)
• Tesla (TSLA): ~ $0.8 - $1.2 Trillion (Automotive/Clean Energy - USA)
• Broadcom (AVGO) / TSMC (TSM): These two companies often trade places for the 9th/10th/11th spots.
  • Broadcom: ~ $0.8 - $1.1 Trillion (Semiconductors - USA)
  • TSMC: ~ $0.8 - $1.0 Trillion (Semiconductors - Taiwan)

So, that's one oil company, one holding company and nine tech companies (I guess Amazon at some level is a retailer). If we look at the year the companies were founded

1. Apple: 1976
2. Microsoft: 1975  
3. Nvidia: 1993  
4. Amazon: 1994
5. Alphabet (Google): Google was founded in 1998. The parent holding company, Alphabet Inc., was formed in 2015.  
6. Saudi Aramco: 1933 (Originally California-Arabian Standard Oil Company)
7. Meta Platforms (Facebook): 2004 (Founded as Facebook)
8. Berkshire Hathaway: 1839 (Originally as Valley Falls Company, a textile manufacturer. Warren Buffett took control in 1965, transforming it into a holding company).  
9. Tesla: 2003  
10. Broadcom: 1991 (Founding of Broadcom Corporation). Note: The current entity, Broadcom Inc., was formed when Avago Technologies (with roots tracing back to a Hewlett-Packard division founded in 1961) acquired Broadcom Corporation in 2016 and adopted its name.
11. TSMC (Taiwan Semiconductor Manufacturing Company): 1987  

So, one in 1839, but really 1965, one in 1933, two in the 1970s, one in the 1980s, four in the 1990s, two in the 2000s. So, only three of these companies existed 50 years ago and one of those no-one had heard of.

Gemini 2.5 Pro doesn't give me a definitive list for April 1975, but suggests

Considering these reference points and the economic climate of 1975, the largest companies by market value likely included a mix of these names. While a precise, verified ranking for April 1975 isn't available from the search results, the top 10 probably consisted of companies such as:

1. IBM
2. AT&T
3. Exxon
4. General Motors
5. General Electric
6. Eastman Kodak
7. Texaco
8. Mobil
9. Sears Roebuck
10. Procter & Gamble or DuPont

Other major companies like Gulf Oil, Chevron (Standard Oil of California), Amoco (Standard Oil of Indiana), Shell, Xerox, and 3M were also very large and could potentially have been in the top 10 depending on market fluctuations during that specific period.

So, none of the top ten companies from April 1975 are in the top ten today. We see a preponderance of oil companies. I suppose General Electric did make computers and, of course, Unix and C, for instance, came out of of AT&T, although none of these companies are really tech companies as we think of them today with the possible exception of IBM. (A related issue is what do we mean by tech companies. The Weasel might argue that in some ways the internal cultures of these companies might not be so different from modern ones in some ways. Then again, no-one was sending other people emails, to first approximation.) The global economy is approximately 5.5 to 6 times larger in 2025 than 1975 and we can see that much of that extra value is linked to computers or at least that computer companies have captured much of that value. Of course, this fits with our prediction for a technological singularity (of some kind) at some point in the next few decades: "All That Is Solid Melts into Air".

"

Too Cheap to Meter

Just imagine that in the 1960s the world had built ten (or a hundred) times as many nuclear power stations than it did, so that the use of coal or gas or oil for power generation had been completely removed by, say, 1980. You could use hydro or tidal or geothermal or solar thermal, all technologies that existed at the time. Of course, this would have been difficult. Fossil fuels were cheap and efficient, but it's not as though we didn't build lots of nuclear power stations. Of course, with lots of 1960s- and 1970s-era nuclear power stations, we are going to get lots of nuclear disasters and well as the usual radiation releases, probably at least one full meltdown, but there are ways round that and it might still be preferable to climate breakdown. Nuclear proliferation is going to be a big issue in this world as everyone has nuclear power stations, so might not be a utopia.

You have to get rid of the use of gas for heating, cooking, but, again, that could be OK. We had an electric cooker. You can use hydrogen in internal combustion engines and fuel cells already existed, so you can go down that route too. Glass microbeads might be the safest approach, but I suspect we would also see FeTi, compressed gas, liquid hydrogen at 20K, slush hydrogen, lots of possibilities. Planes, trains and ships could all use hydrogen. So, by 1990 or, we could have completely decarbonised the economy. Of course, it would have been expensive and there are going to be lots of issues, though there would be incentives to make things safer, but this doesn't require any new technology just will. Of course , that was never going to be there and here we are. 

Ten Ages of the World

I am 57 next week. I can't remember much before 1975, so my time window is about 50 years.

1975: our houses in Preston and Seattle were there. We, of course, lived in the one in Preston. Beechwood Avenue and 45th Avenue SW looked, in many ways, much as they do today. We lived in advanced technological society under capitalism (you could, I suppose, argue about whether it was "late"; according to the term was coined in 1928 and was used widely in Germany from the 1960s and especially from 1972, also by Frederic Jameson a great deal, so we could say the capitalism of 1975 was late even if there are significant differences from the hypermodernity of 2025) . But if you told someone in 1975 that in 2025 the richest man in the world would have made his fortune from electric cars and reusable rockets and he would be the shadow president and doing all the other things, 2025 would seem a pretty science fictional place. More about this in a later post. 

1525: Preston existed and you could go to the pub,. The broad out line of the some of the main streets would be the same as today. We know a lot about C16th Europe and in some ways it's not so far from the current day West. We are already in the Early Modern Period after all.People talked something like modern English, although it would probably take a while to adjust, but you could probably navigate the society without that much trouble. Seattle doesn't exist, but the Coastal Salish people are here and we can deduce something of their lives from archaeological and ethnographic evidence. They won't yet be in contact with Europeans, but things are about to start changing even before the direct encounter with the spread of disease, trade goods and new technologies as well as population displacements. The outline of the continents is pretty much what we have today. 

~3000 BCE: there are people in Preston, but what life is like in pre-Bronze Age (so neolithic) Britain is pretty uncertain. The Bronze Age is going elsewhere, but won't reach Preston for the best part of a millennium. There are people living in the Puget Sound area and we can deduce something of their lives from archaeological and ethnographic evidence. The really interesting question is how different were they culturally, technologically, linguistically from the later inhabitants. We know in the case of Preston, very different, but how would that map to Seattle? What language did they speak in Lancashire. Some kind of proto-Celtic or was it more like Basque? There are definitely marshy areas that affect the outlines of some coastlines, so some maps would look slightly strange at high enough resolution.

~48000 BCE: depending on whether or not we are in a ice age or not, the maps will more or less different, but the continents are in their present position. There are probably no people in Puget Sound or Preston, but there are certainly hunter-gatherers trucking around Africa, Asia, Europe and Oceania and they can talk. How much they are like modern hunter-gatherers is debatable and it might take a while to learn their languages as they are probably nothing like any modern ones. Lots of charismatic megafauna around that we don't have today.

~500,000 BP. Probably not modern humans as early as this and probably no language, but definitely fire and tools. Depending on the sea level, lots of interestingly different crinkly bits around the coasts. Rivers and lakes would be different, but much of the EArth wouldn't look that different from space. 

~5,000,000 BP: the continents will be in slightly different places, so things will probably look a bit different from space. Plenty of hominids around as well as lots of familiar and less familiar animal and plant species. Probably a bit Uncanny Valley, but maybe not a bad place to live.

~50,000,000 BP: the continents haven't banged into each other yet, although the broad outlines of the continental masses will be recognizable. We are very much in the Age of the Mammals (no non-avian dinosaurs), but not many familiar species. Early primates though are around and we have trees and grasses and flowering plants. Would be a fascinatingly different place to visit because it would be both so alien and so familiar simultaneously.

~500,000,000 BP: the continents look nothing like today. The Cambrian Explosion has occurred, so the oceans are teeming with life and most of the modern phyla exist, but there very little on land yet, so this is an almost unrecognisably alien world with almost nothing we expect from the Earth present. 

~5,000,000,000 BP: the Sun and the Solar System and the Earth have not been to form yet and won't for another 400 million years. Yet to an observer on a planet in our position in the Milky Way, the night sky would probably look familiar enough. According to the FoAK, "8.8 billion years (5 Gya, z = 0.5): Acceleration: dark-energy dominated era begins, following the matter-dominated era during which cosmic expansion was slowing down", which might be significant, but would be hard to detect at the time. Of course, as I have suggested, this transition does suggest that someone did some large-scale engineering with the universe at this time that affected the fundamental structure of the cosmos causing the acceleration to start.

~50,000,000,000 BP: the universe doesn't exist. Which is weird. Why is the universe so young? Especially given that the future of the universe is so long? The Big Rip might be 22 billion years in the future, but probably won't happen in which case the future might last to we get to

years when a Boltzmann brain could appear in the vacuum via a spontaneous entropy decrease or years when quantum tunnelling in any isolated patch of the universe could generate new inflationary events, resulting in new Big Bangs giving birth to new universes. (yes!). Doesn't that suggest, like the cosmic acceleration, that there's something weird going on? Of course, we can use anthropic reasoning to argue that it's not a surprise that we live at this epoch. The modal person will just before the end. But you can see why Hoyle preferred the steady universe for philosophical reasons or why I think there must be an ensemble of classes of universes. Also, I think there's a novel in the discovery of a star that's 50 billion years old. How can it possibly exist in our universe?

It has been and will be a hell of a ride. How knows what we might get to observe in next 50 years as we approach the Epoch of the Data Center of the Country of Supergeniuses?

Prioritising 50 Projects V

41. design sustainable garden or permaculture experiment.

We need, at some point, to do over the back garden and I want a Beer-Pask cybernetic pond. But this is going to happen until 2026 at the earliest.

42. write a play or screenplay.

Suffers from the problem of all my writing projects - time.

43. poetry course.

Plenty of those at Hugo House. But probably not until 2026 at the earliest.

44. learn a new programming language (Go, Julia, Lisp, Rust).

In the epoch of Cursor though, is that still worthwhile?

45. create an annotated bibliography or research compendium.

This would be a very good idea.

46. experiment with biohacking or cognitive enhancement.

I wonder what sort of things I meant or that could be done. Probably not to 2026.

47. Carping On ("an Old Rectory production") podcast.

But who would want to listen to us?

48. develop an interactive fiction story or text-based game.

Suffers from the problem of all my writing projects - time. I have to make it.

49. create a personal knowledge management system to catalogue and link ideas, projects and notes across my various interests.

Kind of the same as 45, but yes.

50. design and 3D print my own miniatures for wargames.

Would I really want to design them? Although I suppose "design" covers a lot of ground. It's possible, I guess, if I get a 3D printer, but that's not happening for a while.

A(G|S)I Comes at A(G|S)I Time

The Weasel is fond of saying that steam engines come at steam engine time. So, we need a combination of technology (metallurgy having advanced to the point that you can make a boiler that won't burst under pressure), a killer app (pumping water out of mines) and a set of technoeconomic innovations that make it easier to bring together clients, entrepreneurs, inventors, investors and workers. A fourth requirement would be the set of inventions related directly to how the steam engine works.

The thing is that people have been using cannons in Europe since the C14th, so was it really advances in metallurgy that made steam engines practical at that epoch? People have been mining tin in Cornwall for thousands of years, so was there really a sudden need for pumping water out with steam engines? There were certainly a lot of social and economic innovations in the late C17th/early !8th (central banks, stock markets, insurance, limited liability companies, etc, etc), but it is not as though people weren't doing large, medium and small scale commercial and state enterprises for hundreds or thousands of years before that.A technology has intricate as printing became widespread from the mid-C15th, shipyards were churning out ships, people were toiling away in mines and building water and windmills as well as elaborate clocks, so it is not obvious that steam engine time, sat the least couldn't have been at least a century or two earlier.

As I said on Bluesky yesterday, AI-1956, the Dartmouth summer workshop programme, symbolic AI, GOFAI, the Royal Route to Human-Level Intelligence, turned out to be a bust. But that wasn't obvious at the time. The term AI was coined to avoid having to use the C-word, cybernetics for the workshop, because then they would have to have invited Norbert Wiener. But it's hardly the case that there were lots of other approaches at the time - neural nets, Ashby's Design for a Brain, Rosenblatt's Perceptron - that were potentially promising. 1950s computers were incredibly feeble compared to modern ones, but it didn't seem like to people at the time and you only need ~8 kB to start doing really interesting and useful things. It wasn't obvious in 1956 that intelligence couldn't be achieved through the combination of a few algorithms and heuristics. After all, no-one had in a position to try to build a brain up until then.       

So, Lighthill was right! There was no point doing AI research in the 1970s because you just didn't have the scale - compute and data - to do anything useful. Of course, Lighthill wouldn't have thought of it like that, and it might be that a lot of what was going on was academic knife grinding, but, of course, to first approximation, it was true. For decades, AI was 20 or 50 years off because for decades, no-one had the scale to do anything useful It's still possible that AGI is 20 or 50 years off. That seems unlikely in an age where we have robust quasi-AGI already (by robust, I mean that you can ask Gemini 2.5 Pro anything in natural language and it will usually have a pretty good go at answering your question or doing the requested task, even if it doesn't always get it 100% right, that's better than 95% of co-workers in my bitter professional experienc; this capability is probably the most extraordinary thing about FMs - we complain about hallucinations or incomplete tasks, rightly so, but the shift from the brittleness of traditional software engineering is still astonishing), but there could be many gotchas yet and, of course, we probably want hybrid systems that incorporate reinforcement learning and symbolic AI too as well as improved capacity to generalise from small samples and possibly not train with backprop and gradient descent because the brain doesn't. Let a thousand philosophies bloom.

Noam Brown has achieved many extraordinary things - solved poker, systems that play chess like humans, a system for playing Diplomacy - and said at the recent NVIDIA conference that we could have got here a couple of decades earlier. AGI time could have been in the late 2000s rather than the late 2020s! I want to dig deeper his thinking, but it seems to be that more thinking time would help with reasoning models. FMs respond typically very quickly, but what if they thought about a problem for hours instead? But obviously you could go back to the 1970s, as the Weasel suggested to me, to ICL in 1973, and design a programme that would get us to AGI sooner. We will talk about this more later, but you would build neural nets trained with backprop and gradient descent. This could have been done in 1973. You could use an ICL Distributed Array Processor,do lots of OCR applications, create a big training corpus, do self-supervised learning, get to advanced architectures like transformers, mixture of experts, thinking time. But you probably also need a couple of breakthroughs, such as more efficient learning. Get the internet earlier and maybe you can have enough data two decades earlier, even if you raw compute isn't much or any faster. Obviously, specialised processors, like GPUs and neuromorphic chips. It doesn't seem to unreasonable that we could have had systems a bit like modern FMs in 2005. But then hindsight is 20/20.Steam engines in 1580? AGI in 2007? Combine the two. We could have the Singularity in about 1809. There's a novel in that.

Jonbar Junction

A cursory Google reveals three hits for Jonbar Junction. One is to a review of the game Time War in a 1980 edition of Ares that also contains an article containing the relativistic rocket equation. Nothing to here, although I am sure I have used it and claim so on Bluesky, where all the hits are to me. I don't know where I stumbled on the term, there are so many places it might have been, perhaps I just coined it myself, though it seems much likely, given the Ares article, I read it somewhere or perhaps heard it. It just rolls off the tongue so much more easily than "Jonbar Hinge" or "Jonbar Point".

Anyway, I was struck today that, of course, "Junction" suggests a railway station and "Jonbar" somewhere exotic, so Jonbar Junction ought to the name of transtemporal railway station in which the train go to not to different places or even times, but to different timelines. Railway station are liminal zones, like airport, and Tony Judt celebrated them monumental aesthetic presence in an article in the NYRB shortly before he died. With the fog and dark and eerie whistles and platform cafes, we feel like we are in a Sapphire and Steel scenario. Certainly, this could also work for Doctor Who. The junction itself would be an edifice, of course, a character in its own right. Of course, we are probably going to have literal (that is, magical) trains, double-deckers running 3-metre gauge track like the Hilterian Breitspurbahn would have done. The trains would be more like ocean liners than the trains we are familar and would offer ample opportunity for mystery.  Skippy The Bush Kangaroo said "I'd read that". All I need to do now is write it. I had better get cracking. 

Capturing Value

Gross Global Product is ~$100 trillion. How much of that could be captured by AI? 1% would be $1 trillion. Split that up between a few big firms, a few might get $100 billion.. $100 billion in revenue corresponds to a value of $800 billion or so (say 500 million to $1.2 trillion), so if you are a big tech company, you might hope to get your market cap to increase by 50-100% if you can capture a slice of that 1%. (Alphabet is ~$1.9 trillion). So, we can see why the big tech companies might want to make big bets on AI. Firstly, they are IT companies, so doing AI is a continuation of what they have been doing for years and, secondly, if they don't, they might get their lunch eaten. But a $1 trillion and we are nowhere near that in AI-related revenues yet.

Gemini Pro 2.5 tells me

While the direct market value of the software industry (companies primarily creating and selling software) likely accounts for roughly 0.6% to 1% of the Gross Global Product, this figure significantly underrepresents software's overall economic importance.

Software is a foundational layer and productivity driver for nearly every other industry. The broader IT sector represents closer to 5% of GGP, and the software-fueled digital economy constitutes a much larger share, highlighting software's pervasive and critical role in the modern global economy.  

So, we can do a sanity check. Our $1 trillion might represent the whole or more of the market value of the software industry, but alternatively 20% of the broader IT sector. Could AI capture that 20% in 5 years? Clearly I find AI very useful for helping with my IT and programming issues. If were programming 40 hours a week how much more productive would I be? But, of course, the kind of stuff I was doing, ad hoc Jupyter Notebooks, isn't what most programmers do.According to the Weasel, Innovyze has 2 million lines of code. That's hundreds of developer-years of effort over 30 years. Vibes coding might allow you to produce 20,000 lines of code in a weekend, but then you have to debug it. Having said that, it's not as hell isn't other people's code and the quality of code in the Innovyze I know from days there is variable. Of course, writing a water resource management system from scratch isn't something that many people are likely to want to do at this epoch. (API monitoring tools might be a different matter.) AI might be good for helping to get a demo up and running (and will be better at debugging its own code), but how much can it help with a large existing codebase? That's an interesting question Soon, you are going to be able to through the entire codebase into an AI. As Gemini told me

This large context window allows me to process and understand much larger amounts of information at once, like long documents, hours of video, or extensive codebases, and maintain coherence over lengthy conversations.

Most programmers are not l33t, but perhaps there just isn't enough juice in being a Stack Overflow replacement. I am going to use Gemini to help me write a MuZero-type deep reinforcement learning system to play Kurt Vonnegut's GHQ, so we will see how quickly we can  get a demo working. FMs have been very useful to me already, for instance, in my current consulting project, so there is definitely value there for me at the few tens of dollars a month level (what I pay for Gemini Pro). A billion people paying $300/year is $300 billion, but then most of the costs are going to be hidden from the end-user. Yr company is psaying your Office 365 subscription, so how much of that is going to going on AI in a couple of years? If you have one DIRW working 168 hours a week, 52 weeks a year, who knows what state your codebase is going to end up in soon enough?

The Ten Per Cent Solution

Yesterday we talked about Cathie Wood who apparently that we're "in the middle of the most magnificent period of truly disruptive innovation in history" and predicted real GDP growth could jump from 3% to potentially 7-10%.There must be something in the air because today I find Microsoft CEO Satya Nadella, speaking at the 2025 annual meeting of Madrona, a Seattle-based VC company, saying

“My formula for when can we say AGI has arrived? When, say, the developed world is growing at 10%, which may have been the peak of the Industrial Revolution. That’s a good benchmark for me.”

We might say well, they would say that, wouldn't they? They are both trying to sell something, whether it is software licences or investment services. The notion that the world might suddenly leap in half a decade or so to rates of global rates of growth never before seen seems prima facie absurd, although, as I discussed in my previous article, a case can be made and it doesn't necessarily even have to involve TDMNT or humanoid robots. In the world of the DIRW, what would humans do? Go and get jobs in the caring professions? I mean when cellular rejuvenation and anti-dementia drugs kick in, even that's not going to be as much of a option.Everyone will be a middle manager and spend their time going to (virtual) meetings about (virtual) meetings? Or run artisanal whisky bars? I guess hobbyist stuff with a lot of free time would work very well for a lot of people, but is the trajectory of large capitalism in that direction? I am not sure how that works under kleptocratic kakistocracy/kakistocratic kleptocracy.

As I have said, there is a lot of low hanging fruit. You can built a lot of infrastructure not just in developing countries. Decarbonising the economy is a big project that will absorb a big slice GGP. In the US, to solve the housing crisis, you need to build tens of millions of housing units (the iron law of supply and demand). How do you that in Seattle? Flying cars might solve the commute problem (where would they land?) and you could build a huge new city in Ellensburg or somewhere, perhaps with a hyperloop link to Seattle. Improved tunneling technology would be amazing if Seattle could get a proper metro in a few years rather than decades. All of those superyachts and private spaceships will count towards GGP as will everyone having their own personal virtual physician and therapist.

Nadella is right that if we do get a economic boom that can be ascribed to AI, we could call that AGI because it would be clear that the world has been transformed by AI and so it would be fair to think of that AI has something qualitatively different to GOFAI. Of course, it already is qualitatively and quantitatively different to GOFAI. Nothing was solving undergrad physics questions a decade ago. Still, of course, I want to see the first bona fide AGI scientific breakthrough. Something like room temperature superconductors would be a good start because it would be a revolutionary technology and undeniably a growth driver. But there are other things. What about solving maths. Or at least solutions to the outstanding Hilbert problems

8. Problems of prime numbers (The "Riemann Hypothesis").

13. Impossibility of the solution of the general equation of 7th degree by means of functions of only two arguments.

16. Problem of the topology of algebraic curves and surfaces.

A DCoSG could also help with problems 1, 2, 5, 6,[a] 9, 11, 12, 15, and 22, which have solutions that have partial acceptance, but there exists some controversy as to whether they resolve the problems. There are many other unresolved problems in mathematics. The solution of any one of them would be extraordinary. I wonder if that will happen (soon; 2025? 2027? 2030? 2038?)? It wouldn't even necessarily be AGI, but it would be amazing.

 

Abundance

Cathie Wood , apparently a well-known investor, apparently said recently, presumably at the recent Abundance 2025 Summit, that we're "in the middle of the most magnificent period of truly disruptive innovation in history" and predicted real GDP growth could jump from 3% to potentially 7-10%. Wow! Slow down there! But what would that be like? There's a lot of low hanging fruit, of course, not just in developing countries, but obviously that's difficult to unlock in sclerotic neoliberal states for a variety of reasons. But here we are talking about the data centre of (super)geniuses and what that leads to. A(G)I doesn't displace jobs, it displaces roles, but displace enough roles and you start displacing jobs and Leopold Aschenbrenner's drop-in remote worker is displacing jobs, so that could be massively disruptive to the economy if you are suddenly displacing a lot of people and the DIRWs are better than humans. That could drive a lot of growth by removing a lot of friction in all sorts of businesses.

We didn't have humanoid robots a decade ago, not ones you could buy for $10k, nor AI that was any good, but now we have both with lots of robotics startups and powerful AI and regular breakthroughs in AI for robotics, so it is hard to imagine that they are not going to a thing. Robots are perhaps where FMs were in 2020, but again those data centres of supergeniuses. Cheap, powerful robots could certainly have a disruptive impact on the economy and drive growth to historically high levels. A lot of growth revolves around building shit and that's hard in a lot of countries. I remember reading somewhere back in the mid-2000s that it would be better to build buildings with TDMNT because it would be slow, but that might not be true and the robots, humanoid and not, could help. To solve the housing crisis we have to build lots of new houses. There's plenty of land in the world and we really need to build lots of new cities. They did this in China after all. Flying cars solves the commute problem and you can expand Spokane to 10 million people.

The ultimate drivers are biotech and phystech, especially though not exclusively TDMNT. You can do lots of coll stuff without TDMNT, for instance, room temperature superconductors or fusion reactors or programmable matter or quantum computers or vat-grown meat or personalised medicine. But it's definitely easier to do cool stuff with TDNMT. We might never have our own personal nanofactories, but we can certainly imagine that most things will be grown in local facilities from air, dirt, sunlight and water. It would certainly disrupt global supply chains.

Of course, I believe the world will undergo profound change over the next decade. It's hard for me right now in March 2025 to really believe that that change won't be awful, the clash of geopolitical and technoeconomic forces seems much more likely to literally rend the world asunder, but it's all we've got. We've never been here before, but there might be some timelines that can survive the oncoming storm.

True North

"Canada, the True North, strong and free." So, I have finally managed to get to Canada. It's very metric, except when it isn't. Really freaky that distances and speeds are in kilometres(/hour). There definitely seems to be more British-type stuff in the shops, such as chocolate bars, and there are definitely lots of British accents to be heard in Whistler, both working in the businesses and in the street. I wonder what visas those workers are on. It feels practically like being back in London. Also the infrastructure seems better than in the US, but maybe that's just wishful thinking. Certainly it feels here like being in the future. 

"Come with us. It's going to a wild ride."

The first mention of AGI and the Singularity on this blog was on Monday, 9 February 2004, over 21 years ago. My interests haven't changed.

***

Two Essential Mailing Lists and Some Other Things

If you don't already subscribe to these lists, do so now.

SL4 mailing list

www.sl4.org/intro.html#join

and the AGI mailing list

agiri.org

There is significant overlap in both the membership of the lists and the topics discussed. The AGI l(Artificial General Intelligence) list focuses more on technical discussion of artificial intelligence research while the the SL4 (Shock Level 4) list concentrates more on topics relared to the Technological Singularity. As the two subjects are intimately related, it is best to read both lists. Both lists are moderated and posts are generally of a high quality. Ben Goertzel (www.goertzel.org) regularly posts. He's leader of the Novamente Project and is definitely not off his meds. He has an online book The Path to Posthumanity at

agiri.org/path/index.htm

Here's an interesting looking link to another online book about the Singularity by Bill Hibbard. Hibbard is an academic and has done a great deal of thinking on this topic

www.ssec.wisc.edu/~billh/gotterdammerung.html

***

The SL4 URL still exists though the last post to the mailing list was in 2012. Yudkowsky went on to many probably more lucrative ways of things monetising his angle. The AGIRI URL also still exists though, again, it doesn't look like it's been updated in years. Ben's website was updated in November 2024, possibly more recently. He is still active (the AGI-25 conference is in Iceland in August), of course, and might pop up on the AGI list now and then, although it's pretty moribund. The link to Bill Hibbard's book still exists. I wonder how it stacks up nearly a quarter of century in given what we are on the cusp of. It's interesting that all the sites are still there even if not been updated. The ideas are still out there and we got old without noticing and Singularitarians are the kinds of people obsessed by preserving the past as well as creating the future. As I said yesterday we can't really believe that we are here now and, of course, it feels real and unreal at the same time. As I said in our very first post on 6 February 2004, "Come with us. It's going to a wild ride." Today, I renew that invitation. The next 21 years are going to be an even wilder ride and I don't know if we are going to be get, but if not, we'll die trying.   

If you don't already subscribe to these lists, do so now.

SL4 mailing list

www.sl4.org/intro.html#join

and the AGI mailing list

agiri.org

There is significant overlap in both the membership of the lists and the topics discussed. The AGI l(Artificial General Intelligence) list focuses more on technical discussion of artificial intelligence research while the the SL4 (Shock Level 4) list concentrates more on topics relared to the Technological Singularity. As the two subjects are intimately related, it is best to read both lists. Both lists are moderated and posts are generally of a high quality. Ben Goertzel (www.goertzel.org) regularly posts. He's leader of the Novamente Project and is definitely not off his meds. He has an online book The Path to Posthumanity at

agiri.org/path/index.htm

Here's an interesting looking link to another online book about the Singularity by Bill Hibbard. Hibbard is an academic and has done a great deal of thinking on this topic

www.ssec.wisc.edu/~billh/gotterdammerung.html

***

The SL4 URL still exists though the last post to the mailing list was in 2012. Yudkowsky went on to many probably more lucrative ways of things monetising his angle. The AGIRI URL also still exists though, again, it doesn't look like it's been updated in years. Ben's website was updated in November 2024, possibly more recently. He is still active (the AGI-25 conference is in Iceland in August), of course, and might pop up on the AGI list now and then, although it's pretty moribund. The link to Bill Hibbard's book still exists. I wonder how it stacks up nearly a quarter of century in given what we are on the cusp of. It's interesting that all the sites are still there even if not been updated. The ideas are still out there and we got old without noticing and Singularitarians are the kinds of people obsessed by preserving the past as well as creating the future. As I said yesterday we can't really believe that we are here now and, of course, it feels real and unreal at the same time. As I said in our very first post on 6 February 2004, "Come with us. It's going to a wild ride." Today, I renew that invitation. The next 21 years are going to be an even wilder ride and I don't know if we are going to be get, but if not, we'll die trying.   

The Data Centre of Idiot Savant PhD-Level Biotech Researchers

So, we have the idea of the data centre of (super)geniuses. They have assimilated every scientific paper and textbook ever written, can reduce and analyse masses of data instantaneously, are brilliant at coming up with original theories, can develop and run complex simulations  and can direct and, in some cases, through machine and robotic interfaces, carry out experiments. So, we get, a few months to years, profound breakthroughs in every area of science from astronomy to zoology. We solve mathematics (yes, obviously,Gödel's Theorem means we can't run out of mathematics), cure most diseases, invent radical life extension technologies, unify quantum mechanics and general relativity, understand turbulence, gain new ways of understanding cybernetic systems that have profound implications across biochemistry, cell biology, condensed matter physics, cosmology, ecology, fluid mechanics, historiography, human geography, materials science, physiology, planetology and sociology, find room temperature superconductors and build fusion torchships. And, of course, perfect True Drexlerian Molecular Nanotechnology and the Personal Nanofactory.

Ah, yes, the PNF. A WMD in every home.

And all this is going to happen in 5-10 years from, say, 2027, so 2032 to 2037. Or perhaps even quicker. Once we have TDMNT, we don't have to worry about it taking years to build a fab or roll out new energy generation capacity. There will still be characteristic timescales for things to happen, especially with humans in the loop, but a lot of interesting stuff might be done in months. It will be a hell of a ride even if someone can cook up mirror life or a hafnium bomb in their garden shed. Humh... Perhaps those PNFs shouldn't be P.

The thing is we have been telling ourselves this story for decades. I was on a panel about this at the 1995 Worldcon in Glasgow. And 30 years later, we can't believe it anymore because it's something that's always going to be lurking out there a few decades away. Even if it's hard to deny that there's been some progress in AI over the last 30 years and even if we might wonder what another 30 years of progress in AI might bring. I'd only be 86, younger than my mum in 2055 after all. 

So the Amodei scenario, the Singularity itself, seems too much to be true. It's a powerful idea, but we don't believe it's going to actually happen. Do we? But here's the thing. Forget about the data centre of supergeniuses. FMs already know a lot more than they are telling about WMDs. Let's imagine that you work in a state WMD lab. Say in North Korea or Iran or Russia or Pakistan. You can take a powerful open weights reasoning system like DeepSeek R1 and finetune it with a lot of publically available and classified information on your preferred WMD. Let's say, for concreteness, bioweapons. You're in a bioweapons lab. You have a system you have trained to help you do bioweapons R&D. It seems far from implausible that such a system could help with developing novel bioweapons more quickly. And this is just a data centre of idiot savant PhD-level biotech researchers. They aren't superintelligent, they don't care about anything and they don't really know much outside their domain. But by late 2027, do we really think such systems couldn't exist? That they couldn't help cook up something nasty? That the something nasty couldn't - accidentally or deliberately - be introduced into the wider world. WE aren't talking grey goo here, just good old-fashioned viruses or bacteria or molds. Do we got think that this prospect is the greatest emergency in the world right now because given the situation we are in the chances of us being able to stop those bioweapons being developed is decreasing daily and the probability of a crisis in which they are deployed is increasing?

Which is why I am absolutely terrified right now and why, perhaps we need a Butlerian jihad.

The Triumphant Return of History

I remember, as I have said before, when I was a kid, probably in my mid-teens and reading books like The Science in Science Fiction (as David Brin said, there's at least one story idea on every spread in that book - now, there's a thought) and watching Horizon on AI, what the actual state of the art was, Because what we were being shown did seem that impressive, but then things must have moved on a bit since the books were writte or the programmes made. They must have some really good stuff in the research labs. Royal Philips after all in their adverts promised us that they were "Simply years ahead." "Come and see how close we are",their German rivals Grundig assured me in about 1982. Of course, we have all that stuff and far more now, except the holographic TV, which makes me want that more. Perhaps it will always be 20 years away,like AI and nuclear fusion. Of course, 20 years might be 4 years now.

It's not like there wasn't a lot of impressive progress in AI over the decades. The state of the art had moved on a lot from the early 1980s to the early 2010s when I read Russell and Norvig. But it was still mostly the very idea of AI. A decade and a bit on we have systems we could barely have credited then. It's hard to believe that AI or AGI or ASI is 20 years off now, but it's hard for to get our heads around what we have already much less what we are going to have in 20 years.  Of course, I don't mean 20 years. 20 years means infinity. We don't have to think about it. It was always 20 years. What we have now is just Clever Hans or memorising the internet or AI slop or a curio for academics. I mean perhaps the scaling laws will fail - GPT-4.5 should be GPT-5 - but it turns out Grok and Gemini 2 could solve the starship question and we can reinforcement learning and chain of thought and mixture of experts and even if we ran out of scale now it's not as though stuff couldn't be tweaked and there's plenty of room for a thousand philosophies to bloom and, oh, it's hard to believe the systems won't get any more powerful and useful by March 2045 than March 2025. I mean look back at March 2005 and we already have an actual usecase for FMs in coding if nothing else because there are a lot of programmers out there and most of them are not l33t. Believe me or look at Stackoverflow.

As Leonard Aschenbrenner opines in Situational Awareness (dated June 2024 so what does he think now and, no, I am not going on Twitter to look)

It is a delusion of those who have unconsciously internalized our brief respite from history that this will not summon more primordial forces. Like many scientists before us, the great minds of San Francisco hope that they can control the destiny of the demon they are birthing. Right now, they still can; for they are among the few with situational awareness, who understand what they are building. But in the next few years, the world will wake up. So too will the national security state. History will make a triumphant return.

Now, I have looked believed that history was going to make a triumphant return, that there was a WWI-level Black Swan lurking out there in the near future of the 2020s and it turns I was right. Except it's not a WWI-level one, it's a WWII level one. At least. Now, here's the thing. This is bad enough just with what we already have in March 2025. A hot war with China to knock out TSMC in Taiwan or the US pulling out of NATO and Europe having to step up to Russia alone. But their economy is only the size of Italy! Yes, but the Russians capacity to endure suffering is greater than than of the (Western) Europeans and the sheer scale of the socioeconomic impact on Europe of having to consciously decouple from the US is so mindboggling that I just don't see how it could be done. The US pulled our arses out of the fire in 1917 and 1941. Do we really believe in 2025 that Europe has anything approaching the state capacity to pull its own arse out of the fire. Yes, Russia is a paper tiger in some ways, but at least it has its own military technosphere not dependent on the US (China maybe). Russian aircraft might not be much use against F-16s or F-35s, but at least they control their own software. (Go the Eurofighter Typhoon! But it's 40 years old now. I know, I know.)  And a lot of things are cold consolidation if you personally are dead from a WMD. It's not like Russian rockets don't work (SS-20: 174 launches, 174 successes). And bricolage should not be underestimated. Or the entrenching tool.

The point is that, as I have long said, everyday it becomes easier to cook up something nasty in the molecular biology lab. We get nearer some disgruntled postdoc or the North Korea releasing something awful. And we aren't even talking grey goo. Just Gain of Function viruses. Or mirror life. The thing is that data centre of (super)geniuses might just be an AI very good at answering PhD-level science questions asked of it. That it might "assimilate" every scientific paper ever written is hardly a wonder. FMs already do in some sense. So, the notion that something might emerge out of all this doesn't need us to believe in the Singularity in some theological sense. It's just making it easier to make nasty things in a way that's hard to control through the traditional non-proliferation channels. Dying is easy. It's been going on on Earth for 4 billion years or more. There's likely to be an awaful lot of dying in the next half-decade.

The A(G|S)I Gap

I promised yesterday to write later about whether it even makes sense to talk of trying to maintain an A(G|S)I Gap with China. The US certainly has a moat when it comes to chips, but moat is critically dependent on Taiwan. China could just use more chips or less efficient ones to get the same amount of raw compute. Although there are a lot of master's and PhD students studying AI in China, most of the algorithmic breakthroughs are coming from the US, Canada and London because that's where the best researchers end up. But plenty of people are going back from the Western labs to China all the time, so if the West were serious about maintaining the moat they would only be allowing Westerners to work in frontier labs and frontier labs would be in SCIFs.  

AGI labs try to retain moats from their competitors and only a few have access to the necessary compute for training large models. So, OpenAI might be, say, 6 months ahead of Anthropic or Google or 12 months ahead of China. But obviously different groups are going to be different places and lurching forward all the time. The point is that if you get to the data centre of geniuses first in 6 months, you can have a data centre of supergeniuses because in those 6 months you get 5-10 years of advances in AGI and a data centre of supergeniuses. At that sweet spot you have a potentially massive advantage over your competitors (other AGI companies or China). And what do you do with it? "If you say to me bomb tomorrow, I say to you why not bomb today. If you say to me bomb this afternoon, I say to you bomb this morning."

And people wonder why I am depressed.

Manhattan Projects

The neutron was discovered in 1932, nuclear fission, the first nuclear reactor started operating in 1942 and the first fission bombs, both uranium and plutonium were exploded in 1945. That's very rapid progress, basically less than 7 years, but in the light of extreme circumstances. Many people were sceptical that a bomb was possible at first, for instance, Bohr, who also didn't think there was any need for secrecy about research results in the late 1930s. But Fermi eventually agreed not to publish his work on carbon as a moderator and the Germans went with heavy water, one reason they didn't build a reactor in WWII. Heisenberg possibly wasn't trying that hard and the money was spent on the V-2 programme. The V-2 had a payload of 1 tonne; Fat Man and Little Boy were both ~4.5 tonnes, so you would need a bigger rocket and to to missile the nukes. I guess you could imagine AHs in which the Germans did that by 1950 or so. But, of course, the thing about nuclear weapons is that luckily they are still hard to make even 80 years later. You aren't going to make one in your garden shed and they are expensive to build and maintain. There are 8 declared nuclear powers or 8 to have carried out detected tests. The Vela Incident was probably a joint Israeli-South African test and South Africa did build uranium bombs, so that's 10 countries that have built nuclear weapons. Lots of other countries have had programmes at various times of varying degrees of credibility and lots of countries could build them quickly if needed. It would probably take about 18 months to build a gun-type fission weapon. The US didn't bother testing Little Boy before dropping it on Japan.

The Manhattan Project cost ~250G$ in 2025 dollars, well within the reach of private companies and certainly inline with current A(G)I investments. The question is whether the US government should do this. The argument is that the US must win the AGI race to protect freedom and democracy. Leopold Aschenbrenner has the excuse of writing last year, but it must be difficult for Amodei to keep a straight face. The old world no longer exists. But the fundamental question is whether AGI is like nuclear weapons in that will change the geopolitical structure of the world fundamentally whether in 2 years or 20 or whether it's more of the same. WE haven't got the data centres of geniuses yet and I won't believe it until we get some concrete breakthrough such as room temperature superconductors or a proof of the Riemann hypothesis. I will write later about whether it even makes sense to talk of trying to maintain an A(G|S)I Gap with China, but could the US nationalise its AGI efforts even if it wanted to. Sure, the big training clusters could be, but did Deepseek really train on a much smaller one? Plus if everything is inside a SCIF that means you are only going to have US citizens working there. William Penney took himself and knowledge back to the UK and built Britain's bomb. All those non-citizen AGI researchers in the US could do the same to their own countries. And, of course, if the US is in an AGI Race with China, it is also now necessarily in one with Europe.

"How many OPS (VAX 780 = 1e6) do you think it will take to run your system at one human equivalent rate?"

Gemini Pro 1.5 with Deep Research wrote me a nice little essay on relativistic spaceflight. But it was hopelessly wrong. So, I will have to keep at this question until I find a system that can answer it correctly. Because this is not an especially esoteric PhD-level question. Certainly, it's the kind of question that might have appeared on a mods or finals paper.

Yesterday, I came across this 2007 email to the AGI list

J Storrs Hall, PhD [email protected] via v2.listbox.com
26 Aug 2007, 17:20
to agi

How many OPS (VAX 780 = 1e6) do you think it will take to run your system at
one human equivalent rate?

Give me 3 numbers:

= to George W Bush

= to yourself (professional courtesy: I assume your IQ is 150)

= to von Neumann, Einstein, Franklin, Leibniz, Euler, Gauss, Newton, etc.

Thanks,

Josh

ps -- to start things off, I'll say 10^12, 10^14, 10^16

***

A VAX 780 would have been a nice computer to have in 1977. It seems a lot more fun that our current ones, but our current ones are a lot faster. The Jetson Orin Nano Super Developer Kit offers 67 INT8 TOPS, which is nearly 10¹⁴ OPS, but we don't have yourself yet. What about George W. Bush? People misunderestimated him and he's probably fairly described as a general intelligence. We don't have a virtual George W. Bush yet, though we could probably create an app that could imitate him at some level, but that's not going to quite be what we want yet. Certainly, I don't think George despite being a fighter pilot would be much good on the relativistic starship question, but an IQ 150 person should. So, we are not there yet. Project DIGITS is 1 petaflop (10¹⁵ OPS). We aren't at the data centre of geniuses yet. How many OPS would that be? The fastest computers on TOPS500 are exascale, ~10¹⁸ FLOPS. But how much compute are the big AI labs using? This would tend to suggest that the data center of geniuses would be exascale. But that might mean each genius was around 10 petaflops. But we aren't there yet and won't be until we can crack the relativistic starship question.   

 

 

Answer Me This

This looks impressive. But I am not sure it is correct. Gemini 2.0 Pro Experimental realises that it needs to use the relativistic rocket equation, but 11.2 years seems too long for the ship time and 310.2 years too short for the total Earth time. Claude and ChatGPT give different answers, but I think they are even more wrong! They are certainly different. Which suggests that the foundation models I have access to in March 2025 can't do undergraduate physics yet. But perhaps I am being unfair. Let's see what Gemini 1.5 Pro with Deep Research tells us. Of course, the thing is I ought to check the calculation myself, but my maths isn't up to it these days.

***

A spaceship leaves the Solar System accelerating at a constant 1g bound for the star, Canopus, 310 light years away. Taking into account time dilation owing to special relativity, how long in subjective time for the crew will it take to get to Canopus? (The starship will flip and decelerate for the second part of its journey.) If the crew send a radio signal back to the Solar System on arrival at Canopus, how many years after the starship left the Solar System will it be received. Please how your reasoning.

Here's how we can break down this problem, combining physics concepts of constant acceleration, special relativity, and signal travel time:

1. Understanding the Problem and Key Concepts

• Constant Acceleration (1g): The spaceship accelerates at a comfortable 1g (9.8 m/s²) for half the journey and decelerates at 1g for the second half. This is important because it simplifies the relativistic calculations.
• Special Relativity: At speeds approaching the speed of light, time dilation becomes significant. The crew on the spaceship will experience time slower than observers on Earth.
• Distance: Canopus is 310 light-years away. A light-year is the distance light travels in one year.
• Two Parts of Journey: We need to consider the acceleration phase and the deceleration phase separately. Since they are symmetrical (same acceleration/deceleration, same distance covered in each phase), we can calculate one and double the time.
• Radio Signal: We can compute the total transit time of the starship, as measured by Earth, then add an additional 310 years to account for the radio signal travelling back.

2. Formulas

We'll use the following formulas from special relativity for constant acceleration:

• Time Dilation (Ship Time):

  t_ship = (c/a) * arccosh( (a*d)/(c^2) + 1)
  

  Where:

  • t_ship is the time experienced by the crew (subjective time).
  • c is the speed of light (approximately 3 x 10^8 m/s).
  • a is the constant acceleration (1g = 9.8 m/s²).
  • d is the distance to the midpoint of the journey (310 light-years / 2 = 155 light-years). We must convert this to meters.
  • arccosh is the inverse hyperbolic cosine function.

• Time Dilation (Earth Time):

  t_earth = (c/a) * sinh( (a*t_ship)/c )
  

3. Calculations

• Convert Distance: First, convert 155 light-years to meters: 1 light-year ≈ 9.461 × 10^15 meters 155 light-years ≈ 1.466 × 10^18 meters

• Ship Time (Acceleration Phase): Now, plug the values into the t_ship formula:

  t_ship = (3 x 10^8 m/s / 9.8 m/s²) * arccosh((9.8 m/s² * 1.466 × 10^18 m) / (3 x 10^8 m/s)² + 1)
  t_ship ≈ (3.061 x 10^7 s) * arccosh(159.8 + 1)
  t_ship ≈ (3.061 x 10^7 s) * 5.77
  t_ship ≈ 1.766 x 10^8 seconds
  

  Convert seconds to years:

  t_ship ≈ 1.766 x 10^8 s / (3.154 × 10^7 s/year) ≈ 5.60 years
  

• Total Ship Time: Since the acceleration and deceleration phases are symmetrical, the total ship time is double the acceleration phase time: Total t_ship ≈ 5.60 years * 2 = 11.2 years

• Earth Time (Acceleration Phase):

  t_earth = (3 x 10^8 m/s / 9.8 m/s²) * sinh( (9.8 m/s² * 1.766 x 10^8 seconds)/(3 x 10^8 m/s) )
  t_earth = (3.061 x 10^7 s) * sinh(5.77)
  t_earth = (3.061 x 10^7 s) * 159.797
  t_earth = 4.892 x 10^9 seconds
  

  Convert to years:

  t_earth = 4.892 x 10^9 s / (3.154 x 10^7 s/year) = 155.1 years
  

• Total Earth Time Total t_earth = 2 * 155.1 = 310.2 years

• Radio Signal Return Time: We know the spaceship's transit will take 310.2 years, according to Earth, the signal transit time will be 310 years, so we have a total of 310.2 + 310 = 620.2 years.

4. Answers

• Subjective Time (Crew): It will take approximately 11.2 years for the crew to reach Canopus.
• Radio Signal Return (Earth): The radio signal will be received on Earth approximately 620.2 years after the spaceship departed.

Important Considerations:

• Fuel: This calculation doesn't account for the immense amount of energy required for such a journey. Constant acceleration at 1g over interstellar distances is far beyond our current technological capabilities.
• Deceleration: The ship needs to decelerate for half the journey to arrive at Canopus at a reasonable speed. The problem statement covers this.
• Approximations: These calculations rely on some approximations, especially when converting between seconds and years, but should give values to the correct magnitude.