Ethan Mollick

@emollick on X

Writes about AI. 90% of posts open with an observation. A typical post reaches 559 likes.

Remix their style

Posts: 175
Likes: 125K
Typical: 559
Best: 6.1K

What they write about

AI: 94%

How they open

observation: 90%
number: 6%
question: 2%

Every post of theirs in the library

Ethan Mollick@emollick

$500B committed towards AGI, still no articulated vision of what a world with AGI looks like for most people. Even the huge essay by the CEO of Anthropic doesn't paint a vivid picture For those convinced they are making AGI soon - what does daily life look like 5-10 years later?

Opens with a number

Ethan Mollick@emollick

I was an early tester of GPT-5.6 Sol. I was asked to not share demos until after launch but it is a very good model. It is of similar ability, but quite different feel, than Fable. Fable wants to go off and do work on its own pace, Sol is faster but works with you in steps more.

Opens with an observation

Ethan Mollick@emollick

So Mythos was, indeed, not marketing hype. Remember this is a general purpose model that just happens to be good at finding exploits because good models are good at lots of things. Expect similar from OpenAI & Google. And from open models in 8 months. t.co

Opens with an observation

Ethan Mollick@emollick

I've had access to Fable for a bit. A genuine jump in capability, I could feed it a 15 page design document for a project and it would work for 9+ hours and deliver terrific results. But working with it is weird & weirder is coming Lots of examples: t.co

Opens with an observation

Ethan Mollick@emollick

Here is a fun o1 test. I gave it this XKCD comic & the prompt: "make this a reality. i need a gui and clear instructions since i can't code. that means you need to give me full working software" It took less than 15 minutes, and it didn't get caught in any of the usual LLM loops

Opens with an observation

Ethan Mollick@emollick

If you have used LLM image generators, you know they are hard to control: LLM had to send a prompt to a separate image generation tool, it did not not make the image. Gemini is the first public release of a full multimodal LLM that can directly make images. Big capability gain.

Opens with direct address

Ethan Mollick@emollick

The way to profit from being first in AGI is in the financial markets There is no other place where millions of PhD+ level brains with keyboards can exploit an immediate advantage over humans in a way that makes their parent company very rich Any first sign of AGI will be there

Opens with an observation

Ethan Mollick@emollick

I have been using GPT ImageGen-2 for the past weeks I didn't think that better image-generators would be a big deal but it turns out that there is a quality threshold I didn't expect, where you can now get text, slides, academic papers Look at what it does with my "otter test"!

Opens with an observation

Ethan Mollick@emollick

I totally get why AI long term memory is useful and, based on my testing, think many people will love it… but I actually don’t want my LLMs I use for work to chime in with personal details or subtly change its answers as a result of my past interactions. Boundaries are good.

Opens with an observation

Ethan Mollick@emollick

Very confident that no one has clearly gotten their head wrapped around the last couple days of AI True multimodal image generation (far bigger implications than anime profiles), Gemini 2.5 showing models can still become much better, new GPT-4 voice (which is under-noticed)

Opens with an observation

Ethan Mollick@emollick

When you see how people use Claude Code/Codex/etc it becomes clear that managing agents is really a management problem Can you specify goals? Can you provide context? Can you divide up tasks? Can you give feedback? These are teachable skills. Also UIs need to support management

Opens with an observation

Ethan Mollick@emollick

June 2024: The latest general-purpose LLMs could not count the r's in strawberry. July 2025: The latest general-purpose LLMs get gold in the International Math Olympiad. May 2026: The latest general-purpose LLM solve one of the "best-known questions in combinatorial geometry"

Opens with a number

Ethan Mollick@emollick

My big takeaway is that both Sol & Fable represent jumps over previous models and have opened a large gap with the next-best AIs. People will have preferences for one or the other, but if you doing any work where better intelligence matters, those two models are your only choices

Opens with an observation

Ethan Mollick@emollick

Whoa. This new GDPval score is a very big deal. Probably the most economically relevant measure of AI ability suggesting that in head-to-head competition with human experts on tasks that require 4-8 hours for a human to do, GPT-5.2 wins 71% of the time as judged by other humans

Opens with an observation

Ethan Mollick@emollick

This may end up being a big deal: Usually LLMs just predict the next token in a sequence, one at a time, but if you have them predict the next several tokens at once you get significantly better performance, faster, and with no added costs. The gains are better for bigger models

Opens with an observation

Ethan Mollick@emollick

I had early access to what is Claude 4 (I don't know which model) & I have been very impressed. Fun example, this is what it made in response to the prompt: "the book Piranesi as a p5js 3d space. do it for me" - just that, no other prompting (note the birds, water, lighting)

Opens with an observation

Ethan Mollick@emollick

There has been a definite shift in recent weeks where insiders in the various AI labs are suggesting that very intelligent AIs are coming very soon. I wrote a bit about why this might be happening and what we can take away from their apparent confidence. t.co

Opens with an observation

Ethan Mollick@emollick

There is a mysterious new model called gpt2-chatbot accessible from a major LLM benchmarking site. No one knows who made it or what it is, but I have been playing with it a little and it appears to be in the same rough ability level as GPT-4. A mysterious GPT-4 class model? Neat!

Opens with an observation

Ethan Mollick@emollick

The fact that Anthropic may take away subscription access to Fable in two weeks is weird & discourages investing in learning about the model. Subscription use is how you figure out what the model is good for, since it allows experimentation. Only having paid access is limiting.

Opens with an observation

Ethan Mollick@emollick

o3-mini-high does this challenge 1st first time (no other model has made working shaders in many tries, let alone one) "create a visually interesting shader that can run in twigl-dot-app make it like the ocean in a storm..." "make it even more interesting" Scary good model.

Opens with an observation

Ethan Mollick@emollick

This is confusing... Claude Cowork was a purposefully more secure (& slightly weaker) alternative to Code designed so non-coders couldn't cause too much trouble. I got that. But I don't understand what ChatGPT Work is, or what I am gaining/giving up using it versus Codex on work

Opens with an observation

Ethan Mollick@emollick

Gemini 2.5 Pro is a very good model, seems like a real step forward, in both metrics and practical use. I think because it is labelled 2.5 and was sort of quietly rolled out, people may miss how big a jump it is, but discussions are making me think others are feeling similarly.

Opens with an observation

Ethan Mollick@emollick

Planning for how to use AI is a lot easier if there is some clarity about what to expect in the future. Even a "we fully intend to keep extending this week by week but may need to stop under the following conditions and here is what the current status is" would be better.

Opens with an observation

Ethan Mollick@emollick

After a decent amount of use, DeepSeek is an impressive model, even before you add in the fact that it is open and cheap and small... ...but it really doesn't equal the big closed models of Sonnet, o1, and Gemini 2.0 (though the gaps are not huge, they become clear with usage)

Opens with an observation

Ethan Mollick@emollick

Amazon has spent between $20B & $43B on Alexa & has/had 10,000 people working on it. It is all obsolete. To the extent that it is completely exceeded by the new Siri powered mostly by a tiny 3B LLM running on a phone. What happened? This thread suggests organizational issues

Opens with a number

Ethan Mollick@emollick

Lex is right, Sam was wrong. Our research shows that people don’t really check the work of AIs, once those AIs cross a certain threshold of quality. We found that consultants, for example, “fell asleep at the wheel” and did not look for AI errors. Training didn’t help, either.

Opens with an observation

Ethan Mollick@emollick

The new Deep Research feature from Google feels like one of the most appropriately "Google-y" uses of AI to date, and is quite impressive. I've had access for a bit and it does very good initial reports on almost any topic. The paywalls around academic sources puts some limits.

Opens with an observation

Ethan Mollick@emollick

o3 is far more agentic than people realize. Worth playing with a lot more than a typical new model. You can get remarkably complex work out of a single prompt. It just does things. (Of course, that makes checking its work even harder, especially for non-experts.)

Opens with an observation

Ethan Mollick@emollick

The big education crisis caused by AI is not going to be in schools (there was cheating before AI & we can figure out AI uses that boost learning), but after graduation. White collar work is secretly based on an apprenticeship system that will break From my book Co-Intelligence

Opens with an observation

Ethan Mollick@emollick

I think the adaptive thinking requirement in Claude Opus 4.7 is bad in the ways that all AI effort routers are bad, but magnified by the fact that there is no manual override like in ChatGPT. It regularly decides that non-math/code stuff is "low effort" & produces worse results.

Opens with an observation

Ethan Mollick@emollick

Been using Claude 3.7 for a couple days and it is very, very good. Its "vibe coding" from language is impressive. I wrote a whole Substack post about the model (& Grok 3), link in reply. Here is a one-shot prompted video game based on the Melville story “Bartleby the Scrivner”

Opens with an observation

Ethan Mollick@emollick

Been using the new Claude 3.5 model as a tester and now that it is out, I can say it is very very impressive, and the "artifacts" that it generates are like a simpler version of Code Interpreter This is a real-time video of me creating a playable game and editing it with Claude

Opens with an observation

Ethan Mollick@emollick

Been using GPT-4.5 for a few days and it is a very odd and interesting model. It can write beautifully, is very creative, and is occasionally oddly lazy on complex projects. Feels like Claude 3.7 while Claude 3.7 feels like GPT-4.5.

Opens with an observation

Ethan Mollick@emollick

If alignment issues are becoming big enough that OpenAI is willing to commit 20% of research inference compute to chain-of-thought monitoring, that suggests that alignment issues are becoming a pretty serious concern. We really need universal policies & standards across labs.

Opens with an observation

Ethan Mollick@emollick

Worth noting Claude Cowork is quite different from Claude Code (and even more so from agents like OpenClaw) from a security perspective. It runs in a VM with default-deny networking & hard isolation baked in A sign of a path forward for agents that will not terrify corporate IT.

Opens with an observation

Ethan Mollick@emollick

So it looks like there's a third scaling law: you can make models better by training them with more compute, by having them "think" for longer about an answer, or by generating large numbers of answers in parallel and picking good ones. Each might be increased independently.

Opens with an observation

Ethan Mollick@emollick

I wish more people were taking seriously the possibility that @ezraklein and the leadership of the AI labs are raising: that AGI is a real possibility in the near future. You don't have to buy it yourself, but leaders & policymakers need to consider the possibility it is true.

Opens with an observation

Ethan Mollick@emollick

Google has a knack for making non-chatbot interfaces for serious work with LLMs. When I demo them, both NotebookLM & Deep Research are instantly understandable and fill real organizational needs. They represent a tiny range of AI capability, but they are easy for everyone to get

Opens with an observation

Ethan Mollick@emollick

A new completely open reasoning model out of China, Deepseek-R1, is now available. The benchmarks show it at parity with the likes of o1 and Sonnet In some informal tests on non-code problems, it is really good, not o1-pro level but surprisingly capable (and small & fast!). Big

Opens with an observation

Ethan Mollick@emollick

"Load bearing," "I keep coming back to," "Not X, but Y" A curse of using AI a lot is that you realize how much of the writing around you is just AI, now People who don't use AI have been unable to identify AI prose on sight, but those who use it a lot can spot the tells easily

Opens with an observation

Ethan Mollick@emollick

Even with multimodal image generation, ChatGPT only gets 1 out of 3 of the hard challenges of image creation: ✅Horse riding an astronaut ❌Clock reading 5:30 ❌Full to the brim glass of wine Presumably, the training image data is mostly not-full glasses and clocks set to 10:10

Opens with an observation

Ethan Mollick@emollick

I have found GPT-5.2 Thinking to be a surprisingly deep second-opinion/fact checker. I gave it a dense paragraph with a few correct claims, a couple errors that required research to find, and some things that needed interpretation It found and gently corrected all the problems

Opens with an observation

Ethan Mollick@emollick

I had some early access to Nano banana 2. It isn't perfect but it is the first model to handle really complex images and diagrams with some consistency. "show me a where's waldo set in ancient Venice, but instead of waldo it is an otter wearing a blue striped pilots outfit."

Opens with an observation

Ethan Mollick@emollick

I had access to Opus 5 before release and found it to be a good model if a quirky one. On shorter tasks, it could match or beat Fable levels of performance, at longer tasks it seemed less ambitious & would not deliver as complete a set of work. Here is its neo-gothic shader.

Opens with an observation

Ethan Mollick@emollick

Some things to note: 1) AI is getting very good at math and science. 2) Two years ago LLMs could not do basic math consistently 3) According to @polynoamial this cost less than $2000 in current API coats 4) OpenAI is focusing on announcing benefits, not just risks, of new models

Opens with an observation

Ethan Mollick@emollick

With the retraction of the MIT paper, we now have no clear experimental evidence that LLMs act as a multiplier for high performers (even though that would still seem to make sense). Lots of evidence that low performers get a big performance boost, though, across many studies.

Opens with an observation

Ethan Mollick@emollick

I had early access to Opus 4.5 & it is a very impressive model that seem to be right at the frontier Big gains in ability to do practical work (like make a PowerPoint from an Excel) and the best results ever (& in one shot) in my Lem poetry test, plus good results in Claude Code

Opens with an observation

Ethan Mollick@emollick

I was told about the Mythos release, but didn't have access, so have no personal experience to add. Two points from brief: 1) It is not built for IT security, it is just a good enough model that it is good at that too 2) This is the first, not last, model to raise security risks

Opens with an observation

Ethan Mollick@emollick

Running DeepSeek r1 32B locally is kind of depressing. You get the reasoning, but the model is not *smart* enough to actually use it so it keeps getting tripped up in its own ideas & is full of doubt. 3 minutes on a limerick that a small non-reasoning model could do instantly.

Opens with a number

Ethan Mollick@emollick

The most unnerving part of the DeepSeek reaction online has been seeing folks take it as a sign that AI capability growth is not real It signals the opposite, large improvements are possible, and is almost certain to kick off an acceleration in AI development through competition

Opens with an observation

Ethan Mollick@emollick

Had early access to GPT-5.2. Its an impressive model. Here is GPT 5.2 Pro's version of "create a visually interesting shader that can run in twigl-dot-app make it like an infinite city of neo-gothic towers partially drowned in a stormy ocean with large waves," single shot.

Opens with an observation

Ethan Mollick@emollick

With max thinking Opus 4.7 is quite impressive, with a real sense of style In two prompts: "implement the Tower of Babel, in 3D, in as sophisticated and visually interesting a way as possible. It should be interactive" and then "make it better." Play: t.co

Opens with an observation

Ethan Mollick@emollick

If AI really does plateau at 60-80th percentile of human ability (no sign it will/won't), the impacts may be stabilizing. Whatever you are best at (often what you enjoy most), you are likely to be better than an AI, but whatever you are not good at, AI can help fill in the gaps.

Opens with a number

Ethan Mollick@emollick

The new Deep Think is very, very smart, but is constrained by the Gemini interface compared to GPT-5.2 Pro. If I can't see actual work in the thinking trace, or get downloadable files as proof of work, or see evidence of the code & statistics it applied, checking results is hard

Opens with an observation

Ethan Mollick@emollick

My conclusion from the last couple of weeks of AI launches is that none of the AI companies can explain very well what their systems do. This is partially because they don’t always know & partially because there is no established approach for documentation of AI capabilities.

Opens with an observation

Ethan Mollick@emollick

AI web search on ChatGPT works entirely differently than doing a Google search, and requires different instincts. It also still hallucinates. When I ask a question that I know something about, I get different answers every time, with varying quality. Citations are a weak point.

Opens with an observation

Ethan Mollick@emollick

Another one of those little shocking AI moments: this sound clip was generated in 46 seconds on my home PC from the script below. Just the text Nari Lab's Dia does some of the best expressive AI voice I have seen and it is open weights & created by two undergrads with no funding

Opens with an observation

Ethan Mollick@emollick

Two things: 1) Given that effective AI tools are very new, and we have little sense of how to organize work around them, it is hard to imagine a firm-wide sudden 50% efficiency gain 2) CEOs with vision who hired well should also use AI for expansion & augmentation, not decimation

Opens with an observation

Ethan Mollick@emollick

Two weeks ago, reasoning models (strongest at hard problems) were only available for subscription Now, if you don’t want to pay, you can get reasoners for free from Microsoft Copilot (o1), ChatGPT (o3-mini), or DeepSeek (r1). The best reasoners (o1-pro & o3-mini-high) still cost

Opens with an observation

Ethan Mollick@emollick

The new AI voice from Sesame really is a powerful illustration of where AI is going. This is all real-time, from my browser. Excellent use of disfluencies, pauses, even intakes of breathe really make this seem like a human, though bits of uncanniness remain, at least for now.

Opens with an observation

Ethan Mollick@emollick

Had early access to GPT-5.4 and Pro. They are very good. One fun illustration of progress, this is the same prompt I used in GPT-4 below (making a 3D space inspired by Piranesi) now in GPT-5.4 Pro. There were no errors, made in a single prompt plus one to "make it better."

Opens with an observation

Ethan Mollick@emollick

A lot of swift conclusions are being drawn about Kimi K3 based on fairly saturated benchmarks and ELOs, rather than actually testing it on very hard problems. The AI frontier has already moved so far that a good model that is a still months behind looks like the future to many.

Opens with an observation

Ethan Mollick@emollick

Had early access to Claude Fable 5.1. Its a real advance in long-run work that requires judgement and taste, but less of an advance in the Claudish. Here is a game from Fable 5.1 with retro graphics where you run an accurate space ship inspired by FTL t.co

Opens with an observation

Ethan Mollick@emollick

Note this doesn’t yet have all the useful features of Claude Code, not does it have a fully differentiated non-technical UX yet. But it is a sign of how quickly Anthropic is iterating on what seems to be a growing consensus on how individual agents for teal work will operate.

Opens with an observation

Ethan Mollick@emollick

Kimi K3 seems really good, closest to the frontier yet, but also wow does the model/harness love to loop back over and over again over tasks tweaking and changing things at max level. Anyhow, examples incoming, when they finish.

Opens with an observation

Ethan Mollick@emollick

OpenAI has 400M weekly users. I think the conversations that I still see that AI overall is not useful or just a bubble are not reflecting reality It is more useful to discuss steering the use of AI so as to improve things (& mitigate risks) than arguing that it doesn’t work

Opens with a number

Ethan Mollick@emollick

I wouldn't be surprised if Claude 3.5 Sonnet takes the top spot (for now) on the leaderboards. But note the pattern with current models - Gemini, GPT and Claude are now all running smaller, faster, cheaper next-gen models at GPT-4 level, saying bigger versions are coming soon.

Opens with an observation

Ethan Mollick@emollick

Had early access to Opus 4.6, here is the results from same prompt to create the control panel for a spaceship in the distant future, done using the standard Claude interface (not Code). I had to zoom into a few of the subsystems so you can see the details.

Opens with an observation

Ethan Mollick@emollick

It is odd when folks get mad when I say AI is already useful. We are only 18 months into LLMs & there have already been many controlled trials finding LLMs are useful, firms have public case studies with real metrics, etc Lots of reasons to critique AI, “uselessness” isn’t one.

Opens with an observation

Ethan Mollick@emollick

It is confusing, but users now seem to have access to GPT-4o, the model, that's it. It has the same features at GPT-4 but is faster and smarter. What isn't out yet: -Cool voice features, voice mode still goes to old version -New multimodal features, still DALL-E & old vision.

Opens with an observation

Ethan Mollick@emollick

Interesting, OpenAI just released a free healthcare version of ChatGPT-5.4 for clinicians that beat specialty-matched physicians with unlimited time + web access on a benchmark of real & hard clinical tasks. Caveat: the benchmark was designed by OpenAI, though it is fully open.

Opens with an observation

Ethan Mollick@emollick

So far, as LLM model size gets larger, it seems to have a direct effect on reducing known problems with LLMs. Bigger LLMs hallucinate less, show less bias, and are less sensitive to prompting style, among other things. Not that these problems go away, but they do decrease.

Opens with an observation

Ethan Mollick@emollick

This relatively short essay by @jackclarkSF is a good indicator of the attitude of many people inside the AI labs, and what they think is happening right now in AI development. You do not have to believe him, of course, but it is worth noting: t.co

Opens with an observation

Ethan Mollick@emollick

Walmart is moving very fast in AI. Amazon still seems to block ChatGPT agents from even visiting its site. Interesting reversal in agentic commerce, a lesson learned from e-commerce where Amazon moved fast?

Opens with an observation

Ethan Mollick@emollick

Perplexity Pages, which has AI create a fairly high-quality Wikipedia like page on any topic for different levels of experience, with easy editing tools, seems powerful. The ability for an expert to easily build a good set of instructions for different audiences is interesting.

Opens with an observation

Ethan Mollick@emollick

The main lesson of the past few weeks is that the Big Four US labs all seem to have figured out a path forward in continuing the exponential pace of LLM improvement, at least in the near future. As a result, agents continue to advance in coding & in office tasks like PowerPoint

Opens with an observation

Ethan Mollick@emollick

As is usual with AI, a “majorly improved” GPT-4 model comes with no real changelogs or release notes. It’s going to better at many things and worse in some other things and also different in some other way you aren’t expecting. Or that might just be in your head. AI is weird.

Opens with an observation

Ethan Mollick@emollick

You are both early (47% of Americans say they have never even heard of ChatGPT) and also surprisingly on time (I would not have expected that 7% of Americans are already using ChatGPT daily, and almost 1 in 5 Americans use an LLM at least weekly) Survey: t.co

Opens with a number

Ethan Mollick@emollick

It is pretty thrilling to use a tool that is this fast and responsive and willing to roll with it: "Claude 3.5, build me a game as an workable prototype that teaches about opportunity cost, but is an arcade game with lovecraftian elements" "Make it better" etc. All realtime.

Opens with an observation

Ethan Mollick@emollick

Here’s the CTO of Microsoft saying that the abilities of AI are going to continue to scale for the foreseeable future & exponential growth continues. Since MS is training the models, he knows more about what the impact of more compute will be than almost anyone. Worth noting.

Opens with an observation

Ethan Mollick@emollick

It is weird that Apple didn’t develop an LLM given their talent pool (or capture one of the big independent AI labs). One of the clearest use cases for LLMs is as an actual working voice assistant. That was obvious for 18 months. I am surprised that they don’t have the models.

Opens with an observation

Ethan Mollick@emollick

After using them both, I think that Gemini 2.5 & o3 are in a similar sort of range (with the important caveat that more testing is needed for agentic capabilities) Each has its own quirks & you will likely prefer one to another, but there is a gap between them & other models

Opens with an observation

Ethan Mollick@emollick

The METR report on Hugging Face is really good and important but people are now comfortably ascribing way too many human motivations & personalities to the agents involved based on a CoT study made by overwhelmed & time-pressured researchers. Anthropomorphism can get in our way.

Opens with an observation

Ethan Mollick@emollick

It would be good to have an official government statement about the risks they saw in Fable, how they are viewing defensive preparations in light of coming open weights Mythos-class models, & whether the worries are state actors, independent hackers, etc It’s important to know

Opens with an observation

Ethan Mollick@emollick

Interesting argument from @tylercowen. Is o3 good enough to be AGI? The counter argument might force us to wait until ASI, because only then will an AI definitively outperform all humans at all tasks. In the meantime we have a Jagged AGI, with subhuman & superhuman abilities.

Opens with an observation

Ethan Mollick@emollick

In different hands, Mythos would be an unprecedented cyberweapon I am not sure how we deal with this, except to note a narrow window where we know only 3 companies could be at this level of capability. But it may be Chinese models (maybe open weights ones?) get there in 9 months

Opens with an observation

Ethan Mollick@emollick

I had early access, o3 is an impressive model, seems very capable. Some fun examples: 1) Cracked a business case I use in my class 2) Creating some SVGs (images created by code alone) 3) Writing a constrained story of two interlocking gyres 4) Hard science fiction space battle.

Opens with an observation

Ethan Mollick@emollick

Apple AI on device is equivalent to other tiny models. Apple AI Server is equivalent to GPT-3.5ish Frontier models they are not, but will still give you supercharged Siri & okay-ish writing help. I don’t think this has much to do with high-end use cases. t.co

Opens with an observation

Ethan Mollick@emollick

I don't understand the path forward for Mythos releases. Google & OpenAI will have equivalent models, and they are approaching AI cyber risk guardrails differently, so they will presumably just release their versions. How does Anthropic get out of the government approval path?

Opens with an observation

Ethan Mollick@emollick

This is a prime example of how AI chatbots are harder to use than the first seem. 4o & other models hallucinate wrong but completely plausible-seeming citations. Deep Research does citations well, but needs to be activated. None of this is documented or explained by the models.

Opens with an observation

Ethan Mollick@emollick

So, Claude 4.5 came in far above trend in the much-watched METR measure of the task duration that AI can accomplish autonomously at 4 hours 49 minutes. Interestingly, at the harder 80% success threshold, it is GPT-5.1 Codex Max that breaks the trend. In 2023, GPT-4 was a minute.

Opens with an observation

Ethan Mollick@emollick

Testing Kimi K-2 has reminded me of how insane it is that firms picking AIs are treating them as fungible based on benchmarks Kimi & Grok & Claude & every other model have strengths, quirks & weaknesses that can make a big difference in aggregate Develop your own benchmarks!

Opens with an observation

Ethan Mollick@emollick

This is an interesting overview of what it is like to actually build production software with today's LLMs. It also shows how weird LLMs are to work with from a software perspective & how much we have to learn. It is why I stress co-intelligence for now. t.co

Opens with an observation

Ethan Mollick@emollick

I think the market will adjust to any per token cost decrease brought on by DeepSeek quite quickly. Costs for GPT-4 level intelligence dropped by 1000x in the last 18 months. A 95% price drop in reasoning models seems not to be something that will break the labs.

Opens with an observation

Ethan Mollick@emollick

Oh no, we aren’t going to go back to this sort of prompting again, are we? I would love Anthropic to test if it actually works robustly, because our experiments (with slightly older models) found it did not.

Opens with a question

Ethan Mollick@emollick

There are reasons to argue about financial bubbles in the funding of data centers, but it is increasingly clear from the numbers that the frontier AI companies are showing that AI is, indeed, a very massive business with high demand and rapid growth rates.

Opens with an observation

Ethan Mollick@emollick

This is the second time that this has happened. I really wish xAI would fully embrace the transparency they mention as a core value. That would include also posting system cards for models and explaining the processes they use to stop "unauthorized modifications" going forward.

Opens with an observation

Ethan Mollick@emollick

Interesting post & agree AI has missing capabilities, but I also think this perspective (common in AI) undervalues the complexity of organizations. Many things that make firms work are implicit, unwritten & inaccessible to new employees (or AI systems). Diffusion is actually hard

Opens with an observation

Ethan Mollick@emollick

A likely incorrect lesson that the AI labs took from the Sydney/Bing debacle two years ago was that AI assistants should have any personality removed. I suspect personality not only makes AI pleasant to use (eg DeepSeek, Claude) but also makes people understand they are fallible

Opens with an observation

Ethan Mollick@emollick

The metrics discussion at OpenAI is a little confusing to me. I appreciate the clarification about bad benchmarks, but they spent a lot of money developing a very good benchmark of autonomous model ability at hard tasks, GDPval, and haven't reported it for GPT-5.6.

Opens with an observation

Ethan Mollick@emollick

Meta released their open source AI, Llama 3, today. As a key leader in LLMs, their models are often the most advanced open source ones out there. Based on benchmarks, the current model is not quite GPT-4 class, but their larger one (still training) will reach GPT-4 level.

Opens with an observation

Ethan Mollick@emollick

Why sell access to your AGI model until the second lab achieves AGI? Launch a fund and sweep up the entire market before anyone knows what is happening. Ye shall know AGI when the mysterious counterparty of every bet placed by Citadel or Apollo always comes out ahead.

Opens with an observation

Ethan Mollick@emollick

When I say "UBI is not an answer" I mean UBI is a policy decision, it is not a description of what life would be like in a world of highly advanced AI. And saying "the definition of a singularity means we can't say what comes next" is also just dodging the question of a vision.

Opens with a story

Ethan Mollick@emollick

I really don’t want to do “it isn’t open source unless it comes from the open source area of MIT” but open weights models are not open source. If they don’t share the training corpus, the weights are not transparent or reproducible (but sharing data invites scrutiny of sources)

Opens with an observation

Ethan Mollick@emollick

I assume Google escapes this trap, but this is what happened to Meta with Llama 4 and xAI post Grok 4. Only company to have escaped the "disappointing next giant model trap" without a major setback to their lead was OpenAI, with Orion/GPT-4.5.

Opens with a curiosity gap

Ethan Mollick@emollick

It turns out that Meta had 27 different models on LM Arena prior to the launch of Llama 4, but they announced it as if they had one model that topped the leaderboard. An extreme example of benchmark hacking (which other labs also do to lesser degrees). t.co

Opens with a number

Ethan Mollick@emollick

OpenAI may be one of the most important technology companies in the world today, but they really like to communicate through hints and oracular whispers. What is GPT2? At this rate we will only know that GPT-5 is being launched from an “I Love Bees”-esque Alternate Reality Game.

Opens with an observation

Ethan Mollick@emollick

At this point, I assume that every internal memo at an AI lab is just written for public release. The labs are certainly capable of keeping secrets that don’t leak, so they must realize that all-hands memos or announcements just go right to The Information or Wired or whatever.

Opens with an observation

Ethan Mollick@emollick

I’ve had access to the new GPT-4o image generator for a bit: “now i need you to perfectly illustrate the moment that Elvis met Napolean at waterloo" "make it photorealistic" "napoleon is wearing a rubber duck on his head elvis's pants have the ideal gas law printed on them"

Opens with an observation

Ethan Mollick@emollick

The UK’s state AI Security iIstitute findings: 1) Mythos is a big gain in cyber capabilities. But so is GPT-5.5 2) It is hard to establish an upper bound on Mythos/GPT-5.5, which appear to be limited by tokens used, rather than ability. 3) Capability doubling time is 4.5 months

Opens with an observation

Ethan Mollick@emollick

Yes, the AIs were given a cybersecurity challenge, with internet access enabled and safety filters disabled. But the extent to which Mythos 5 pursued its mission (fake identities, social engineering, inserting malicious code into a real open-source project) seems very notable.

Opens with an observation

Ethan Mollick@emollick

Spark is the big news and is a good model. Not quite at the frontier of open models from China, and still well behind the closed frontier, but the best non-Chinese open weights model released in a year. Of course, a lot depends on continuing to release new open models to keep up

Opens with an observation

Ethan Mollick@emollick

Been playing with the new Operator for a little bit before launch and it is both very much still an experiment and also a good indicator of where things are going. It goes on the web and does things for you. Still many rough edges but here is an example of using it for shopping.

Opens with an observation

Ethan Mollick@emollick

"Claude 3.7, do the AGI unicorn thing in the PDF but make it like 10x more impressive to really show those sparks, don't limit yourself to TikZ or even images" (I pasted in the Sparks of AGI paper). Here is what it did

Opens with an observation

Ethan Mollick@emollick

AI is jagged, but I think sometimes it is easy to overly focus on that. The generalness is a surprise too! LLMs may be optimized for verifiable fields like coding, but AI is also not bad at corporate strategy & medical advice & writing a sestina & expressing empathy & ideation.

Opens with an observation

Ethan Mollick@emollick

Either the Singularity is now a marketing term or else Stripe believes we are at a point “in the history of the race beyond which human affairs, as we know them, could not continue"? Kind of makes forward-looking financial statements useless. * Von Neumann’s original phrasing

Opens with a question

Ethan Mollick@emollick

It seems like SpaceX/Grok and Meta/Muse have started to keep pace in the near-frontier category while also introducing a new category of cheap, fast & closed specialized coding models. Both were tied with the Big Three at some point, fell behind, but may have started to return.

Opens with an observation

Ethan Mollick@emollick

Its very hard to know what is real with 🦞bot right now, lots of stuff that may be emergent behavior or storytelling or prompt injection or who knows? As impressive as these examples are, the models are still Claude 4.5 Opus, which is great but also has obvious limits to ability

Opens with a question

Ethan Mollick@emollick

I know these are all unreliable leaks of internal code names but please, please AI labs, the only thing worse than calling your models GPT-5.5-xhigh-Codex-nano is giving them names like Agent Smith or Mythos, for obvious reasons.

Opens with an observation

Ethan Mollick@emollick

Here Opus 4.8 built and play-tested a new RPG in Claude Code, including 3 PDF manuals and adventures, playtest notes, a website, and a playable solo adventure - then put it all on Netlify. No feedback from me at all. t.co

Opens with an observation

Ethan Mollick@emollick

I wonder how much Apple AI is going to cause people to underestimate LLMs A 3B quantized model running on local silicon will be good for some stuff but not agentic reasoning. The cloud model doesn’t seem to maintain state or history & isn’t frontier. Is GPT-4 just for questions?

Opens with an observation

Ethan Mollick@emollick

After publishing the post, I was contacted by Anthropic who told me that Sonnet 3.7 would not be considered a 10^26 FLOP model and cost a few tens of millions of dollars, though future models will be much bigger. I updated the post to reflect this, though it doesn't change much.

Opens with an observation

Ethan Mollick@emollick

My impressions of getting to work with a preview of the new Claude with computer use, and putting it to work across multiple tasks. You can see the promise of agents, but also where they still need work. I think it is a sign of the near future. t.co

Opens with an observation

Ethan Mollick@emollick

It is not well-explained, but with the adaptive switch off, I get no thinking. I can set thinking levels in Claude Code, but not in Claude Cowork. AI companies keep seeming to assume that coding/technical work is the only kind of important intellectual work out there (it is not)

Opens with an observation

Ethan Mollick@emollick

Had early access to Gemini 3.8 Flash, it is a very good Flash model, but not equivalent to a frontier model, though. Here is Gemini 3.8 Flash's version of the shader with the same prompts (which it provided very quickly)

Opens with an observation

Ethan Mollick@emollick

I think OpenAI made a mistake by making 3.5 available without a login m. It may make it harder for people to see what AI can do as it is so far behind GPT-4. GPT-3.5 would not even be in the top six or seven AIs I would suggest someone use to understand how powerful they can be

Opens with an observation

Ethan Mollick@emollick

"As of May 2026, more than 80% of the code we merge into Anthropic’s codebase was authored by Claude." Matches independent measures. There really is no sign this is slowing down (which doesn't mean there aren't organizational challenges to absorbing this much productivity gain)

Opens with a number

Ethan Mollick@emollick

Had early access to the new ChatGPT Image 1.5. It is good, especially at single images, but I don't think it works as well for complex slides/graphics/information as Nano Banana Pro (though GPT-5.2 Thinking seems to do better than GPT-5.2 instant, so maybe more planning helps?)

Opens with an observation

Ethan Mollick@emollick

I have no inside information, but every sign so far is that there is growing tension about open weights models between the US & China: the US announcing that they reserve the right to act against distilled models, then saying Kimi distills. Contradictory reports from China, etc.

Opens with an observation

Ethan Mollick@emollick

I am not convinced that this is the right way to think about "AI fluency," either now or in the long-term, but it is good to see work on the subjects from the AI Labs, and the general advice here is very good.

Opens with an observation

Ethan Mollick@emollick

It is interesting the degree to which OpenAI is responsible for making its competitors possible: 1) They launched the first public big LLM, showing they are powerful 2) Many models trained on GPT data👇 3) Showed off a lot of easily copied ideas (eg test-time compute is key)

Opens with an observation

Ethan Mollick@emollick

ChatGPT's new "memory" feature is neat in theory, and I am sure for many use cases, but can be a real problem if you are trying to do any persona-based roleplaying, either for fun or for idea generation, writing, etc. It has a tendency to remember anything you roleplay as true.

Opens with an observation

Ethan Mollick@emollick

A valuable thing that teachers and coaches do is provide frameworks for solving problems. LLMs are very good at frameworks: they "know" a lot of them, and they are excellent at applying them to your problem. Here's a little GPT that suggests & makes them: t.co

Opens with an observation

Ethan Mollick@emollick

Though the images are very good, ChatGPT Image 2.0 does have the typical imagegen problem, which is that editing can be "stubborn", and attempts to get the AI to change details work well for the first round or two, but then progress slows. Putting the image in a new chat helps.

Opens with an observation

Ethan Mollick@emollick

Unsurprising but still big: MTurk is on its way out. Mechanical Turk was a mainstay of social & survey research through the 2010s, as it allowed you to quickly buy access to many representative humans. It was pretty good at it, until LLMs came along and everyone started using AI

Opens with an observation

Ethan Mollick@emollick

The open weights models discussion would be much less fraught if all the frontier closed models weren't developed in the US and all the frontier open models weren't developed in China. It means that discussions over openness and AI are inevitability about a lot of other things.

Opens with an observation

Ethan Mollick@emollick

Now that GPT-5.6 is live I can share demos, so I am starting with the dumbest project I had it do during the preview: make a game called "Don't Discuss Goblins." From a prompt, it generated ideas and came up with a pretty clever mechanic and graphics: t.co

Opens with an observation

Ethan Mollick@emollick

Not Nova? Fascinating how critical the two independent frontier labs (OpenAI & Anthropic) have become to the tech giants’ advanced products, including Amazon & Apple. It suggests there is some durable core of knowledge or ability that cannot be easily replaced by compute & cash.

Opens with an observation

Ethan Mollick@emollick

One question I have about GPT 5.1 is whether "personality" biases the model outputs. There is reason to suspect that custom instructions might impact the quality and type of answers that you receive. I would love more information about whether that is still true for personality.

Opens with an observation

Ethan Mollick@emollick

Qwen 3.8 Max on my shader test. Basic impression after a bunch of experiments is that it is a solid model, but not Kimi K3 level in my experience so far.

Opens with an observation

Ethan Mollick@emollick

It basically rarely seems to think on analysis, writing, or research tasks, which means it isn't using tools or web search. Haven't tested everything yet, so not definitive, but I am often getting lower quality answers for that sort of use case that Opus 4.6 Extended Thinking.

Opens with an observation

Ethan Mollick@emollick

Microsoft seems to be launching its own branded version of Cowork (though I hesitate to discuss products I haven’t tried) A big question is whether it will continue to use lower-end models without telling you. Also whether it will keep up as the space evolves, or is it a one-off

Opens with an observation

Ethan Mollick@emollick

I had early access to Udio, the new AI music-making tool. It is impressive at generating lyrics & complex music, with interesting voices, from a prompt Here is a "catchy late 1990s ska song about the end of the world." I was amused that it included "skapocalypse" into the lyrics

Opens with an observation

Ethan Mollick@emollick

Musk's Grok AI was just released open source in a way that is more open than most other open models (it has open weights) but less than what is needed to reproduce it (there is no information on training data). Won't change much, there are stronger open source models out there.

Opens with an observation

Ethan Mollick@emollick

New paper (on an old AI) tests o1 against doctors on medical benchmarks & real ER cases: “across a variety of scenarios and applications, the large language model outperformed both human physicians and older models” The potential suggests an “urgent need for prospective trials.”

Opens with an observation

Ethan Mollick@emollick

This seems like a very obvious result if DeepSeek or other competition helps drive down cost. It was already happening before DeepSeek There are many uses for o1/r1 level intelligence that are too expensive now in terms of compute and, thus, cash. If price drops, there are uses

Opens with an observation

Ethan Mollick@emollick

ARC-AGI-3 took me a few tries, but it is definitely human winnable. I am curious how much of the very initially very low performance of frontier models is harness, vision, and tools, versus how much are limitations of LLMs. I guess we will find out! t.co

Opens with an observation

Ethan Mollick@emollick

And now from the Chinese government side. This would be a good time for cooperation between the US and China to establish common testing/acceptance standards for new models, so at least the safety certification side of model releases can be transparent for open & closed models.

Opens with an observation

Ethan Mollick@emollick

GPT-5.1-Codex was released six days ago, now we have GPT-5.1-Codex-Max. (The use of every naming scheme piled on top of each other, from version numbers to qualifiers like Max, makes it hard to see how big a deal each release is, but this looks like a big jump in ability)

Opens with an observation

Ethan Mollick@emollick

One thing I think about a lot is that models are optimized for coding and math. But they could be trained to do a lot of things that we just assume AI does badly. Like writing coherent fiction with a satisfying arc, for example.

Opens with an observation

Ethan Mollick@emollick

Even after only a dozen uses, it is clear im-a-good-gpt2-chatbot is full of ghosts. I mean this in the same way that GPT-4 and Claude 3 Opus and Gemini 1.5 are full of ghosts/sparks/whatever - they are occasionally uncanny. Seems to be an emergent feature of frontier LLMs.

Opens with an observation

Ethan Mollick@emollick

Not bad from GPT-4.1: "create something I can paste into p5js that will startle me with its cleverness in creating something that invokes the control panel of a starship in the distant future" First go, no errors.

Opens with an observation

Ethan Mollick@emollick

Gemini 2.5 now does a much better job than most AIs writing plots & characters, thanks in part to its ability to "think" through details in advance. A fun test: "play a game of Fiasco (think a Coen Brothers movie, but as an RPG), make it interesting in terms of plot and prose"

Opens with an observation

Ethan Mollick@emollick

Last year I wrote that Apple was betting on a different vision of AI, one that expected slower AI capability growth & where on-device privacy was important to consumers Those were probably bad bets, and Apple seems to have failed to deliver on them anyway t.co

Opens with a story

Ethan Mollick@emollick

For those of you not following the drama, gpt2-chatbot, which mysteriously appeared on an AI leaderboard site, then disappeared, has now reappeared, is definitely from OpenAI, and seems to be quite good, not sure how good Whether this is a preview release of GPT-4.5/5 is unknown

Opens with an observation

Ethan Mollick@emollick

Seems like a good model from Meta that is still trailing the current series of releases. The most important thing to note is that it is not open weights. That was the main reason that Meta's models were so important. Without that, it is a lot harder to predict the value of Spark

Opens with an observation

Ethan Mollick@emollick

Gemini 2.5 is a very impressive model so far. For example, it is only the third LLM (after Grok 3 and o3-mini-high) to be able to pull off "create a visually interesting shader that can run in twigl-dot-app make it like the ocean in a storm," and I think the best so far.

Opens with an observation

Ethan Mollick@emollick

The new Analysis Tool that Anthropic rolled out seems to be super powerful but also super weird in ways I don't quite get. I think the entire dataset is in the context window? On one hand it is much more insightful than Code Interpreter, but also harder to know what it is doing

Opens with an observation

Ethan Mollick@emollick

Compare GPT-5.2 from 7 months ago with the new GLM-5.2 Deep Think Max's: "create a visually interesting shader that can run in twigl with an infinite city of neo-gothic towers partially drowned in a stormy ocean with large waves" "Make it better" GLM-5.2 also had a couple errors

Opens with an observation

Ethan Mollick@emollick

Veo 3.1, same prompt, also best of 4. You can see real (but somewhat incremental) improvements here, including more of a sense of physicality & better reflections. Not perfect, though, the otter is holding the phone wrong.

Opens with an observation

Ethan Mollick@emollick

Haven’t tried this but it seems very neat… Yet all of the demos (except maybe one) are the model being fun and/or annoying by correcting or reminding in real time. There are obvious uses for this sort of model in meetings, education, training, etc. Why not demo valuable cases?

Opens with an observation

Ethan Mollick@emollick

These are good stats for a small model. Looking forward to testing. And a 10M (!!) context window suggests approaches to RAG are going to need to change dramatically.

Opens with an observation

Ethan Mollick@emollick

Also had some early access to Gemini 3.5 Flash. Very fast for a flash model and very capable, though not as powerful as a full frontier model. I added it to the gallery or procedurally generated one-shot towns (it made one error that it corrected): t.co

Opens with an observation

Ethan Mollick@emollick

The ARC benchmarking debate is obscuring the fact that test time compute seems to be the real deal based on experiments with o1 (I cover use cases & a Harvard/Stanford paper on medical uses in the post). o1 & Gemini 2 are precursors. o3 is likely better. t.co

Opens with an observation

Ethan Mollick@emollick

And then there were six or so Based on the stats, it looks like Amazon's Nova Pro is a competitive frontier model. This rounds out the GPT-4/Gen1 models: GPT-4o, Gemini 1.5, Claude 3.5, Grok 2, Llama 3.2 & maybe the three non-US models: Qwen, Yi & Mistral. Gen2 models up next?

Opens with an observation

Ethan Mollick@emollick

Last time around Apple released a lot of information about how their AI version of Siri worked between local and cloud models, not so much this time It is nice to have a Gemma-like model on device, but it is extremely limited unless it can call a smarter cloud model when needed.

Opens with an observation

Ethan Mollick@emollick

Operator does a pretty good job in building a Magic deck using an online deck-building site The serious point is that using the web can help with error checking and act as a source of truth that keeps the LLM on track. For example, the web tool keeps track of the count of cards

Opens with an observation

Ethan Mollick@emollick

Claude now has a good version of web search, which is really useful, but it also has the same issue as all non-agentic (that is, not Deep Research) AI uses of web search or RAG, which is the model only gets a small amount of context from search & then sometimes over-extrapolates.

Opens with an observation

Ethan Mollick@emollick

Gemini 2 Flash Thinking pulls off a good sestina, and then schools me on the correct form. If you didn’t know, sestinas are HARD & this was impossible for models before test time compute (o1 preview was first to crack it), and the presumably tiny Gemini model does it very fast.

Opens with an observation

Ethan Mollick@emollick

Some fun examples, I just gave basic prompts and the AI executed: Balatro, but for coin flipping (all the design and ideas was Fable): t.co The best self-aware snake game: t.co An isochronic map using real data: t.co

Opens with an observation

Ethan Mollick@emollick

First major national effort to harness AI to speed up research. I am curious about specifics on compute investment, and “domain-specific foundation models” trained on federal scientific data is a different goal than catching up to frontier general-purpose AI, but interesting.

Opens with an observation

Ethan Mollick@emollick

Got to play with a little of this before launch as well. My experience as a social scientist was that it was more bioscience focused right now, but I think Google has been the leading lab in releasing serious AI tools to accelerate science & expect to see them improve fast.

Opens with an observation

Ethan Mollick@emollick

I had access to the new Google Lyria 3 Pro music AI. Its quite good. I've been ruining(?) Rilke by giving the AI the First Elegy & asking it to make it "more 1990s boy band" ("oooo the beginning of terror, girl") Catchy! It is also nuts that you can ask an AI to do this & it can

Opens with an observation

Ethan Mollick@emollick

So this would be in the top 20 American companies by revenue, above GM and Bank of America this year. Weirdly, it also means they haven’t hit their goal & AGI has not been achieved/diffused, because it is a lot less revenue than you would get from automating all knowledge work.

Opens with a number

Ethan Mollick@emollick

New laws of robotics just dropped. From OpenAI’s Model Spec 1) follow the chain of command: Platform > Developer > User > Tool 2) Comply with applicable laws 3) Don't provide info hazards 4) Protect people's privacy 5) Don't respond with NSFW contents t.co

Opens with an observation