2025-08-15 06:00:13
As luck would have it, I watched two videos today about how exceptionally widely listened to people got scientific facts very wrong. Here's Hank Green explaining a lot of good info about how vaccines work and how RFK's recent statements about a "universal vaccine" are adjacent to how things work, but actually misrepresent how things work.
And then there's a new Climate Town video about a chart that Joe Rogan has brought up a bunch on his mildly popular podcast that "proves" that climate change isn't a big deal.
In addition to being the internet's dad, Hank is also a long-time science educator and Rollie is a climate science graduate from Columbia. Rollie's video especially does a good job reviewing the science behind this chart and explains why it's not showing what Joe Rogan and Mel Gibson (yes, that Mel Gibson) thinks it shows. In fairness, half of his argument comes from reading the article underneath the chart, which it's unclear that Rogan actually did because it makes no sense how he gets to the conclusion he does if he actually read it.
Anyway, just my regular reminder to be careful where you get your science info.
2025-08-15 05:30:29
Theo is an interesting guy who's found himself as a bit of an AI YouTuber over the last year (used to talk about JavaScript stuff), and he was invited by OpenAI to test GPT-5 ahead of its announcement last week. Long story short:
The video above is not one of his best videos, but it's basically an apology video that isn't really an apology because the GTP-5 that's out now is not the same as the GPT-5 he got to use in early testing. I call it not one of his best because it seems his takeaway is that being genuine is not worth it because he got so much shit for this – that is not the lesson!
One lesson here is that one should never never never never make extremely confident statements about the quality of a new LLM in the first few days of using it. We all know the cycle at this point where whatever the latest new flagship model gets hyped up for a few days as "THE NEW CLEAR LEADER" and then after that people find the faults and limitations, and things settle down a bit. This happened with Grok 4 a few weeks ago and now Grok is generally considered 💩 (unless you're Elon-pilled). It happens with every Gemini release. Now it's happening with GPT-5. You can talk about these models right away and you can be excited about them, but be mindful that your thoughts will likely change.
Another lesson is that despite him saying it didn't have an impact, it really seems like being there with OpenAI and being involved in the launch had an impact on his impressions. I'm sure he genuinely thinks it didn't but if you liked the model and you got to be part of early access and you got to be part of their marketing material, it's going to amplify those feelings. It happens when people go to WWDC and Google I/O as well, and while I think professional reporters have a better knack for setting the excitement aside, influencers just don't do a great job of this. I'll include myself in that group, and I don't even go to big events like this. I was 100% more excited during my Comfort Zone podcast recorded with Chris and Federico in a special room at the conference than I would have been if we were all just on a normal Riverside call like we do every week.
So absolutely be genuine, this is exactly what people want from the people they follow. Get excited about what gets you excited! Be annoyed when something isn't as good as you wish it was! But be careful saying anything definitive about a product after using it for one day, and be 10x as careful when saying that about an LLM.
For what it's worth, I think GPT-5 is pretty good and I appreciate the less personal tone that it has. It's better at coding that GPT-4o in my experience, and an do quite well using tools in apps like Cursor, but Claude 4 Sonnet remains my go-to model for all things code.
2025-08-14 08:57:06
This study published yesterday by The Lancet is getting linked to all over the place with one takeaway: AI makes you dumb because as The Verge's headline says, "Some doctors got worse at detecting cancer after relying on AI". The response is clear, right? Using ChatGPT and other LLMs makes you dumb.
Well, one thing stood out to me in the summary:
Between Sept 8, 2021, and March 9, 2022, 1443 patients underwent
2021? ChatGPT hit the scene in late 2022, so it couldn't be that. What "AI" are they talking about?
Well, I had to know, so I dropped $35 to get access to the study itself with all the details, and of course, it's more complicated than it appears. I'm not permitted to redistribute the study itself, so I'll just quote some of the relevant things here.
First up, what "AI" are they talking about?
Colonoscopies were done with high-definition end oscopy processors (EVIS X1 CV 1500, Olympus Medical Systems, Tokyo, Japan) with compatible endoscopes (CF-H190L, CF-HQ190I, CF-HQ1100DI, PCF-H190I; Olympus Medical Systems, Tokyo). The AI system used was ENDO-AID CADe (OIP-1, Olympus Medical Systems, Tokyo).
This is the ENDO-AID CADe and while it does say it's powered by AI, it's not using LLMs, it's using something more like what we used to call "machine learning" and is tech much more like the tech in Apple/Google Photos that detects faces and objects than generative AI. If you take one thing away from this post, make it that this study was not about LLMs. Still, there can be some good learnings in here, so let's keep going.
In the observed period between Sept 8, 2021, and March 9, 2022, after excluding the 100 colonoscopies conducted after the introduction of AI and those done in participants who met prespecified exclusion criteria, a total of 2177 colonoscopies were conducted, including 1443 without AI use and 734 with AI. Our analysis focuses on the 1443 patients who underwent standard, non-AI assisted colonoscopies before (n=795) and after (n=648) AI introduction.
Basically, there were three similarly sized sets of data:
Here's the big finding:
ADR at standard, non-AI assisted colonoscopies decreased significantly from 28·4% (226 of 795) before AI exposure to 22·4% (145 of 648) after AI exposure, corresponding to an absolute difference of –6·0%
A higher ADR (adenoma detection rate) is better, and there's a clear drop in results in the group after AI was introduced into their workflow. But what about those other 734 colonoscopies that did use AI? They weren't a primary focus of this study, so that's relegated to their supplementary analysis near the end.
In the supplementary analysis of patients who did have AI-assisted colonoscopy, the ADR was 25·3% (186 of 734). The multivariable logistic regression analysis in all 2177 colonoscopies (including those that used AI) showed that AI exposure in those without AI-assisted colonoscopy (as shown in the main analysis), female sex of the patient, and patient age younger than 60 years were independent factors that were significantly associated with ADR, when adjusted for endoscopist as a random effect. Compared with colonoscopies before the introduction of AI, use of AI was not significantly associated with a change in ADR (OR 0·80 \[95% CI 0·63–1·02\]); appendix p 6).
Their finding is that there was a slight drop in performance when using the AI tool, although they caveat it as being notable but not technically enough to be statistically definitive.
Finally, here's the line all the AI haters are looking for:
Another challenge is to understand why the detrimental effect occurred. We assume that continuous exposure to decision support systems such as AI might lead to the natural human tendency to over-rely on their recommendations, leading to clinicians becoming less motivated, less focused, and less responsible when making cognitive decisions without AI assistance.
First off, let me shout it again from the rooftops, this study was not about LLMs. Got it? Good.
That said, it is about using technology that tries to assist humans in doing challenging tasks and make it easier for them. This is but one study, and the replication crisis demands we look at the findings of single studies as data, not absolute proofs of the nature of the universe. Still, it's interesting data that very plainly shows in this case:
My takeaway is that I'd have buyer's remorse if I spent a lot of money on that ENDO-AID CADe thing, but my broader takeaway is similar to what I think about AI tools in general: they don't make you smarter, but they can make you faster.
Let's look at a different hypothetical. What's 137 x 46? Can you do it in your head? Do you think having a calculator with you at all times every moment of your life makes you worse at doing mental math since you don't have to do it unless you really want to? I know I'm not as good as I was in middle school when I had to do this without a calculator.
Or how about my camera-loving friends out there. Do you think you're better or worse at film photography today because you have a totally digital workflow and you probably have for 20 years? How about this: are your digital photos better than your film ones? Maybe? Yes? Tomato/tomato?
How about reading a physical map?
The list goes on, and I think it's typical of people to get less good a things once they have technology that helps them do it. I would be better at mental math if I needed to do it all the time, but I have a computer with me that can do it, so it's a muscle I can relax. Strip away the mysticism and ire around LLMs that we live in right now and we should recognize they're just another tool. Tools can be used for good or bad, they can be helpful or not helpful, and they can be useful in some situations but not others. I think it's fair to say LLMs are a significant tool, and therefore all of these things are heightened, but it's still a tool at the end of the day. If it helps us be better, even marginally, at something, then we're probably going to get less proficient in doing that thing without help from the tool.
I think the important thing is for us to use the tool to let us relax the muscles that we can afford to relax and not the ones we shouldn't. This is not an easy thing to answer, in fact I'd say the answer is as challenging to balance as it's ever been (members post with a similar thought). In the case of this ENDO-AID CADe, I'd say that if this data is widely applicable, it's not a good device and hospitals should spend their money on something else. Oh, and don't take this study that doesn't even talk about LLMs be your proof that LLMs are bad for people (it makes it look like you used AI to summarize it 😉).
2025-08-13 08:35:02
Joe Rossignol on MacRumors: Elon Musk Threatens to Sue Apple Over App Store Rankings
"Apple is behaving in a manner that makes it impossible for any AI company besides OpenAI to reach #1 in the App Store, which is an unequivocal antitrust violation. xAI will take immediate legal action," Musk said in a post on X (Twitter), which he also owns.
Musk failed to provide evidence to support his claim.
The billionaire’s claim of our time.
2025-08-13 08:00:38
I was listening to this week’s Upgrade and the topic of Tim Cook’s gifting of a trophy to Donald Trump came up. The short version is that both Myke and Jason believe that Cook didn’t personally enjoy this, they both think he “debased” himself by doing it, and that he really didn’t have a choice because the real problem here is that the US government is corrupt and this is the sort of thing they make companies do.
I appreciate this position, and I 100% agree that the primary problem here is the Trump administration itself. It is absurd that the supposed only way for Tim Cook to run Apple right now is “debasing” himself and kissing the emperor’s ring. This is how things work in Russia, they are not how it’s supposed to happen here.
However, I don’t like the presumption that Tim Cook and Apple are just passengers here in our descent into fascism. I also don’t like the suggestion some make (not exactly Myke and Jason, to be clear) that Apple is a soulless corporation who doesn’t care about the things that they believe are important or are willing to sacrifice profit margins in support of those goals in some cases. Obviously some of that is useful marketing as it makes the company look better, but these aren’t evil robots running the company, they’re real people who I believe on the whole want to make the world better.
If Apple committed to doing more American manufacturing so as to avoid the tariffs, I think that would have been fine. Doing so in a joint press conference with Trump would get a sad head shake from me, but not ire. What bothers me is celebrating Trump by publicly presenting him with a trophy that signals to the world that you really, really support this guy. I appreciate that Cook isn’t someone who spends much time sharing his opinions on political and social issues in public, but let’s be clear, whenever he has shared an opinion this year, it’s 100% been in support of whatever Trump wants. In fact, it’s been more public adoration than basically anyone else in the massive tech companies besides Elon Musk (and even Musk has expressed disagreements with Trump) have given him.
To Cook’s credit, Apple continues its diversity initiatives and climate goals, both of which operate directly in opposition to what Trump wants for the US. But let's say Trump notices those and farts out a post on Truth Social saying Apple must stop these programs; my money's on them being gone in a week. We can't upset the mad king.
Maybe Cook actually does hate all of this, and he's willing to take the reputational hit to protect his company and the people who work there. I'm not talking shareholders, I'm talking the employees working there who might have to get laid off if the government targeted Apple and financially hurt them. Maybe in his mind he's just doing what needs to be done, even if he personally greatly dislikes Trump. I actually think there's a decent chance this could be the case. What I'll say, though, is that if you prop up and celebrate and "thank you sir, can I have another?" with the bad guys long enough, you're not a good guy "playing the game" to get by, you're just on the bad guys' team now.
Mr. Cook, I've respected you for many years and I adore the products your company makes. Being the CEO of Apple must be one of the hardest, most complicated jobs in the world, it's not something I could ever do myself, and I appreciate that in 2025 you are put in impossible positions on many fronts. 9 years ago one of your VPs said it took "courage" to push phone technology forward by removing the headphone jack from iPhones. We are in trying times as a country and the stakes are higher than they were for that headphone jack; if you have any more of that courage stored up, we'd love to see it.
You don't need to smack him down or anything, but something, anything to indicate that you're not actually 100% in the bag for this guy would be good to see. Until then, it just looks like you're legitimizing and celebrating a man who's stripping people's rights and actively using the military to take over cities managed by his political opponents.
2025-08-10 04:32:12