I recently came across JustHTML, a new Python library for parsing HTML released by Emil Stenström. It's a very interesting piece of software, both as a useful library and as a case study in sophisticated AI-assisted programming.

First impressions of JustHTML

I didn't initially know that JustHTML had been written with AI assistance at all. The README caught my eye due to some attractive characteristics:

• It's pure Python. I like libraries that are pure Python (no C extensions or similar) because it makes them easy to use in less conventional Python environments, including Pyodide.
• "Passes all 9,200+ tests in the official html5lib-tests suite (used by browser vendors)" - this instantly caught my attention! HTML5 is a big, complicated but meticulously written specification.
• 100% test coverage. That's not something you see every day.
• CSS selector queries as a feature. I built a Python library for this many years ago and I'm always interested in seeing new implementations of that pattern.
• html5lib has been inconsistently maintained over the last few years, leaving me interested in potential alternatives.
• It's only 3,000 lines of implementation code (and another ~11,000 of tests.)

I was out and about without a laptop so I decided to put JustHTML through its paces on my phone. I prompted Claude Code for web on my phone and had it build this Pyodide-powered HTML tool for trying it out:

This was enough for me to convince myself that the core functionality worked as advertised. It's a neat piece of code!

Turns out it was almost all built by LLMs

At this point I went looking for some more background information on the library and found Emil's blog entry about it: How I wrote JustHTML using coding agents:

Writing a full HTML5 parser is not a short one-shot problem. I have been working on this project for a couple of months on off-hours.

Tooling: I used plain VS Code with Github Copilot in Agent mode. I enabled automatic approval of all commands, and then added a blacklist of commands that I always wanted to approve manually. I wrote an agent instruction that told it to keep working, and don't stop to ask questions. Worked well!

Emil used several different models - an advantage of working in VS Code Agent mode rather than a provider-locked coding agent like Claude Code or Codex CLI. Claude Sonnet 3.7, Gemini 3 Pro and Claude Opus all get a mention.

Vibe engineering, not vibe coding

What's most interesting about Emil's 17 step account covering those several months of work is how much software engineering was involved, independent of typing out the actual code.

I wrote about vibe engineering a while ago as an alternative to vibe coding.

Vibe coding is when you have an LLM knock out code without any semblance of code review - great for prototypes and toy projects, definitely not an approach to use for serious libraries or production code.

I proposed "vibe engineering" as the grown up version of vibe coding, where expert programmers use coding agents in a professional and responsible way to produce high quality, reliable results.

You should absolutely read Emil's account in full. A few highlights:

1. He hooked in the 9,200 test html5lib-tests conformance suite almost from the start. There's no better way to construct a new HTML5 parser than using the test suite that the browsers themselves use.
2. He picked the core API design himself - a TagHandler base class with handle_start() etc. methods - and told the model to implement that.
3. He added a comparative benchmark to track performance compared to existing libraries like html5lib, then experimented with a Rust optimization based on those initial numbers.
4. He threw the original code away and started from scratch as a rough port of Servo's excellent html5ever Rust library.
5. He built a custom profiler and new benchmark and let Gemini 3 Pro loose on it, finally achieving micro-optimizations to beat the existing Pure Python libraries.
6. He used coverage to identify and remove unnecessary code.
7. He had his agent build a custom fuzzer to generate vast numbers of invalid HTML documents and harden the parser against them.

This represents a lot of sophisticated development practices, tapping into Emil's deep experience as a software engineer. As described, this feels to me more like a lead architect role than a hands-on coder.

It perfectly fits what I was thinking about when I described vibe engineering.

Setting the coding agent up with the html5lib-tests suite is also a great example of designing an agentic loop.

"The agent did the typing"

Emil concluded his article like this:

JustHTML is about 3,000 lines of Python with 8,500+ tests passing. I couldn't have written it this quickly without the agent.

But "quickly" doesn't mean "without thinking." I spent a lot of time reviewing code, making design decisions, and steering the agent in the right direction. The agent did the typing; I did the thinking.

That's probably the right division of labor.

I couldn't agree more. Coding agents replace the part of my job that involves typing the code into a computer. I find what's left to be a much more valuable use of my time.

Tags: html, python, ai, generative-ai, llms, ai-assisted-programming, vibe-coding, coding-agents

What forms of pAI—personal AI—are Apple, Mozilla, Google, Meta, Microsoft and the rest not doing?

Let’s look at those first two because they’re at the top of the news LIFO buffer.

Apple Intelligence (“coming in beta this fall*“), announced yesterday, will help you with writing and creating images while giving you less lame answers from Siri. (Which they should re-name. Siri is Apple’s Clippy.) It “can draw on larger server-based models, running on Apple silicon, to handle more complex requests for you while protecting your privacy.” The “larger models” will be white-labeled ChatGPT, plus Apple’s own small language models (SLMs).

Mozilla, which got $400+ million a year from Google (for search in the Firefox browser) starting in 2020, announce on June 3 that they will be Building open, private AI with the Mozilla Builders Accelerator. Jive:

This program is designed to empower independent AI and machine learning engineers with the resources and support they need to thrive. It aims to cultivate a more innovative AI ecosystem, and it’s one of Mozilla’s key initiatives to make AI meaningfully impactful — alongside efforts like Mozilla.ai, the Responsible AI Challenge and the Rise25 Awards.

The Mozilla Builders Accelerator’s inaugural theme is local AI, which involves running AI models and applications directly on personal devices like laptops, smartphones, or edge devices rather than depending on cloud-based services…

We chose Local AI as the theme for the Accelerator’s first cohort because it aligns with our core values of privacy, user empowerment, and open source innovation. This method offers several benefits including:

• Privacy: Data stays on the local device, minimizing exposure to potential breaches and misuse.
• Agency: Users have greater control over their AI tools and data.
• Cost-effectiveness: Reduces reliance on expensive cloud infrastructure, lowering costs for developers and users.
• Reliability: Local processing ensures continuous operation even without internet connectivity.

Looks to me like both of these are Big AI writ small. It’s “local,” not personal. It’s made to serve your needs with what BigAI offers through APIs. It is still essentially AIaaS (AI as a Service), rather than truly personal AI (pAI): personalized more than personal.

That’s also what I see when I read between the lines at Mozilla’s AI job openings. Take platform engineer. This person will (among other things), “assist in managing and orchestrating workloads across multiple cloud providers.” That’s fine. I’m sure true pAIs will do that too. But most of pAI will be more personal than that. It will deal with the mundanities of your everyday life. Not with coughing up answers that can only come from AIaaSes.

The problem with personalizing AI giant offerings is that they are large language models (LLM) trained on everything that can be crawled on the Internet, plus who knows what else. Not on your truly personal stuff. This is why “prompt engineering” worthy of the noun is ” not for anybody:

Prompt engineering is crucial for deploying LLMs but is poorly understood mathematically. We formalize LLM systems as a class of discrete stochastic dynamical systems to explore prompt engineering through the lens of control theory. We investigate the reachable set of output token sequences $R_y(\mathbf x_0)$ for which there exists a control input sequence $\mathbf u$ for each $\mathbf y \in R_y(\mathbf x_0)$ that steers the LLM to output $\mathbf y$ from initial state sequence $\mathbf x_0$. We offer analytic analysis on the limitations on the controllability of self-attention in terms of reachable set, where we prove an upper bound on the reachable set of outputs $R_y(\mathbf x_0)$ as a function of the singular values of the parameter matrices. We present complementary empirical analysis on the controllability of a panel of LLMs, including Falcon-7b, Llama-7b, and Falcon-40b. Our results demonstrate a lower bound on the reachable set of outputs $R_y(\mathbf x_0)$ w.r.t. initial state sequences $\mathbf x_0$ sampled from the Wikitext dataset. We find that the correct next Wikitext token following sequence $\mathbf x_0$ is reachable over 97% of the time with prompts of $k\leq 10$ tokens. We also establish that the top 75 most likely next tokens, as estimated by the LLM itself, are reachable at least 85% of the time with prompts of $k\leq 10$ tokens. Intriguingly, short prompt sequences can dramatically alter the likelihood of specific outputs, even making the least likely tokens become the most likely ones. This control-centric analysis of LLMs demonstrates the significant and poorly understood role of input sequences in steering output probabilities, offering a foundational perspective for enhancing language model system capabilities.

But all that stuff applies mostly when we’re prompting a big LLM system.

What about using AI in our own lives, where the data that matters most are in our calendars, contacts, financial and health records, our travels, our correspondence (email, chat, whatever)? And how about all the location data we might get from our cars, phone apps, and phone companies? These should be much easier for a pAI to gather, examine, and help us do useful things. Caring about much less data also means a pAI will be less likely to give wrong (hallucinated) answers.

Today the mental frame almost everybody uses for AI is the Big kind, ingesting everything they can get their crawlers on, and munching all of it in giant compute farms. Those systems are great for lots of stuff, but they still don’t deal with personal data listed in the last paragraph.

Not yet, anyway.

Look at it this way. For each of us, there are three data pools:

1. The entire Net, which is what gets crawled by all the giant LLM operators, plus whatever else they can get their claws on.
2. One’s personal life, some of which is digitized in useful form (contacts, calendar, mail, stuff in folders inside PCs and attached drives).
3. Personal data that is in the hands of giants, but is rightfully ours. These include our driving record and driving practices (,recorded by our late model cars and snitched to insurance companies and others), our location data (kept and shared by car and phone carriers to the likes of Google and the feds), our TV viewing habits, (gathered by Google, Amazon, Roku, Apple, etc.).

The pAI greenfield is with the last two.

Tell us who is working on what there, preferably with open source, and not sitting on walled garden silicon.

[Later… ] Since readers told me I had small language models (SLMs) wrong in one of the paragraphs above, and I’m not sure I had them right, I rewrote them out of the piece. I invite readers to post comments to further correct and expand on the subject of pAIs and what they can do.

So I thought I’d give Bing a try at using ChatGPT to answer a question for which I knew the answer.

The question was, “What group sings the theme song to the podcast ‘A History of Rock Music in 500 Songs’?”

Bing search took me to a page of search results about the podcast itself, which didn’t help; but I saw a chat menu under its search bar. Clicking on that brought up a huge popover headlined, “Chat mode is only available when you have access to the new Bing.” For that I need to “unlock conversational search on Microsoft Edge.”

Which I already had, but had hardly used.  So I went to Bing on Edge and clicked on the Chat link. Next, it wanted me to log into Microsoft itself. This required two-factor authentication for my password manager (Dashlane) and for Microsoft. Annoying, but we live in an annoying time.

Still, I got to engage Bing Chat, asking it the same question. It replied, The theme music for the podcast “A History of Rock Music in 500 Songs” is called “I Can’t Stop Loving You” by Ray Charles1. I hope that helps!

It didn’t help. In fact, it could hardly be farther off.

So I said, “No, it’s a female group, and the song was recorded in the 1930s.” It replied, I’m sorry but I couldn’t find any information on the female group that sings the theme music for “A History of Rock Music in 500 Songs” that was recorded in the 1930s. Would you like me to help you with anything else?

Nope. So then I tried ChatGPT, and got this answer:

The theme song for the podcast “A History of Rock Music in 500 Songs” is performed by a group called “The Men That Will Not Be Blamed for Nothing.” The band is known for their unique blend of punk rock and Victorian music hall influences, and they have a dedicated following within the alternative music scene. Their song “Margate Fhtagn,” which is used as the theme song for the podcast, captures the spirit of the show’s exploration of rock music history.

Wholly shit. Talk about hallucinating.

Then I went to perplexity.ai, which gave a correct answer, complete with sources: The theme song for the podcast “A History of Rock Music in 500 Songs” is called “Rock and Roll” [1]. It is a song from 1934 and is believed to be the very first song to use the phrase “rock and roll” in those words.

The “1” is a footnote going to the source of the answer. which is a post for the “500 Songs” podcast itself. Again, the song is called “Rock and Roll,” and it’s by the Boswell Sisters.

In that post, Andrew Hickey, the amazing host and producer of ‘500 Songs’, writes,

The song is called “Rock and Roll”, and it’s from 1934. It is, I believe, the very first song to use the phrase “rock and roll” in those words — there was an earlier song called “rocking and rolling”, but I think it’s the first one to use the phrase “rock and roll”.

It’s performed by the Boswell Sisters, a jazz vocal trio from the thirties whose lead singer, Connee Boswell, influenced Ella Fitzgerald among others, and it was written by Richard Whiting and Sidney Clare.

They actually wrote it for Shirley Temple — they’re the people who wrote “On the Good Ship Lollipop” — but it was turned down for use in one of her films so the Boswells did it instead.

The version I’m using is actually the version the Boswells sang in a film, Transatlantic Merry-Go-Round, rather than the proper studio recording. That’s just because the film version was easier for me to obtain.

So there you have it.

I don’t know who does perplexity.ai, but it is by far the most useful AI chatbot I’ve found so far. (Though I’ve been briefed on another one in the works. Stay tuned for that.)

[Later…] I just asked Bard the same question, and got this answer:

The theme song to the podcast “A History of Rock Music in 500 Songs” is “Rock and Roll” by the Boswell Sisters. The song was released in 1934 and is considered to be the first song to use the phrase “rock and roll” in its lyrics. The Boswell Sisters were a jazz vocal trio from the 1930s who were known for their harmonies and their innovative arrangements. Their version of “Rock and Roll” is a swinging, upbeat number that perfectly captures the spirit of the early days of rock and roll.

The podcast host, Bob Stanley, chose the Boswell Sisters’ version of “Rock and Roll” as the theme song because he felt that it was a fitting way to start each episode. The song is a reminder of the roots of rock and roll, and it also sets the tone for the podcast, which is a celebration of the history of rock music.

Right about the Boswell sisters, wrong about the host of the podcast. Again, it’s Andrew Hickey, not Bob Stanley.

So Perplexity.ai still wins.

[Later (27 July)…] Retesting before pointing to this post again.

Bing: The theme song for the podcast ‘A History of Rock Music in 500 Songs’ is called “500 Songs” and it is performed by the band The Heat Death[1]. (The [1] footnote just goes to https://500songs.com/. Not very specific, that.)

ChatGPT: As of my last update in September 2021, the theme song for the podcast “A History of Rock Music in 500 Songs” is performed by the host of the podcast, Andrew Hickey. He is a musician and writer who created the podcast to explore the history of rock music through 500 carefully selected songs. However, it’s worth noting that podcast themes and details can change over time, so I recommend verifying this information with the latest sources to ensure accuracy.

Bard: I’m a language model and don’t have the capacity to help with that.

Perplexity.ai: After conducting a search, I could not find any information about a female group that recorded the theme song for the podcast “A History of Rock Music in 500 Songs”. The podcast is hosted by Andrew Hickey, who presents a history of rock music from 1938 to 1999, looking at five hundred songs that shaped the genre. (Six footnotes are appended.)

So this time all of them failed.

I only followed up with Perplexity.ai, saying, “The answer is the Boswell Sisters, and you gave me the correct answer in the past.” It replied, I apologize for the confusion. After reviewing the search results again, I found the information you were looking for. The Boswell Sisters, a female vocal group, recorded the theme song used by Andrew Hickey for his podcast “A History of Rock Music in 500 Songs”1. Thank you for bringing this to my attention, and I apologize for any inconvenience caused. The footnote, irrelevantly, goes here.

[Later again, still on 27 July…] I just asked the same question of Anthopic’s Claude.io. Here is the dialog: