What Were the Results of Vercel's Experiment?
Vercel pitted two ways of giving an AI up-to-date knowledge against the same test. One was a Skill, a bundle of expert manuals you pull out when needed; the other was AGENTS.md, a single text file always open in the project folder. The questions were made deliberately hard, using only new features that appeared after the model's training cutoff.
The scores say it all. Bare, with no help, scored 53. Given a Skill and left to use it freely, still 53. Told firmly to use it, 79. And AGENTS.md scored 100, full marks on build, checks, and tests alike. The sophisticated method lost to the plain one.
Why Did the Skill Lose?
First, the AI did not open the manual. Even knowing it existed, it skipped past it about six times out of ten. It knows the manual is there, yet works from old memorized knowledge, so it errs.
Second, the Skill was far too sensitive. Merely changing word order, 'read the manual first' versus 'read the project first, then the manual', swung the score. That means if you do not instruct it very precisely, it can go off the rails at any moment.
Why Did AGENTS.md Win?
The reason is surprisingly simple: it never made the AI choose. A Skill forces a decision at every moment about whether to pull it out now, and that point of deliberation is exactly where mistakes happen. With AGENTS.md already in front of it, there is nothing to decide about whether, when, or in what order to look.
Make it choose and it errs; do not make it choose and it does not. We believe that bolting on a smarter tool will help, but the smarter the tool, the more the AI stumbles while weighing whether to use it. It is better to keep the information laid out in plain sight at all times.
So When Do You Use a Skill?
The Skill is not dead. For big jobs you invoke deliberately at a chosen moment, like migrating a whole project to a new version or reworking the entire structure, a Skill is exactly right.
In short, it is not one or the other. Keep the everyday baseline knowledge always open in AGENTS.md, and call up a Skill for the special, heavy tasks as they come. That division of labor is the answer.
How Do You Apply This in Practice?
Create a single AGENTS.md at the top folder of the project, and instead of writing at length, jot down only the essentials, like a table of contents. Vercel actually compressed the document from 40k to 8k, an 80% cut, and the score stayed at 100. Keeping only the outline rather than the whole was cleaner.
Then put this one line at the top of the document: judge from this document first, not from memorized knowledge. AI keeps trying to work from old memory, and this single sentence blocks that. This one line is what separated 100 from 53.