Give Your Agent Exemplars

In the long run, the most successful people will find ways to use AI to improve their own understanding and work with the mindset to do things better than before. Studies are showing that using AI for shortcuts, such using it to do homework, is leading to the opposite outcome where students do worse on exams: see this post from Ethan Mollick. The analogue for work is if you delegate everything without ever taking a pause to learn, you don't get better and continue to make mistakes, create bugs, and walls where progress becomes painful. This is a quick blog to share one pattern I have found effective to both help me learn and do things better. That pattern is to find exemplars for whatever I am doing and put those exemplars directly into my project directory for agents to reference. After setting those up, whenever I need examples, ideas for how to do something, suggestions on what is typically done, best practices, etc. the agent can use and cite those exemplars directly.

Whenever I have a project where I know there should be good, related learning materials or exemplars out there, I ask an agent to give me suggestions for what is out there in the context of that project. I review the results myself to make sure they are good and actually related. Most times that ends up just being a filtering down to the best ones - the systems today are pretty good at this type of research. With this list in hand, for coding projects I git clone each one into a reference/ folder (shallow clones with just the main branch to save space). I also tend to directly include the documentation for whatever libraries I am working with. If it's a dependency, particularly a newer or lesser known one, I will also add an instruction to my AGENTS.md file like:

Locally available references in the `reference` folder:
- `docker-docs` - <optional description of when to use it>
- ...

In my interop-router library, I even have a skill that keeps these references for dependencies up to date. A large part of the value doesn't only come from referencing dependencies, but from the exemplar repos or documents. Whenever I am unsure of what the best way to do something is, I can ask questions like "propose to me options for how to do this, with examples (including paths I can click on) from the references." and then dig in further myself or ask further questions.

Recently I started working on a new project that is an open source "product" and as I am setting up the project, I have a ton of questions and ambiguities for what the right way to do things is, from where to put development instructions, how to make it easy to self-host, architectural questions, and the list goes on. To start with a simple example, the project has a Python backend with different concerns (API, jobs, etc.) that need to share some code (like database access) but not other things. So this led me to look for the standard way to handle this in uv. When I went to the uv docs, there were a few options: extras, groups, and workspaces. When I first asked for recommendations based on my requirements, it suggested extras. Instead of going with it, I asked "Can you tell me the tradeoffs between these and what projects typically do?" which led it to go and look through the docs and exemplar projects. Through that it found that most uv projects actually use groups.

While exploring this topic in the exemplars with the agent, I also learned about add-bounds = "major". This is a uv feature that handles something that I have been doing manually or giving agents instructions for when configuring a project: setting the maximum version of each dependency. The agent suggested it because it found that PostHog does this in their pyproject.toml. Now simply a command like uv add "procrastinate" will set the bound automatically, resolving to what I was doing manually in pyproject.toml: "procrastinate>=3.9.0,<4.0.0",. As a thought exercise, the two other ways I could have gone about this were not using AI at all or letting it do everything. In this case I was already leaning to go with groups, but wanted some confirmation - so no change in what I would have done overall, but I got out the extra dependency trick from having the exemplars in context + the agent know the current state of what I was doing! In the other case, letting the agent do everything would have led to the worst outcome where I would have had the objectively less ergonomic choice, I would not have learned anything, or had the opportunity to improve my workflow.

For the same project, a major part of it is making it easy to develop and self-host using Docker Compose. I have set it up before, but for this project my goal is to build it to the absolute highest standard so I wanted to learn what other projects usually do. From things like the postgres image to optimizations and security configs, the agent went through the other projects and gave me a rundown of references that used postgres, and what the most common patterns were. For anything I was unsure of how it applied to what I am working on, I could ask it to directly go to either the postgres or Docker docs to find out and I got the references to read more. Having both the exemplars and the docs led to learning about the considerations that are necessary for a production-grade project, while still having them available to figure out what the latest version might have changed. Again, this to me is the best approach - I can set up something better than I was able to before and take away my own learnings from other successful projects.

One more example from my work at Microsoft, I wanted to create a skill for amplifier-agent to help others use it more effectively. For this, I first did research on skills for other major projects and it found things like Anthropic's examples, GitHub's copilot-sdk, and others. I then did a /goal session in Amplifier (goal iterates until a separate evaluator deems the task to be complete) to have it take a shot at building a skill for amplifier-agent where the goal was to match the scope and quality of the exemplars. The result was a skill that needed few modifications to be effective out of the box, was relatively compact, and delegated to other docs/files/tools - like the exemplar skills.

At least once per day, instead of asking an agent to do something for you, work with it to learn the best way to do it. Over time this will make you better and you will find yourself building your own exemplars to lean on.