Add agent instructions for Jamulus#3785
Conversation
|
|
I agree. I'm happy for AI to show me how to do something, but then I re-implement the technique myself. I wouldn't want to include code that I don't understand how or why it works! |
|
Yes sure. But I still think that if the code is understood and well written it's fine. But yes, we need to update CONTRIBUTING.md |
|
My experience has been that an agent should author this kind of guidance, and the agent should be asked to evolve the guidance when an agent makes an error and then recovers from the error with or without human intervention ("a teachable moment"). I say something like, "You made an error while fixing/making this, and then we recovered from the error. Let's add guidance to AGENTS.md that would help prevent this kind of error next time." I asked Fable to review this PR against my own CLAUDE.md (basically same function as AGENTS.md). This is primarily because a model like Fable can produce great guidance that a lesser model like Sonnet can follow. Its overall view is: good foundation, accurate throughout, worth merging after additions. Here are 5 observations it provided:
My view on this: This is some pretty technical, fundamental stuff, and I think I know why it mentions this. I had added a mechanism to my servers to keep out known attack vectors (ASNs and a few IP ranges), but my lookup was synchronous, and every 20 seconds, the whole server would freeze for 100ms, for everyone. People wouldn't stay. :) If it was easy-breezy for me to introduce that problem, perhaps it's basic guidance for any agent to avoid making such a blunder.
My view: Maybe this guidance gets too far into the weeds in some ways, or maybe not. When your project has non-negotiable architectural laws, it's good to include them in tose terms. I started with a "CONSTITUTION" in the earliest comments of a script file, with numbered LAWS. The location (first comments, or AGENTS.md) assures these always get read, and fundamentally unwise changes just don't happen (because they'd break the LAW).
My view: I agree. Spell out what it should do.
My view: Perhaps so. And notice that AGENTS.md can and probably should link through to CONTRIBUTING.md and any other pre-existing guidance. Agents will investigate these hyperlinked guidance Markdown files, but AGENTS.md (or CLAUDE.md) are considered the start points. So we'd link from AGENTS.md to COMPILING.md, CONTRIBUTING.md, SECURITY.md, and I guess README.md. this is exactly the kind of upgrade an agent can perform for us, and if we add AGENTS.md, I can ask Fable for an agent-ready makeover of these files, with AGENTS.md as the root file.
Maybe pick out the fixes or ideas you find most compelling for this PR and merge it, and we'll start evolving all of our guidance Markdown content over time, especially when we see a real-world error that could have been avoided by a little tip in AGENTS.md. Regarding the larger discussion, I am also concerned that it's now easy to produce PRs, but it's ultimately a blessing, even if it means we need vigilant gatekeeping by humans. My fork of Jamulus has a CLAUDE.md that might interest you. One thing I notice is all those architecture pitfalls. I've found different library requirements for deployment to Ubuntu 22 vs. 24 vs. 26, on ARM vs. x86, as well as gotchas when on very small (half-gb) RAM scenarios. |
This was done with open code and corrected by hand. I instructed it to be brief also as I wanted to make sure that small (local) models can also benefit from it. |
|
This YouTube video explains why small models running locally aren't suited to agentic programming tasks, but AGENTS.md is typically loaded into context before every interaction with an agent while using any model, so concision is always in style. |
|
Qwen3 27b works for easy tasks. Maybe that counts as medium sized. Will check the video soon. |
Let me put it this way, none of my local machines is capable of running a local LLM (well, maybe above 2b). Anything in the Jamulus source directory that is going to mislead Cursor or VSCode running against cloud instances is going to have negative impacts - and reducing the scope of what the LLM considers as potential context is as good as misleading it. |
|
Agree. Also I believe context must be as concise and information dense as possible. |
Let's be as concise and dense as possible, but go as deep as we must to prevent common errors and pitfalls.
LLMs will "bridge the gap" and "fill in the blanks" which is fascinating, but the goal of AGENTS.md is to prevent speculations that just lead to failures and problems. I'd like to see a culture here where people submit Markdown PRs for agent use that (concisely and densely) would have prevented the pitfalls they experienced when agentically programming this repo. |
my little rant: |
|
Agree. ARCHITECTURE.md or something comparable should exist - also for humans - in docs. Comparable to the protocol. |
Only if it's clear that the contributor doesn't have understanding of the code. If they know what it does I'm definitely fine with it. |
We will only find out once the PR is open and time has been spent with the code. This should be mitigated and I don't know how other than ban PRs written by LLMs completely. |
|
A robust application of LLM tactics would give you the time protection you want. A submitter with little understanding of the code might drop the hot feature of the century. Today it'll show up alongside 5 loser PRs. Historically we implicitly thought of the submitter as the first reviewer, and that part isn't true anymore. We maintain the gate where the approver needs to really believe in the change regardless of the submitter reputation or apparent understanding. And we know that's not enough, so we insist on two approvers. If we had robust AGENTS, COMPILING, PROTOCOL, and ARCHITECTURE Markdown, the hot feature of the century would show up, and the other 5 wouldn't. Or maybe 1 of those 5 would be redirected by the agent into something worthwhile. So you only need to consider 2 PRs rather than 6. So your thoughtful comments here wouldn't have been necessary, because the agent would have nudged me here before I submitted anything. It's much easier to submit a PR now, and this has led to PRs of dubious merit. AI with robust coding guidance would radically improve the quality of each PR. AI triggered the new problem we've been seeing, but it's also the mitigation you're hoping for. |
I don't think we should accept any generated code, unless we generate it ourselves from a given set of rules. When there is no author we don't need one at all so a corresponding commit makes clear it wasn't contributed by a human. |
Co-authored-by: Peter L Jones <pljones@users.noreply.github.com>
|
For the port TODO: d46a77a is one commit behind — the final version after all the review rounds in #3789 is 22 lines: https://github.com/mcfnord/jamulus/blob/59a167fb0d9d12af5f547a16d9c2ddbbf7fbddbe/AGENTS.md It has the granularity requirement as hard rules at the top (one PR per new file / independent fix / typo; every PR a standalone option reviewable in minutes), the renumbering rule removed, and everything else as links. Two of the linked files exist only if their PRs merge — drop the docs/ARCHITECTURE.md entry unless #3791 lands and the docs/DEPLOY.md entry unless #3790 lands. Take any or all of it. |
Short description of changes
Adds agent instructions for opencode/claude code/... since we have an increasing amount of agent/AI PRs.
CHANGELOG: SKIP
Context: Fixes an issue?
No
Does this change need documentation? What needs to be documented and how?
This is documentation
Status of this Pull Request
Ready for review.
What is missing until this pull request can be merged?
Review for conciseness. @mcfnord had AI contributions - I assume that he knows if this prompt is good enough.
Checklist