Revive the Atomic Forge as a shadcn-style tool registry

[Eigenwise]

Why

Nobody uses the Atomic Forge. That's not because the tools are bad, it's because the Forge was never actually distributable.

Before this PR, the only way to get a tool out of the Forge was to launch a Textual TUI and click through it. That means:

• No agent or script could use it. An AI assistant can't drive a TUI, so when you asked one for a weather tool it just wrote you a new one from scratch. Every time. The Forge was invisible to the exact workflow that should have been its biggest consumer.
• Downloads were broken in a subtle way. The copy step stripped pyproject.toml and requirements.txt, so you got source code with no dependency metadata and had to reverse-engineer the imports.
• There was no private story at all. The source repo was a hardcoded GitHub URL. If you had internal tools, the Forge did nothing for you.
• No machine-readable catalog. Nothing could enumerate the tools without cloning and walking directories.
• No CI. Forge tool tests didn't run. Two suites were quietly red.
• No standard. Nothing defined what a Forge tool is, so drift went unnoticed.

The original idea was for Forge to work like shadcn: a registry that hands you source code you then own. This PR builds that.

What Forge is now

A shadcn-style vendored-code registry, not a hosted runtime or a package you depend on.

You point atomic at one or more Git repositories, list what's in them, and copy a complete tool package into your project. From that moment the code is yours: in your repo, in your review, in your version control, editable without asking anyone.

Deliberately out of scope: MCP/server interoperability. Forge is a distribution path. Git is the source of truth.

How it works

01 SOURCE   A source is a public or private Git repository, branch, and tools directory.
02 INDEX    Each Forge publishes a generated index.json with tool metadata.
03 SELECT   List every source together. Qualify the source when two tools share a name.
04 VENDOR   Copy the complete standalone package, including dependency metadata and tests.

The CLI

atomic with no subcommand still opens the TUI. Everything below is new.

# See every configured Forge tool
atomic list

# Add a private standalone Forge repository
atomic sources add company \
  https://git.example.com/company/atomic-forge.git \
  --tools-path tools

# Install a tool into the current directory
atomic download company/pdf_reader

# Or pick the destination explicitly
atomic download pdf_reader --dest tools/

Command	Behavior
atomic list	Prints source-qualified tools. One broken source does not hide healthy sources. Exits nonzero only when all configured sources fail.
atomic download	Refuses unknown names, ambiguous unqualified names, and an existing destination rather than overwriting your code.
atomic sources	Lists, adds, and removes source definitions stored at ~/.atomic-assembler/sources.json.

Usage recipes

Start with the official Forge

atomic list

# inspect the names, then vendor one
atomic download weather

# you now own ./weather/
cd weather
uv run pytest tests

Install into an application folder

atomic download pdf_reader --dest src/my_app/tools

# no overwrite if the target already exists
git add src/my_app/tools/pdf_reader
git commit -m "Add vendored PDF reader"

Use a private company Forge

# Git auth already works through SSH or your credential helper
atomic sources add company \
  git@git.example.com:platform/forge.git \
  --branch main --tools-path tools

atomic list
atomic download company/internal_search

Resolve a naming collision

# "weather" exists in more than one source
atomic download weather
Ambiguous tool name: weather

# choose exactly which implementation you want
atomic download official/weather
atomic download company/weather

Manage sources cleanly

atomic sources list

# remove a source, never its downloaded tools
atomic sources remove company

# the source config lives here
cat ~/.atomic-assembler/sources.json

Author and publish a Forge tool

# create the normal standalone package first
cd atomic-forge/tools/my_tool
uv run pytest tests

# regenerate the registry index from the repo root
uv run python atomic-forge/scripts/generate_index.py
uv run pytest atomic-forge/conformance

Private Forges: authentication stays outside Forge

This is the design decision I most want reviewed, because it's the one that keeps the feature safe.

Forge never handles credentials. It clones with plain Git and lets your existing setup do the authenticating: SSH keys, a credential helper, whatever your enterprise Git already uses.

Forge does	Forge refuses to do
Clone the source using its configured branch	Store, print, prompt for, or embed credentials
Read its generated index and copy the selected package	Accept source URLs containing user info or credential-like query/fragment values
Support private repositories through ordinary Git access	Follow an index path or a tool-package symlink outside the configured tools directory
	Print terminal control sequences supplied by an untrusted index

• Credentials — use SSH, a Git credential helper, or your existing enterprise Git flow.
• Paths — absolute paths, .. traversal, and escaping symlinks from an untrusted Forge are rejected.
• Ownership — the downloaded tool is ordinary code in your repo, under your review and version control.

A configured Forge source is untrusted input. Its index can name anything it likes, and the code that reads it treats every string as hostile until proven otherwise.

Keeping the registry honest

Forge package contract (enforced by atomic-forge/conformance/):

• Standalone package files, tests, README, pyproject.toml, and requirements.txt
• Typed Atomic Agents schemas, config, and explicit BaseTool[Input, Output]
• Non-empty BaseIOSchema docstrings
• Index name, version, and dependency declarations checked against the package

Automation around it:

• A stdlib-only generator rebuilds atomic-forge/index.json deterministically (with explicit LF output, so it doesn't churn on Windows)
• CI fails if the index is stale, a tool lacks tests, or any tool suite fails
• Plugin skills check Forge before creating a new tool from scratch

Security review trail

The functional work was done first, then audited. Every finding below was found by review, fixed, and re-reviewed. Four audit rounds, three fix rounds.

Round	Finding	Fix
1	Credential-bearing source URLs persisted and displayed	Reject credential-bearing Forge source URLs
1	Malicious index paths could escape the cloned tools directory	Contain Forge index tool paths
1	atomic list returned success when every source failed	Fail list when every Forge source fails
2	Credentials in URL query/fragment bypassed the userinfo-only check	Harden Forge source and index handling
2	copytree dereferenced symlinks, so a hostile tool could exfiltrate ~/.ssh/id_rsa	same
2	Index name/description printed raw, allowing ANSI/OSC terminal injection	same
3	Terminal injection still possible via index tool paths on successful download	Fix Forge source and output hardening regressions
3	Valid ssh://git@host/... URLs were wrongly rejected	same
4	Source metadata output (name, redacted URL, branch, tools path) still raw	Sanitize Forge source terminal output

Final release-gate audit over the whole diff: no confirmed findings.

Also fixed along the way

• GithubRepoCloner accepted a branch argument and then ignored it
• Downloads stripped pyproject.toml and requirements.txt
• atomic download <name> without --dest compared a str against a Path, so its existence check never fired
• searxng_search/requirements.txt declared sympy instead of aiohttp
• bocha_search tests required a real BOCHA_API_KEY
• youtube_transcript_scraper tests mocked an API shape the library no longer has
• Duplicate unused atomic-forge/atomic_tools/tavily_search/ tree removed; test_tavily_seach.py typo renamed
• docs/contributing.md documented a nonexistent atomic-assembler create-tool command and a stale BaseTool example

Verification

uv run pytest atomic-assembler          46 passed, 3 skipped
uv run pytest atomic-forge/conformance  104 passed, 14 warnings
all 13 Forge tool suites                pass
uv run black / flake8                   clean
git diff --check origin/main...HEAD     clean

Two things I want to be straight about:

1. The 3 skips are new and intentional. Those are the symlink-escape tests. They build fixtures with Path.symlink_to, which fails at setup with WinError 1314 on Windows unless the shell holds SeCreateSymbolicLinkPrivilege. CI is ubuntu-latest so it always ran them, but the suite hard-failed for Windows contributors. The last commit probes symlink support and skips rather than failing. They do run in CI.

2. 14 conformance warnings are known and deliberate, not silent failures: 13 tools have no uv.lock (optional), and tavily_search has a legacy requirements.txt/pyproject.toml mismatch on sympy. Warnings, so they're visible without blocking.

One pre-existing failure on main, untouched by this PR: atomic-agents/tests/context/test_chat_history.py::test_get_class_from_string. This branch changes no files under atomic-agents/.

Review notes

Worth a careful look:

• atomic-assembler/atomic_assembler/constants.py — URL validation and the display-safety policy. This is the security boundary.
• atomic-assembler/atomic_assembler/utils.py — index path containment and symlink handling during copy.
• The judgment call that Forge stores no credentials ever, and whether pushing users to SSH/credential-helper is the right call for your setups.

🤖 Generated with Claude Code