Add OpenCode + NVIDIA Nemotron custom app - #473
Conversation
|
If this is for a NVIDIA demo, would it be fine staying in/running from a branch? Does it need to be merged into master? |
@pantherman594 Technically speaking, we can pull in a custom app from a feature branch, so we don't need to merge into |
|
Decision: Wait on merge until after demo |
| messages=[{"role": "user", "content": "Hello!"}], | ||
| max_tokens=100, | ||
| ) | ||
| print(response.choices[0].message.content) |
There was a problem hiding this comment.
did you test this? When I tried, using response.choices[0].message.content didn't work, but response.choices[0].message.reasoning} did - apparently this is a characteristic of the Nemotron model's output format?
|
|
||
| ## Changing the Model | ||
|
|
||
| The model tag lives in `OLLAMA_MODEL` in `docker-compose.yaml`. Edit it in your |
There was a problem hiding this comment.
nit: it would be great if this was more configurable than having to edit the docker-compose.yaml file before app creation, especially bc you can change the config of a Workbnech app after it's stopped, so if you wanted to move from A100 -> L4, it'd break
3 participants
Adds
src/opencode-nemotron: code-server with the opencode agent driving a local NVIDIA Nemotron model on Ollama. Nothing leaves the VM. For the NVIDIA dev day demo. Derived fromfeature/applegath/ollama_gemma-4-26B-A4B.This PR:
opencode1.18.22.nemotron-3.5-lightning:30b— 30B MoE, tool calling, 25 GB of weights. Needs an A100 40 GB.~/.config/opencode/opencode.jsonon create and restart, withshare: "disabled"so sessions never reach opencode's hosted sharing service.OLLAMA_CONTEXT_LENGTH=32768. Tool calls are unreliable below ~16k.Two deviations worth review:
docker-compose.yaml, not a template option. The VM substitutes onlylogin,cloud,containerImage,containerPort,shmSize, andmemoryLimit(startupscript/butane/050-parse-devcontainer.sh). A custom option stays literal and breaksdocker-compose config.ghcr.io/devcontainers/features/java. That feature is currently broken for all 13 apps that use it, includingvscode-with-llmandnemo_jupyter: Microsoft's Java 17 SDKMAN identifier is now17.0.20+1-ms, and the feature's regex rejects the+1segment. Upstream: java: major-only version fails to resolve once a vendor publishes a four-component release (21.0.12.1 → 21.0.12+1-ms) devcontainers/features#1712, fix in #1713. Pinning the digest does not help — the lookup is against SDKMAN's live catalog. The other apps need a separate ticket. This app stays unaffected by the upstream outcome either way;install-java.shonly needs/usr/bin/java.Testing
Deployed to
samh-sandboxona2-highgpu-1g(1x A100 40 GB, us-central1-c):ollama ps→nemotron-3.5-lightning:30b, 25 GB, 100% GPU, context 32768.opencode run "Read hello.txt and tell me exactly how many lines it has"→ agent called its read tool and answered correctly.java17.0.19 andwbon thePATH, sopost-startup.shsucceeds. code-server answers on 8443.Not registered in
test-pr.yaml: CI runsdevcontainer up, which would pull 25 GB on a GPU-less runner.BENCH-9850