Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .gitmodules
Original file line number Diff line number Diff line change
Expand Up @@ -11,3 +11,6 @@
path = docs/pingcap-docs/docs
url = git@github.com:pingcap/docs.git
branch = release-8.5
[submodule "ref/fs"]
path = ref/fs
url = git@github.com:tidbcloud/fs.git
150 changes: 77 additions & 73 deletions AGENTS.md

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,7 @@ live-e2e-db: build
$(LIVE_E2E_RUN) -run '^TestLiveDB'

live-e2e-fs: build
$(LIVE_E2E_RUN) -run '^TestLive(FSResourceRegistryLifecycle|FSCommandSurface|FSConfigurationFreeAccess|FSDataPlaneLifecycle|FSMountRuntime|FSWebDAVMountRuntime)$$'
$(LIVE_E2E_RUN) -run '^TestLive(FSRemoteInventoryLifecycle|FSCommandSurface|FSConfigurationFreeAccess|FSDataPlaneLifecycle|FSMountRuntime|FSWebDAVMountRuntime)$$'

live-e2e-fs-git: build
$(LIVE_E2E_RUN) -run '^TestLiveFSGit'
Expand Down
37 changes: 21 additions & 16 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,14 +16,14 @@ With `ti`, an agent can persist state between sessions, share files across sandb
1. Create a file system and obtain the file system token (performed once, outside the sandbox).

```shell
export TI_FS_TOKEN="$(ti fs create-file-system --file-system-name agent-workspace --region <REGION_CODE> --wait --query fs_token --output text)"
export TI_FS_TOKEN="$(ti fs create-file-system --region <REGION_CODE> --wait --query fs_token --output text)"
```

2. Mount the filesystem to a local path and use it as a normal POSIX-compliant filesystem (performed within the sandbox)

```shell
export TI_FS_TOKEN="<FS_TOKEN>"
ti fs mount-file-system --file-system-name agent-workspace --mount-path /path-to-workspace --region <REGION_CODE>
ti fs mount-file-system --mount-path /path-to-workspace --region <REGION_CODE>
echo "Hello Sandbox Workspace!" >> /path-to-workspace/hello.txt
```

Expand Down Expand Up @@ -103,7 +103,7 @@ Automation should move to `TI_*` and `TIDB_CLOUD_*` environment variables. The v

- Authentication: a TiDB Cloud Public Key and a Private Key from the [TiDB Cloud API Keys](https://tidbcloud.com/org-settings/api-keys) console.
- Default region: one of aws-us-east-1, aws-us-west-2, aws-eu-central-1, aws-ap-northeast-1, aws-ap-southeast-1, or ali-ap-southeast-1.
- Regions support TiDB Cloud Filesystem: aws-us-east-1, aws-us-west-2, aws-ap-southeast-1, and ali-ap-southeast-1.
- Regions support TiDB Cloud Filesystem: aws-us-east-1, aws-us-west-2, aws-ap-southeast-1, or ali-ap-southeast-1.
- Regions support TiDB Cloud Starter: aws-us-east-1, aws-us-west-2, aws-eu-central-1, aws-ap-northeast-1, aws-ap-southeast-1, or ali-ap-southeast-1.

Set up a default profile with one command:
Expand Down Expand Up @@ -156,37 +156,41 @@ An integration can add optional process-scoped attribution without changing a pr

### TiDB Cloud Filesystem

The following example uses `jq` to extract the server-assigned ID and one-time token from one create response.

```shell
mkdir ~/my-workspace
ti fs create-file-system --file-system-name my-workspace --wait
ti fs mount-file-system --file-system-name my-workspace --mount-path ~/my-workspace
umask 077
ti fs create-file-system --wait > ./filesystem.json
export FILE_SYSTEM_ID="$(jq -r '.file_system_id' ./filesystem.json)"
export TI_FS_TOKEN="$(jq -r '.fs_token' ./filesystem.json)"
rm ./filesystem.json
ti fs mount-file-system --file-system-id "$FILE_SYSTEM_ID" --mount-path ~/my-workspace
```

Automatic mounting uses FUSE on Linux and WebDAV on macOS and Windows. macOS users can install macFUSE and explicitly add `--driver fuse` for the full FUSE experience.

One profile can manage multiple file systems. ti never infers which resource a command targets, so provide `--file-system-name` for one-off commands or set `TI_FS_FILE_SYSTEM_NAME` for repeated commands:
`ti fs list-file-systems` reads the region-scoped remote inventory through TiDB Cloud credentials. A profile can access multiple file systems, including resources created on another machine. Data-plane commands never infer a resource from the number of local credentials, so provide `--file-system-id` or set `TI_FS_FILE_SYSTEM_ID`:

```shell
ti fs create-file-system --file-system-name scratch
ti fs list-file-systems
ti fs describe-file-system --file-system-name scratch
export TI_FS_FILE_SYSTEM_NAME=scratch
ti fs describe-file-system --file-system-id "$FILE_SYSTEM_ID"
export TI_FS_FILE_SYSTEM_ID="$FILE_SYSTEM_ID"
ti fs list-files
```

`create-file-system` returns an file system token (`fs_token`) in its JSON result. This is the file system owner credential and should be handled as a secret. A configured machine can provision a file system and capture the token without printing the full result:

```shell
export TI_FS_TOKEN="$(ti fs create-file-system --file-system-name agent-workspace --wait --query fs_token --output text)"
```
`create-file-system` does not accept a user-defined name. Drive9 assigns the stable `file_system_id`, and the command returns the owner credential as `fs_token` once in its JSON result. Treat it as a secret. The example above captures both fields from one provisioning request and removes the temporary owner-only JSON file immediately.

An agent sandbox can then use that existing file system without running `ti configure` or providing TiDB Cloud API keys:

```shell
export TI_FS_TOKEN="<FS_TOKEN>"
ti fs mount-file-system --file-system-name agent-workspace --mount-path /path_to_workspace --region aws-us-east-1
export TI_REGION_CODE="aws-us-east-1"
ti fs mount-file-system --mount-path /path_to_workspace
```

The token contains its file system ID, so a clean sandbox does not need `TI_FS_FILE_SYSTEM_ID`. Set that variable only as an optional consistency assertion. To persist an existing token on another configured or unconfigured machine, run `ti fs import-file-system-token --from-file ./fs-token`; subsequent commands can select its ID without resupplying the token.

### TiDB Cloud Starter

`ti db` currently implements TiDB Cloud Starter only. Commands that do not identify an existing cluster require `--db-cluster-type starter`; there is no implicit default. Commands with `--db-cluster-id` discover the authoritative service plan and route internally without accepting a type flag. Essential, Premium, Dedicated, unknown, and conflicting plan metadata are rejected before the requested product operation.
Expand Down Expand Up @@ -228,6 +232,7 @@ ti db format-db-connection-string
ti db execute-sql-statement

ti fs create-file-system
ti fs import-file-system-token
ti fs delete-file-system
ti fs list-file-systems
ti fs describe-file-system
Expand Down Expand Up @@ -294,7 +299,7 @@ ti update
ti update --target-version v0.1.1
```

`ti update` downloads and verifies both `ti` and its `ti-drive9` companion before replacing either binary in the user-writable install directory. It never requests sudo. The old `tdc update` command cannot migrate to a differently named executable; install `ti` once with the new installer instead.
`ti update` downloads and verifies both `ti` and its `ti-drive9` companion before replacing either binary in the user-writable install directory. It never requests sudo. The old `ti update` command cannot migrate to a differently named executable; install `ti` once with the new installer instead.

## Documentation

Expand Down
2 changes: 1 addition & 1 deletion docs/pingcap-docs/docs
Submodule docs updated 390 files
71 changes: 35 additions & 36 deletions docs/present.md
Original file line number Diff line number Diff line change
Expand Up @@ -147,63 +147,62 @@ SQL 默认通过 HTTPS SQL API 执行,一次命令只执行一个 statement。

## 4. 创建并管理 Filesystem

创建名为 `ti-demo-workspace` 的资源。一个 profile 可以注册多个 Filesystem;每个资源都必须通过名称显式选择
创建一个由服务端分配稳定 ID 的资源。远端 inventory 是资源状态的权威来源,本地只保存按 ID 索引的访问凭证

```bash
bin/ti fs create-file-system \
--file-system-name ti-demo-workspace \
--dry-run
bin/ti fs create-file-system --dry-run

bin/ti fs create-file-system \
--file-system-name ti-demo-workspace \
--wait
export FILE_SYSTEM_ID="$(bin/ti fs create-file-system \
--wait \
--query file_system_id \
--output text)"

bin/ti fs list-file-systems --output text
bin/ti fs describe-file-system \
--file-system-name ti-demo-workspace \
--file-system-id "$FILE_SYSTEM_ID" \
--output text

bin/ti fs check-file-system \
--file-system-name ti-demo-workspace \
--file-system-id "$FILE_SYSTEM_ID" \
--output text
```

创建命令的 JSON 结果包含一次性的 `fs_token`。它是资源 owner credential,不能写入日志或公开传递。资源元数据和凭证分别存储在 `~/.ti/fs_resources/<profile-key>/<resource-key>/` 下,不写入主 `~/.ti/credentials`。
创建命令的 JSON 结果包含一次性的 `fs_token`。它是资源 owner credential,不能写入日志或公开传递。凭证存储在 `~/.ti/fs_credentials/<profile-key>/<file-system-id-key>/credentials`,不写入主 `~/.ti/credentials`。

## 5. 使用 Data Plane 操作文件

```bash
bin/ti fs create-directory \
--file-system-name ti-demo-workspace \
--file-system-id "$FILE_SYSTEM_ID" \
--path /demo

printf 'hello from data plane\n' | bin/ti fs copy-file \
--file-system-name ti-demo-workspace \
--file-system-id "$FILE_SYSTEM_ID" \
--from-stdin \
--to-remote /demo/from-data-plane.txt \
--tag source=data-plane \
--description "created through ti fs data plane"

bin/ti fs list-files \
--file-system-name ti-demo-workspace \
--file-system-id "$FILE_SYSTEM_ID" \
--path /demo \
--output text

bin/ti fs read-file \
--file-system-name ti-demo-workspace \
--file-system-id "$FILE_SYSTEM_ID" \
--path /demo/from-data-plane.txt

bin/ti fs describe-file \
--file-system-name ti-demo-workspace \
--file-system-id "$FILE_SYSTEM_ID" \
--path /demo/from-data-plane.txt \
--output text
```

Unix-style alias 只缩短命令名,flags 仍使用完整名称:

```bash
bin/ti fs ls --file-system-name ti-demo-workspace --path /demo --output text
bin/ti fs cat --file-system-name ti-demo-workspace --path /demo/from-data-plane.txt
bin/ti fs ls --file-system-id "$FILE_SYSTEM_ID" --path /demo --output text
bin/ti fs cat --file-system-id "$FILE_SYSTEM_ID" --path /demo/from-data-plane.txt
```

## 6. 挂载并验证双向可见性
Expand All @@ -213,7 +212,7 @@ export MOUNT_PATH="/tmp/ti-demo-${DEMO_ID}"
mkdir -p "$MOUNT_PATH"

bin/ti fs mount-file-system \
--file-system-name ti-demo-workspace \
--file-system-id "$FILE_SYSTEM_ID" \
--mount-path "$MOUNT_PATH"
```

Expand All @@ -231,7 +230,7 @@ cat "$MOUNT_PATH/demo/from-data-plane.txt"
printf 'hello from mount\n' > "$MOUNT_PATH/demo/from-mount.txt"

bin/ti fs read-file \
--file-system-name ti-demo-workspace \
--file-system-id "$FILE_SYSTEM_ID" \
--path /demo/from-mount.txt
```

Expand All @@ -249,14 +248,14 @@ bin/ti fs drain-file-system --mount-path "$MOUNT_PATH"
mkdir -p "$MOUNT_PATH/repos"

bin/ti fs-git clone-git-workspace \
--file-system-name ti-demo-workspace \
--file-system-id "$FILE_SYSTEM_ID" \
--repo-url https://github.com/octocat/Hello-World.git \
--target-path "$MOUNT_PATH/repos/hello" \
--blobless \
--hydrate background

bin/ti fs-git hydrate-git-workspace \
--file-system-name ti-demo-workspace \
--file-system-id "$FILE_SYSTEM_ID" \
--target-path "$MOUNT_PATH/repos/hello"

git -C "$MOUNT_PATH/repos/hello" status --short
Expand All @@ -266,15 +265,15 @@ git -C "$MOUNT_PATH/repos/hello" status --short

```bash
bin/ti fs-git add-git-worktree \
--file-system-name ti-demo-workspace \
--file-system-id "$FILE_SYSTEM_ID" \
--base-path "$MOUNT_PATH/repos/hello" \
--worktree-path "$MOUNT_PATH/repos/hello-feature" \
--branch-name demo-feature

git -C "$MOUNT_PATH/repos/hello-feature" status --short

bin/ti fs-git remove-git-worktree \
--file-system-name ti-demo-workspace \
--file-system-id "$FILE_SYSTEM_ID" \
--worktree-path "$MOUNT_PATH/repos/hello-feature" \
--force
```
Expand All @@ -287,31 +286,31 @@ bin/ti fs-git remove-git-worktree \
export JOURNAL_ID="jrn-demo-${DEMO_ID}"

bin/ti fs-journal create-journal \
--file-system-name ti-demo-workspace \
--file-system-id "$FILE_SYSTEM_ID" \
--journal-id "$JOURNAL_ID" \
--journal-kind agent \
--title "ti demo ${DEMO_ID}" \
--actor agent:demo \
--label demo=present

bin/ti fs-journal append-journal-entries \
--file-system-name ti-demo-workspace \
--file-system-id "$FILE_SYSTEM_ID" \
--journal-id "$JOURNAL_ID" \
--entry-json '{"type":"demo.started"}' \
--entry-json '{"type":"demo.completed"}'

bin/ti fs-journal read-journal-entries \
--file-system-name ti-demo-workspace \
--file-system-id "$FILE_SYSTEM_ID" \
--journal-id "$JOURNAL_ID" \
--output text

bin/ti fs-journal search-journal-entries \
--file-system-name ti-demo-workspace \
--file-system-id "$FILE_SYSTEM_ID" \
--entry-type demo.completed \
--include-entries

bin/ti fs-journal verify-journal \
--file-system-name ti-demo-workspace \
--file-system-id "$FILE_SYSTEM_ID" \
--journal-id "$JOURNAL_ID" \
--output text
```
Expand All @@ -324,13 +323,13 @@ Journal 是 append-only、可验证的 workflow ledger,不是普通文本日
printf 'demo-token\n' > /tmp/ti-demo-token.txt

bin/ti fs-vault create-secret \
--file-system-name ti-demo-workspace \
--file-system-id "$FILE_SYSTEM_ID" \
--secret-name demo-service \
--field ENDPOINT=https://example.invalid \
--field API_TOKEN=@/tmp/ti-demo-token.txt

bin/ti fs-vault read-secret \
--file-system-name ti-demo-workspace \
--file-system-id "$FILE_SYSTEM_ID" \
--secret-name demo-service \
--field ENDPOINT \
--format raw
Expand All @@ -340,22 +339,22 @@ bin/ti fs-vault read-secret \

```bash
export TI_VAULT_TOKEN="$(bin/ti fs-vault create-grant \
--file-system-name ti-demo-workspace \
--file-system-id "$FILE_SYSTEM_ID" \
--agent-id demo-agent \
--scope demo-service/ENDPOINT \
--permission read \
--ttl 10m \
--token-only)"

bin/ti fs-vault read-secret \
--file-system-name ti-demo-workspace \
--file-system-id "$FILE_SYSTEM_ID" \
--secret-name demo-service \
--field ENDPOINT \
--format raw \
--vault-token "$TI_VAULT_TOKEN"

bin/ti fs-vault list-audit-events \
--file-system-name ti-demo-workspace \
--file-system-id "$FILE_SYSTEM_ID" \
--secret-name demo-service \
--limit 20 \
--output text
Expand All @@ -377,17 +376,17 @@ rm -rf "$MOUNT_PATH"

```bash
bin/ti fs-vault delete-secret \
--file-system-name ti-demo-workspace \
--file-system-id "$FILE_SYSTEM_ID" \
--secret-name demo-service

rm -f /tmp/ti-demo-token.txt
```

删除 Filesystem 资源及其本地 registry entry:
删除 Filesystem 资源及其本地 credential entry:

```bash
bin/ti fs delete-file-system \
--file-system-name ti-demo-workspace
--file-system-id "$FILE_SYSTEM_ID"
```

删除演示 cluster:
Expand Down
Loading