Skip to content

Document debuginfo repr directive #2947

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Walnut356 wants to merge 1 commit into
base: main
Choose a base branch
from
+256 −20
Open

Document debuginfo repr directive #2947

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 3 additions & 6 deletions src/debuginfo/lldb-visualizers.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -347,12 +347,9 @@ The category we use will be called `Rust`.
> TIP: all LLDB commands can be prefixed with `help` (e.g. `help type synthetic add`) for a brief
description, list of arguments, and examples.

As of <!-- date-check --> Nov 2025,
we use `command source ...`, which executes a series of CLI commands from the
file [`lldb_commands`](https://github.com/rust-lang/rust/blob/main/src/etc/lldb_commands) to add
providers.
This file is somewhat unwieldy, and will soon be supplanted by the Python API equivalent
outlined below.
In the past, we used `command source ...`, which executes a series of CLI commands from the
file `lldb_commands` to add providers. This file was somewhat unwieldy, and has been supplanted by
the Python API equivalent outlined below.

## `__lldb_init_module`

Expand Down
213 changes: 209 additions & 4 deletions src/debuginfo/testing.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,8 +1,213 @@
# Testing

The debug info test suite is undergoing a substantial rewrite. This section will be filled out as
the rewrite makes progress.

Please see [this tracking issue][148483] for more information.
> [!IMPORTANT]
> The debug info test suite is undergoing a substantial rewrite. This section will be filled out as
> the rewrite makes progress.
>
> Please see [this tracking issue][148483] for more information.

[148483]: https://github.com/rust-lang/rust/issues/148483

Debug info tests check a few important things:

* Are we outputting the information in the way we expect?
* Is what we output readable by the debugger?
* Do our visualizers work the way we expect?

The first question is typically answered by `tests/codegen-llvm`, but debug info generation is often
tested incidentally, rather than deliberately. At time of writing, there is a much larger focus on
the latter two questions, and those will be covered in detail here. The tests that answer those
questions live in `tests/debuginfo`, which is executed by `compiletest`.

For much of the test suite's lifespan, debuggers were discovered automatically, and tests were
tests were comprised of `$DEBUGGER-command` and `$DEBUGGER-check` directives (i.e. raw string
comparisons) that checked variable printing, breakpoint locations, etc. Put bluntly, this system was
a nightmare and lead to a [litany of issues](https://github.com/rust-lang/rust/issues/134682).
To help remedy this:

1. [`tests/debuginfo` is now opt-in](https://github.com/rust-lang/rust/pull/159455) for GDB and LLDB.
2. a new directive was added: `$DEBUGGER-repr`. This directive dispatches to custom logic that polls
the debugger for additional information that isn't visible in the printed output. It also
automatically separates output by target, allowing the tests to be run on different platforms
without conflicts.

# The `repr` directive

> [!IMPORTANT]
> As of July 2026, this command is only supported by LLDB. GDB support is planned, but
> has not been implemented. It is unclear whether this directive will ever be suited for use with
> CDB.

In short, `$DEBUGGER-repr` commands are desugared to:

```
//@ $DEBUGGER-command:repr $VAR_NAME
//@ $DEBUGGER-check:$VAR_NAME ok
```

When the commands are passed to the debugger, our test framework intercepts `repr` pseudo-commands
and runs special logic on them, testing against data stored in
`tests/debuginfo/<test_name>/input/<debugger>_input/<target_group>.json`.

"Target groups" cover the set of targets where we cannot guarantee identical output. Those targets
are defined by the
[`Target` enum in `common.py`](https://github.com/rust-lang/rust/blob/bf9944f0b8006b152ef4d5f408ae75a0dde3d044/src/etc/lldb_batchmode/common.py#L54).
At time of writing, this list includes `non_windows`, `windows_gnu`, and `windows_msvc`. It is
intentionally kept as short as possible, since each target is a new set of test data that must be
updated when changes are made. There is still not a perfect solution for how tests can be
`--bless`-ed by contributors who do not have access to all of the targets.

The input data can be automatically updated for expected changes by adding `--bless` to the test
invocation (e.g. `./x test tests/debuginfo/basic-types/main.rs --bless`). `--bless` updates the
in-memory representation, tests against it, and if no errors occur, saves the data back to the
target file (or creates a new file if necessary).

The schema of the input data is defined by the classes in
[`common.py`](https://github.com/rust-lang/rust/blob/be3d26db984c6f96335faca1f254dc04873cb1c1/src/etc/lldb_batchmode/common.py).
The top-level container is `TargetData`. This
schema is identical for all debuggers.

## Converting existing tests

Nearly any time a variable is tested, the `repr` directive should be preferred over `command` +
`check`. At time of writing, only a single test has been converted over, but more will follow as
part of the test rewrite mentioned above. Thankfully, the conversion process is fairly easy. For
a given check:

```
//@ lldb-command:v foo
//@ lldb-check:<foo output>
```

The equivalent `repr` test is:

```
//@ lldb-repr:foo
```

Once all `command` + `checks` are converted to `repr`, run the tests with the `--bless` option. If
you have access to additional targets, `--bless` the data for the remainder of the target groups
as well (e.g. if you are on a Windows machine, bless once for `x86_64-pc-windows-msvc`, once for
`x86_64-pc-windows-gnu`, and use WSL to bless for `x86_64-unknown-linux-gnu`).


## Implementation

### Ser/De

`TargetData` is converted to a dictionary with `dataclasses.asdict`, and is serialized with Python's
built-in JSON library. When testing, the data is read into a `dict`, converted to a `TargetData`,
and stored in the top level `INPUT_DATA` variable. The current deserialization logic should be
resilient to changes in the schema, but requires that all fields contain ONLY types that can be
directly serialized/deserialized by `json.dumps`. The acceptable types are those that make up
[`common.JsonType`](https://github.com/rust-lang/rust/blob/bf9944f0b8006b152ef4d5f408ae75a0dde3d044/src/etc/lldb_batchmode/common.py#L17)

Since the serialization/deserialization is decoupled from the debugger logic, we can easily switch
to an alternative format if we find a better alternative to json.

The conversion logic from the debugger's internal representation to our schema classes lives in
`from_$DEBUGGER.py`.

Once imported, `common` automatically deserializes any existing input data and [stores it in the
global variable `INPUT_DATA`](https://github.com/rust-lang/rust/blob/bf9944f0b8006b152ef4d5f408ae75a0dde3d044/src/etc/lldb_batchmode/common.py#L523). This
data is what we test against.

> [!NOTE]
> Special care was taken to prevent `lldb_batchmode` from importing `common` unless a `repr` command
> was actually processed. This saves us from reading/writing input data for tests that don't need
> it.

#### Format minutae

Since type information is unique and unchanging once the debug session has begun, types are only
stored once at the top level, and are referred to by name everywhere else.

Pointer values change from run to run. To prevent mismatches, pointer variables do not store their
value. This is equivalent to the wildcard `[...]` used in `-check` directives.

`BlessMetadata` is included in `TargetData`, but is not tested against. It exists solely as a record
of how the test data was generated, to help in diagnosing issues that may occur due to Python or the
debugger changing versions.

### Entry point and `--bless`

Upon encountering a `repr` pseudo-command, `lldb_batchmode.main` dispatches to
`check_$DEBUGGER.check()`. If the `--bless` option was specified, the variable is converted from
the in-memory representation to our equivalent schema class. This includes the variable's type,
visualizers, children, the children's types, etc. Once inserted into `TargetData`, the variable is
tested against the data that was just saved to `TargetData`.

If no exception or errors occurred and the `--bless` option was specified, `INPUT_DATA` is written
to the appropriate file just before `lldb_batchmode` exits. If errors occur, `INPUT_DATA` is simply
discarded.

Currently, the `repr` pseudo-command is checked for directly. GDB and LLDB both support creating
custom CLI commands via Python code. In the future, `repr` may be implemented as a CLI command for
one or both debuggers.

### Check logic

`check_$DEBUGGER.check` converts the debugger's variable object into a `Variable` object and
compares the two. If any mismatches are found, further processing is done to report errors in a more
helpful manner. This means that errors are encountered and reported immediately, which has a number
of advantages. Most importantly, since the debugger state has not changed since the failure, and we
still have access to the debugger's variable object, we can poll the debugger for more information
to provide more useful error messages.

For example, LLDB can be a bit coy when it comes to reporting errors that occur within
synthetic/summary provider calls. This is especially true when running the command within another
command, as the tests do by calling `script import lldb_batchmode; lldb_batchmode.main()` and
executing commands in that context.

When we encounter an error, we can import the appropriate summary provider, pass the variable object
to it, and print the exception ourselves. We can also inspect the synthetic provider class to make
sure it implements all the mandatory functions.

Errors *do not* immediately end the test. This is especially important now that a `--bless` option has been added.
`--bless` updates all of the input data, so we need to print all of the errors so the reader can
make an informed decision about whether or not there are further changes that need to be made. We
absolutely do not want people accidentally blessing bad data purely because the 1st error happened
to be an expected change.

Errors are printed directly to `stdout` to appear as visible output from the `repr` pseudo command.
There are [several error helper functions](https://github.com/rust-lang/rust/blob/e7b595554e664e6bd281c8cf881093d6c71bc0e1/src/etc/lldb_batchmode/common.py#L35-L51)
to keep formatting consistent.

> [!NOTE]
> When LLDB is running a `script` command, it does not print the Python interpreter's `stderr`.
> If the interpreter exits with an exception, it will print that, but none of the rest of `stderr`.
> Instead, if we decide we want to print to `stderr`, we can use the debugger's by calling
> `lldb.debugger.GetErrorFileHandle` which returns a Python `io.TextIoWrapper`.

If no errors occurred for a given variable, `$VAR_NAME ok` is printed to `stdout` for `compiletest`
to match against.

Before `lldb_batchmode` exits, one last check is done to ensure that all the types and variables
that were present in `INPUT_DATA` have been checked against. If this check fails, the script reports
the untested types/variables and exits with an error code.

# LLDB versioning

Apple distributes a fork of LLDB with Xcode that contains Swift support. This fork of LLDB does not
use the same versioning scheme as LLVM's LLDB:

```
# Apple:
lldb-1703.0.236.21 Apple Swift version 6.2.3 (swiftlang-6.2.3.3.21 clang-1700.6.3.2)
# LLVM:
lldb version 22.1.2 (https://github.com/llvm/llvm-project revision 1ab49a973e210e97d61e5db6557180dcb92c3e98)
clang revision 1ab49a973e210e97d61e5db6557180dcb92c3e98
llvm revision 1ab49a973e210e97d61e5db6557180dcb92c3e98
```

It does not appear that the Apple LLDB's version is derived from LLVM's version, so we cannot easily
or automatically convert between the two. Luckily, we can still check the base LLVM version manually
by checking the appropraite release branch in the Swift LLVM repo. For our example above, no branch
exists for `Swift 6.2.3`, but there is one for `6.2.2`. The LLVM version is located in
[`llvm/utils/gn/secondary/llvm/version.gni`](https://github.com/swiftlang/llvm-project/blob/swift/release/6.2.2/llvm/utils/gn/secondary/llvm/version.gni).
As we can see from that example, the Apple LLDB version above corresponds to (roughly) LLVM LLDB
`19.1.5`.

This can be useful when diagnosing or writing new tests, as it allows us to get a better idea of
what features are available in the Apple LLDB used in CI. For example, LLDB 19 was the first version
to support Type Recognizer functions, so we can assume our example Apple LLDB supports them.
54 changes: 44 additions & 10 deletions src/tests/compiletest.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -216,9 +216,14 @@ A simple example of a test using `rustc_clean` is the [hello_world test].

### Debuginfo tests

The tests in [`tests/debuginfo`] test debuginfo generation.
They build a program, launch a debugger, and issue commands to the debugger.
A single test can work with cdb, gdb, and lldb.
>[!IMPORTANT]
> As of [#159455](https://github.com/rust-lang/rust/pull/159455) These tests were made
> opt-in. For further context, see:
> [Stabilizing the state of the debuginfo test suite](https://github.com/rust-lang/compiler-team/issues/1012)

The tests in [`tests/debuginfo`] test how debuginfo is interpreted by the supported debuggers, and
confirm our visualizers still work as expected. They build a program, launch a debugger, and issue
commands to the debugger. A single test can work with cdb, gdb, and lldb.

Most tests should have the `//@ compile-flags: -g` directive or something
similar to generate the appropriate debuginfo.
Expand All @@ -228,20 +233,16 @@ To set a breakpoint on a line, add a `// #break` comment on the line.
The debuginfo tests consist of a series of debugger commands along with
"check" lines which specify output that is expected from the debugger.

The commands are comments of the form `// $DEBUGGER-command:$COMMAND` where
The commands are comments of the form `//@ $DEBUGGER-command:$COMMAND` where
`$DEBUGGER` is the debugger being used and `$COMMAND` is the debugger command to execute.

The debugger values can be:

- `cdb`
- `gdb`
- `gdbg` — GDB without Rust support (versions older than 7.11)
- `gdbr` — GDB with Rust support
- `lldb`
- `lldbg` — LLDB without Rust support
- `lldbr` — LLDB with Rust support (this no longer exists)

The command to check the output are of the form `// $DEBUGGER-check:$OUTPUT`
The command to check the output are of the form `//@ $DEBUGGER-check:$OUTPUT`
where `$OUTPUT` is the output to expect.

For example, the following will build the test, start the debugger, set a
Expand All @@ -262,6 +263,35 @@ fn main() {
fn b() {}
```

Additionally, there is a special command, `//@ $DEBUGGER-repr:$VAR_NAME` intended to verify
variables (and their visualizers) with more granularity than can be achieved with simple string
comparison. This directive should be preferred over the `-command`/`-check` whenever possible.

> [!NOTE]
> At time of writing (July 2026) this command is limited to LLDB, with an implementation coming soon
> for GDB. There are not firm plans to port the logic to CDB.

This command effectivly desugars into:

```
//@ $DEBUGGER-command:repr $VAR_NAME
//@ $DEBUGGER-check:$VAR_NAME ok
```

The `repr $VAR_NAME` command is intercepted by special logic that uses the debuggers' API to inspect
data that isn't reflected in the variable's printed output. The variable in memory is compared
against input data stored in
`tests/debuginfo/<test_name>/input/<debugger>_input/<target_group>.json` and
provides detailed error messages on failure.

> [!IMPORTANT]
> `-repr` directives **are** compatible with the `--bless` option, unlike `-command`/`-check`.
> `--bless`-ing a file with `-repr` commands will automatically create/update the appropriate
> target's input data file.

The implementation details of this command are further described in
[the Testing section of the Debug Info chapter](../debuginfo/testing.md).

The following [directives](directives.md) are available to disable a test based on
the debugger currently being used:

Expand All @@ -272,7 +302,11 @@ the debugger currently being used:
to the given version
- `ignore-gdb-version: 7.11.90 - 8.0.9` — ignores the test if the version of
gdb is in a range (inclusive)
- `min-lldb-version: 310` — ignores the test if the version of lldb is below the given version
- `min-apple-lldb-version: 1703.0.236.21`/`min-llvm-lldb-version: 21.1.0` — ignores the test if the
version of lldb is below the given version.
Note: Apple's fork of LLDB (distributed with Xcode) uses a different versioning scheme that is not
easily mappable to LLVM's LLDB version numbers. As such, the version gates are specified by
vendor. Further info on manually checking version equivalence is available [here](../debuginfo/testing.md#lldb-versioning)
- `rust-lldb` — ignores the test if lldb is not contain the Rust plugin.
NOTE: The "Rust" version of LLDB doesn't exist anymore, so this will always be ignored.
This should probably be removed.
Expand Down
Loading