Repository navigation
[R] Attributes not preserved for list columns #54682
Description
Describe the bug, including details regarding any error messages, version, and platform.
?Schema says the following about metadata and list columns.
When converting a data.frame to an Arrow Table or RecordBatch, attributes from the data.frame are saved alongside tables so that the object can be reconstructed faithfully in R (e.g. with as.data.frame()). This metadata can be both at the top-level of the data.frame (e.g. attributes(df)) or at the column (e.g. attributes(df$col_a)) or for list columns only: element level (e.g. attributes(df[1, "col_a"])).
Here's an example showing that top-level attributes for a data frame and attributes for individual columns are preserved, but not for elements of a list column.
x <- structure(1:2, comment = "test")
test_df <- list(
x1 = x,
x2 = list(1, x)
) |>
list2DF() |>
structure(comment = "test")
attributes(test_df[1, "x2"]) <- list(comment = "TEST")
str(test_df)
#> 'data.frame': 2 obs. of 2 variables:
#> $ x1: int 1 2
#> ..- attr(*, "comment")= chr "test"
#> $ x2:List of 2
#> ..$ : num 1
#> ..$ : int 1 2
#> .. ..- attr(*, "comment")= chr "test"
#> - attr(*, "comment")= chr "test"
arrow_table <- arrow::as_arrow_table(test_df)
arrow_table$metadata
#> $r
#> $r$attributes
#> $r$attributes$comment
#> [1] "test"
#>
#>
#> $r$columns
#> $r$columns$x1
#> $r$columns$x1$attributes
#> $r$columns$x1$attributes$comment
#> [1] "test"
#>
#>
#> $r$columns$x1$columns
#> NULL
#>
#>
#> $r$columns$x2
#> NULL
str(as.data.frame(arrow_table))
#> 'data.frame': 2 obs. of 2 variables:
#> $ x1: int 1 2
#> ..- attr(*, "comment")= chr "test"
#> $ x2: list<double> [1:2]
#> ..$ : num 1
#> ..$ : num 1 2
#> ..@ ptype: num
#> - attr(*, "comment")= chr "test"Created on 2026-10-19 with reprex v2.1.1
I ran into this issue when trying to write a data frame with a list column of named vector with arrow::write_parquet() and losing the vector names.
This seems to be related to #42220 and #51695.
> sessionInfo()
R version 4.6.1 (2026-06-24)
Platform: x86_64-pc-linux-gnu
Running under: Ubuntu 24.04.5 LTS
Matrix products: default
BLAS: /usr/lib/x86_64-linux-gnu/blas/libblas.so.3.12.0
LAPACK: /usr/lib/x86_64-linux-gnu/lapack/liblapack.so.3.12.0 LAPACK version 3.12.0
locale:
[1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C LC_TIME=en_US.UTF-8
[4] LC_COLLATE=en_US.UTF-8 LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8
[7] LC_PAPER=en_US.UTF-8 LC_NAME=C LC_ADDRESS=C
[10] LC_TELEPHONE=C LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
time zone: America/Toronto
tzcode source: system (glibc)
attached base packages:
[1] stats graphics grDevices utils datasets methods base
loaded via a namespace (and not attached):
[1] digest_0.6.39 R6_2.6.1 fastmap_1.2.0 xfun_0.57 magrittr_2.0.5
[6] glue_1.8.1 tibble_3.3.1 knitr_1.51 pkgconfig_2.0.3 htmltools_0.5.9
[11] rmarkdown_2.31 lifecycle_1.0.5 ps_1.9.3 cli_3.6.6 processx_3.9.0
[16] callr_3.7.6 vctrs_0.7.3 reprex_2.1.1 withr_3.0.2 compiler_4.6.1
[21] rstudioapi_0.18.0 tools_4.6.1 clipr_0.8.0 pillar_1.11.1 evaluate_1.0.5
[26] yaml_2.3.12 otel_0.2.0 rlang_1.2.0 fs_2.1.0 Component(s)
R
Activity
marberts commentedon Oct 20, 2026
Upon closer inspection of
R/metadata.R, settingoptions(arrow.preserve_row_level_metadata = TRUE)gives the behavior documented in?Schema, although the news from version 6.0.0 notes the following.