Skip to content

double free or corruption when enable python.IsolateNativeModules #563

@suiwenfeng

Description

@suiwenfeng

How to reproduce?

> environment

container image: ghcr.io/graalvm/graalpy-community:25
jdk: graalvm-jdk-25_linux-x64_bin.tar.gz

> python venv deps installed

pip install torch transformers patchelf

> multi context in maven repo(not packaged as native-image)

// Create a new GraalVM Python context and managed by org.apache.commons.pool2.impl.GenericObjectPool
Context context = GraalPyResources
        .contextBuilder(Path.of("python-resources"))
        .option("python.IsolateNativeModules", "true")
        .allowExperimentalOptions(true)
        .allowIO(IOAccess.ALL)
        .allowCreateThread(true)
        .allowCreateProcess(true)
        .allowHostAccess(HostAccess.ALL)
        .allowAllAccess(true)
        .build();
// use contexts in parallel, scriptPool has 3 context above
String scriptText = ""
        + "import torch\n"
        + "import transformers\n"
        + "\n"
        + "def main():\n"
        + "    print(f'Torch version: {torch.__version__}, Transformers version: {transformers.__version__}')\n";
PythonScriptPool scriptPool = new PythonScriptPool("testScript", scriptText, 3);
java.util.stream.Stream.iterate(0, n -> n + 1).limit(10).parallel().forEach(n -> {
    try {
        scriptPool.executeMethod("main", new Object[] {});
    } catch (Throwable e) {
        e.printStackTrace();
        System.exit(1);
    }
});

> error log

18:18:13  #14 0.761 WARNING: A restricted method in java.lang.System has been called
18:18:13  #14 0.761 WARNING: java.lang.System::load has been called by com.oracle.truffle.polyglot.JDKSupport in an unnamed module (file:/app/repo/target/bb-1.0-SNAPSHOT.jar)
18:18:13  #14 0.761 WARNING: Use --enable-native-access=ALL-UNNAMED to avoid a warning for callers in this module
18:18:13  #14 0.761 WARNING: Restricted methods will be blocked in a future release unless native access is enabled
18:18:13  #14 0.761 
18:18:13  #14 1.081 WARNING: A terminally deprecated method in sun.misc.Unsafe has been called
18:18:13  #14 1.081 WARNING: sun.misc.Unsafe::objectFieldOffset has been called by com.oracle.truffle.runtime.hotspot.HotSpotTruffleRuntime (file:/app/repo/target/bb-1.0-SNAPSHOT.jar)
18:18:13  #14 1.081 WARNING: Please consider reporting this to the maintainers of class com.oracle.truffle.runtime.hotspot.HotSpotTruffleRuntime
18:18:13  #14 1.081 WARNING: sun.misc.Unsafe::objectFieldOffset will be removed in a future release
18:18:21  #14 8.482 [To redirect Truffle log output to a file use one of the following options:
18:18:21  #14 8.482 * '--log.file=<path>' if the option is passed using a guest language launcher.
18:18:21  #14 8.482 * '-Dpolyglot.log.file=<path>' if the option is passed using the host Java launcher.
18:18:21  #14 8.482 * Configure logging using the polyglot embedding API.]
18:18:21  #14 8.482 [To redirect Truffle log output to a file use one of the following options:
18:18:21  #14 8.482 * '--log.file=<path>' if the option is passed using a guest language launcher.
18:18:21  #14 8.482 * '-Dpolyglot.log.file=<path>' if the option is passed using the host Java launcher.
18:18:21  #14 8.482 * Configure logging using the polyglot embedding API.]
18:18:21  #14 8.482 [To redirect Truffle log output to a file use one of the following options:
18:18:21  #14 8.482 * '--log.file=<path>' if the option is passed using a guest language launcher.
18:18:21  #14 8.482 * '-Dpolyglot.log.file=<path>' if the option is passed using the host Java launcher.
18:18:21  #14 8.482 * Configure logging using the polyglot embedding API.]
18:18:21  #14 8.482 [python::CApiContext] WARNING: Loading C extension module torch._C from '/app/repo/python-resources/venv/lib/python3.12/site-packages/torch/_C.graalpy250-312-native-x86_64-linux.so'. Support for the Python C API is considered experimental. See https://www.graalvm.org/latest/reference-manual/python/Native-Extensions/#embedding-limitations for the limitations. You can suppress this warning by setting the context option 'python.WarnExperimentalFeatures' to 'false'.
18:18:21  #14 8.482 [python::CApiContext] WARNING: Loading C extension module torch._C from '/app/repo/python-resources/venv/lib/python3.12/site-packages/torch/_C.graalpy250-312-native-x86_64-linux.so'. Support for the Python C API is considered experimental. See https://www.graalvm.org/latest/reference-manual/python/Native-Extensions/#embedding-limitations for the limitations. You can suppress this warning by setting the context option 'python.WarnExperimentalFeatures' to 'false'.
18:18:21  #14 8.482 [python::CApiContext] WARNING: Loading C extension module torch._C from '/app/repo/python-resources/venv/lib/python3.12/site-packages/torch/_C.graalpy250-312-native-x86_64-linux.so'. Support for the Python C API is considered experimental. See https://www.graalvm.org/latest/reference-manual/python/Native-Extensions/#embedding-limitations for the limitations. You can suppress this warning by setting the context option 'python.WarnExperimentalFeatures' to 'false'.
18:18:21  #14 8.806 double free or corruption (out)
18:18:21  #14 8.806 double free or corruption (!prev)
18:18:21  #14 8.806 #
18:18:21  #14 8.806 # A fatal error has been detected by the Java Runtime Environment:
18:18:21  #14 8.806 #
18:18:21  #14 8.806 #  SIGSEGV (0xb) at pc=0x00007f24b9d9a997, pid=1, tid=52
18:18:21  #14 8.806 #
18:18:21  #14 8.806 # JRE version: Java(TM) SE Runtime Environment Oracle GraalVM 25.0.1+8.1 (25.0.1+8) (build 25.0.1+8-LTS-jvmci-b01)
18:18:21  #14 8.806 # Java VM: Java HotSpot(TM) 64-Bit Server VM Oracle GraalVM 25.0.1+8.1 (25.0.1+8-LTS-jvmci-b01, mixed mode, sharing, tiered, jvmci, jvmci compiler, compressed oops, compressed class ptrs, g1 gc, linux-amd64)
18:18:21  #14 8.806 # Problematic frame:
18:18:21  #14 8.806 # C  [libc.so.6+0x28997]  abort+0x174
18:18:21  #14 8.806 #
18:18:21  #14 8.806 # Core dump will be written. Default location: /app/repo/core.1
18:18:21  #14 8.806 #
18:18:21  #14 8.807 # An error report file with more information is saved as:
18:18:21  #14 8.807 # /app/repo/hs_err_pid1.log
18:18:21  #14 8.835 [8.718s][warning][os] Loading hsdis library failed
18:18:53  #14 39.29 #
18:18:53  #14 39.29 # If you would like to submit a bug report, please visit:
18:18:53  #14 39.29 #   https://bugreport.java.com/bugreport/crash.jsp
18:18:53  #14 39.29 # The crash happened outside the Java Virtual Machine in native code.
18:18:53  #14 39.29 # See problematic frame for where to report the bug.
18:18:53  #14 39.29 #

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions