A runtime in a disposable class loader can't be garbage collected after tearDown #9904

Clone of #9781 for 10.1.

Environment Information

• JRuby 10.1.2.0, 9.4.15.0 and 9.4.14.0 (jruby-complete from Maven Central), embedded with Ruby.newInstance(), no flags or JRUBY_OPTS
• OpenJDK 21.0.10 (BellSoft Liberica), Linux 7.0.0 x86_64
• Found through AsciidoctorJ 3.0.1 in the Asciidoctor Gradle plugin, which converts documents in Gradle workers with class loader isolation. Each worker gets a new class loader with JRuby on it, inside the long-lived Gradle daemon.

Expected Behavior

When a runtime is created in its own class loader, used, torn down with tearDown(), and the class loader is dropped, the class loader and the runtime can be garbage collected.

This program does that four times, and checks after each round how many of the class loaders are still alive:

import java.lang.management.ManagementFactory;
import java.lang.management.MemoryPoolMXBean;
import java.lang.ref.WeakReference;
import java.net.URL;
import java.net.URLClassLoader;
import java.nio.file.Path;
import java.util.ArrayList;
import java.util.List;

public class RuntimeLeak {

    public static void main(String[] args) throws Exception {
        URL[] jruby = {Path.of(args[0]).toUri().toURL()};
        List<WeakReference<ClassLoader>> loaders = new ArrayList<>();
        for (int round = 1; round <= 4; round++) {
            loaders.add(new WeakReference<>(runOnce(jruby)));
            for (int i = 0; i < 5; i++) {
                System.gc();
                Thread.sleep(200);
            }
            long alive = loaders.stream().filter(r -> r.get() != null).count();
            System.out.printf("round %d: class loaders still alive: %d of %d, class space used: %d MB%n",
                round, alive, round, classSpaceUsed() / (1024 * 1024));
        }
    }

    static ClassLoader runOnce(URL[] jruby) throws Exception {
        URLClassLoader loader = new URLClassLoader(jruby, ClassLoader.getPlatformClassLoader());
        Class<?> rubyClass = loader.loadClass("org.jruby.Ruby");
        Object runtime = rubyClass.getMethod("newInstance").invoke(null);
        rubyClass.getMethod("evalScriptlet", String.class)
            .invoke(runtime, "java.lang.StringBuilder.new('jruby').reverse.to_s.upcase");
        rubyClass.getMethod("tearDown").invoke(runtime);
        loader.close();
        return loader;
    }

    static long classSpaceUsed() {
        for (MemoryPoolMXBean pool : ManagementFactory.getMemoryPoolMXBeans()) {
            if (pool.getName().equals("Compressed Class Space")) {
                return pool.getUsage().getUsed();
            }
        }
        return -1;
    }
}

Run it with java RuntimeLeak.java jruby-complete-10.1.2.0.jar.

Actual Behavior

None of the class loaders are collected, and each runtime keeps about 5 MB of class space:

round 1: class loaders still alive: 1 of 1, class space used: 6 MB
round 2: class loaders still alive: 2 of 2, class space used: 11 MB
round 3: class loaders still alive: 3 of 3, class space used: 16 MB
round 4: class loaders still alive: 4 of 4, class space used: 21 MB

9.4.15.0 and 9.4.14.0 give the same result. In a long-lived JVM that keeps creating runtimes this way, the class space and heap grow until the JVM runs out of them. In the Gradle daemon, a documentation build with 20 source directories, so 20 runtimes, added about 100 MB of class space per run. After a few runs the daemon stopped itself because the garbage collector was thrashing.

I looked at heap dumps with Eclipse MAT, following paths to GC roots without weak and soft references, and at the reference chains of Java Flight Recorder's old object samples. Heap dumps don't contain Class.classValueMap, so for ClassValues I also read the class value maps of all loaded classes through Instrumentation. These references keep the class loader alive:

1. jffi's Cleaner thread. com.kenai.jffi.internal.Cleaner creates a java.lang.ref.Cleaner per class loader, and its thread keeps the registered cleanup actions, such as com.kenai.jffi.CallContext$1, whose class is in the runtime's class loader:

   java.net.URLClassLoader
   <- <classloader> class com.kenai.jffi.CallContext$1
   <- <class> com.kenai.jffi.CallContext$1
   <- action jdk.internal.ref.CleanerImpl$PhantomCleanableRef
   <- next jdk.internal.ref.CleanerImpl$PhantomCleanableRef
   <- phantomCleanableList jdk.internal.ref.CleanerImpl
   <- <Java Local> jdk.internal.misc.InnocuousThread "Cleaner-0"
   

   This is the shortest path with 10.1.2.0 and default settings. -Djruby.native.enabled=false removes it.

2. Values left in thread locals of the threads that used the runtime. Ruby.DEDUP_WRAPPER_CACHE, and LibrarySearcher's keyWrapper and rangeWrapper, are per-runtime ThreadLocals created with withInitial:

   jruby/core/src/main/java/org/jruby/Ruby.java

   Line 5120 in 09cff75

   private final ThreadLocal<FStringEqual> DEDUP_WRAPPER_CACHE = ThreadLocal.withInitial(FStringEqual::new);

   jruby/core/src/main/java/org/jruby/runtime/load/LibrarySearcher.java

   Lines 533 to 534 in 09cff75

   	private final ThreadLocal<StringWrapper> keyWrapper = ThreadLocal.withInitial(() -> new StringWrapper(null, 0, 0));
   	private final ThreadLocal<StringWrapper> rangeWrapper = ThreadLocal.withInitial(() -> new StringWrapper(null, 0, 0));

   Nothing removes their values from those threads when the runtime is torn down. freezeAndDedupString only removes the FStringEqual wrapper when it caches a new string, so after a lookup that finds an existing one, the wrapper stays on the thread:

   jruby/core/src/main/java/org/jruby/Ruby.java

   Lines 4961 to 4986 in 09cff75

   	public RubyString freezeAndDedupString(RubyString string) {
   	var context = getCurrentContext();
   	if (!string.isBare(context)) {
   	// never cache a non-natural String
   	string.setFrozen(true);
   	return string;
   	}
   	// Populate thread-local wrapper
   	try (FStringEqual wrapper = FStringEqual.cached(this, string)) {
   	WeakReference<RubyString.FString> dedupedRef = dedupMap.get(wrapper);
   	RubyString.FString deduped;
   	if (dedupedRef == null || (deduped = dedupedRef.get()) == null) {
   	// We will insert wrapper one way or another so clear from threadlocal
   	DEDUP_WRAPPER_CACHE.remove();
   	// Ensure we have an FString
   	deduped = string.dupAsFString(this);
   	return cacheFString(deduped);
   	}
   	return deduped;
   	}
   	}

   The map entries reference their keys only weakly, but the values are instances of FStringEqual and StringWrapper, whose classes are in the runtime's class loader. A ThreadLocalMap only drops such an entry when a later access to the same thread's map runs into it after the key has been collected. Until then, the value keeps the class loader alive. With 10.1.2.0, MAT shows this path for the last runtime of the program:

   java.net.URLClassLoader
   <- <classloader> class org.jruby.runtime.load.LibrarySearcher$StringWrapper
   <- <class> org.jruby.runtime.load.LibrarySearcher$StringWrapper
   <- value java.lang.ThreadLocal$ThreadLocalMap$Entry
   <- table java.lang.ThreadLocal$ThreadLocalMap
   <- threadLocals java.lang.Thread "main"
   

   With 9.4.14.0 and AsciidoctorJ, it was a org.jruby.Ruby$FStringEqual on the same kind of path. Thread pools, like Gradle's worker threads, keep these threads and their maps for a long time.

3. MethodGatherer's static ClassValues. FILTERED_DECLARED_METHODS and FILTERED_METHODS are java.lang.ClassValues whose values are MethodGatherer$PartitionedMethods, a JRuby class:

   jruby/core/src/main/java/org/jruby/javasupport/binding/MethodGatherer.java

   Lines 307 to 328 in 09cff75

   	private static final ClassValue<PartitionedMethods> FILTERED_DECLARED_METHODS = new ClassValue<PartitionedMethods>() {
   	@Override
   	public PartitionedMethods computeValue(Class cls) {
   	return new PartitionedMethods(DECLARED_METHODS.get(cls));
   	}
   	};
   	private static final ClassValue<Method[]> METHODS = new ClassValue<Method[]>() {
   	@Override
   	public Method[] computeValue(Class cls) {
   	try {
   	return cls.getMethods();
   	} catch (SecurityException se) {
   	return EMPTY_METHODS;
   	}
   	}
   	};
   	private static final ClassValue<PartitionedMethods> FILTERED_METHODS = new ClassValue<PartitionedMethods>() {
   	@Override
   	public PartitionedMethods computeValue(Class cls) {
   	return new PartitionedMethods(METHODS.get(cls));
   	}
   	};

   A ClassValue stores its value in the class it's computed for. Creating a runtime already computes them for JDK classes, so the JDK classes, which are never unloaded, keep the PartitionedMethods values and through them the runtime's class loader. With Ruby.newInstance() and tearDown() alone, these JDK classes end up holding values for every runtime: java.lang.Object, java.lang.Class, java.lang.Enum, java.lang.Process, java.lang.ProcessBuilder, java.lang.ProcessBuilder$Redirect, java.lang.ProcessBuilder$Redirect$Type, java.lang.reflect.Method, java.lang.reflect.Field, java.lang.reflect.Constructor and java.util.Map, among others. No option avoids this one. It doesn't show up in MAT because heap dumps don't contain Class.classValueMap.

4. The Java proxy cache. With the default jruby.ji.class.values, the proxies are cached in a StableClassValue on the Java classes. With AsciidoctorJ on 9.4.14.0, Flight Recorder showed this path from a JDK class:

   org.jruby.RubyClass (runtime)
   <- result org.jruby.util.collections.StableClassValue$StableValue
   <- value java.lang.ClassValue$Entry
   <- cacheArray java.lang.ClassValue$ClassValueMap
   <- classValueMap java.lang.Class java.lang.ProcessBuilder$Redirect$Type
   

   -Djruby.ji.class.values=HARD_MAP removes it. With this program on 10.1.2.0, the class loaders are only collected with that option, see below.

5. The shutdown hook of JRubyClassLoader.getTempDir(), which it registers when it creates the temporary directory for jars inside jars. It's an instance of JRubyClassLoader$1, so java.lang.ApplicationShutdownHooks keeps that class loader until the JVM exits. Since the hook is only registered when the directory is created, this pins the first class loader that adds such a jar, not every one. This program doesn't do that; AsciidoctorJ does.

   jruby/core/src/main/java/org/jruby/util/JRubyClassLoader.java

   Lines 117 to 140 in 09cff75

   	public static synchronized File getTempDir() {
   	if (tempDir != null) return tempDir;
   	tempDir = new File(tempDir(), tempDirName());
   	if (tempDir.mkdirs()) {
   	Runtime.getRuntime().addShutdownHook(new Thread() {
   	public void run() {
   	for (File f : tempDir.listFiles()) {
   	try {
   	f.delete();
   	} catch (Exception ex) {
   	LOG.debug(ex);
   	}
   	}
   	try {
   	tempDir.delete();
   	} catch (Exception ex) {
   	LOG.info("failed to delete temp dir " + tempDir + " : " + ex);
   	}
   	}
   	});
   	}
   	return tempDir;
   	}

With roots 1 to 4 out of the way, the class loaders are collected. On 10.1.2.0 I ran the program with -Djruby.native.enabled=false and -Djruby.ji.class.values=HARD_MAP, ran each round in a new thread that ends afterwards, and then cleared, through reflection, the class value maps of the JDK classes that held MethodGatherer values. Afterwards all four class loaders were collected, and class space went from 20 MB back to 1 MB:

round 4: class loaders still alive: 4 of 4, class space used: 20 MB
after removing those values: class loaders still alive: 0 of 4, class space used: 1 MB

Without -Djruby.ji.class.values=HARD_MAP, the class loaders stayed alive after the same clearing.

#8842 is related: it's about classes leaking inside a long-lived runtime through the ClassValue caches, while this is about releasing a whole runtime together with its class loader.