Intermittent leaked worker connections when a future errors

I've been having issues running furrr's tests lately, where I see a lot of this:

Warning message:
In .Internal(gc(verbose, reset, full)) :
  closing unused connection 4 (<-localhost:11913)

If you just go run devtools::test() in furrr, you should see that most of the time. (no longer, i disabled the problematic tests for now futureverse/furrr#308)

I've tracked this down to a minimal-ish future only issue:

library(future)

fn <- function() {
  futures <- vector("list", length = 10)

  for (i in 1:10) {
    futures[[i]] <- future(
      expr = stop("oh no")
    )
  }

  value(futures)
}

# You probably have to run this a few times
plan("multisession", workers = 2)
tryCatch(fn(), error = function(cnd) {})
plan(sequential)
gc()

Alternatively you can run fn() in a loop like this, with warn = 1 so you see warnings immediately

# Alternatively:
options(warn = 1)
for (i in 1:100) {
  plan("multisession", workers = 2)
  tryCatch(fn(), error = function(cnd) {})
  plan(sequential)
  gc()
}

This leak is intermittent, which is probably going to make this hard to track down.

It looks like this:

test1.mov

test2.mov

This is not great!

There are a few things that seem to be required to make this appear:

The future() must error, i.e. it must call stop("oh no") or something similar
I think you need to use plan("multisession", workers = 2). At least, you need a few workers.
I needed to use 10 futures to get it to reproduce pretty reliably. With just 2 futures it did still occur, but way less frequently.

This comes up on the furrr side with tests that look like this:

furrr_test_that("unused components can be absorbed", {
  x <- list(c(1, 2), c(3, 5))

  fn1 <- function(x) {
    x
  }

  expect_error({
    future_pmap_dbl(x, fn1)
  })
})

furrr_test_that() causes the test to be run across different plan() styles, including multisession with 2 workers. And this test is expecting an error, i.e. the future_pmap_dbl() call will error when trying to run fn1 on the worker. Then expect_error() acts as my tryCatch() in my minimal reprex.

This feels like a medium to high ish severity issue? Not being able to shut down workers seems quite bad!

Also, here's a cool anecdote.

I was having trouble isolating the furrr tests where this was an issue, because all I was getting was this:

testthat has a cool feature called set_state_inspector() which lets you check global state before and after each test to ensure it hasn't changed.

I added this to tests/testthat/helper-state.R

set_state_inspector(function() {
  getAllConnections()
})

and on rerun I got:

All 3 of these are cases of using furrr_test_that() with expect_error()!

I'll probably keep this state inspector on permanently in furrr to check for leaks constantly.

This is not great!

This should be a harmless warning. This happens because one or more 'multisession' or 'cluster' futures are cancelled.

Here's a smaller example, illustrating what's going on. Start by setting up a 'multisession' backend with one worker. We look at the details for this worker;

library(future)
plan(multisession, workers = I(1))

getAllConnections()
#> [1] 0 1 2 4
plan("backend")$workers[[1]]
#> RichSOCKnode of a socket cluster on local host 'localhost' with pid 2181497
#> (R version 4.5.2 (2025-10-31), x86_64-pc-linux-gnu) using socket connection
#> #4 ('<-localhost:11825')
str(plan("backend")$workers[[1]]$con)
showConnections()
#>   description         class      mode  text     isopen   can read can write
#> 4 "<-localhost:11825" "sockconn" "a+b" "binary" "opened" "yes"    "yes"

Create a future and immediately cancel it:

f1 <- future(42)
f1 <- cancel(f1)

Cancelling a future ends up signalling SIGTERM to the corresponding worker using parallelly::killNode(). This results in the worker being terminated, which we can see from:

parallelly::isNodeAlive(plan("backend")$workers[[1]])
#> [1] FALSE

We can also see that the PSOCK connection to this worker is no longer valid because of this:

getAllConnections()
#> [1] 0 1 2
plan("backend")$workers[[1]]
#> RichSOCKnode of a socket cluster on local host 'localhost' with pid 2181497
#> (R version 4.5.2 (2025-10-31), x86_64-pc-linux-gnu) using socket connection
#> #4 ('ERROR: invalid connection')
str(plan("backend")$workers[[1]]$con)
#> 'sockconn' int 4
#> - attr(*, "conn_id")=<externalptr> 
showConnections()
#>      description class mode text isopen can read can write

Note how we're still holding on to that connection internally.

Next, let's create a new future:

f2 <- future(42)
#> Warning in value[[3L]](cond) :
#>   [FUTURE INTERRUPTED]: Caught FutureCanceledError with error message:
#>   Future (<unnamed-1>) of class MultisessionFuture was canceled (pid 2181497)
#>   [future <unnamed-1> (afb150cb1f3e5eb2d29d065b6c89495c-1); on
#>   afb150cb1f3e5eb2d29d065b6c89495c@hb-x1-2023<2181451> at 2026-02-06
#>   17:26:02.720798]

This warning happens because the future backend tries to free up the single worker available. When it does so, it notices that the future was cancelled. When this happens, it also relaunches the worker (using parallelly::cloneNode()), i.e. it creates a new PSOCK worker. We can see this from:

parallelly::isNodeAlive(plan("backend")$workers[[1]])
#> [1] TRUE

and

plan("backend")$workers[[1]]
#> RichSOCKnode of a socket cluster on local host 'localhost' with pid 2182077
#> (R version 4.5.2 (2025-10-31), x86_64-pc-linux-gnu) using socket connection
#> #4 ('<-localhost:11825')

Notice how the worker has a different process ID, i.e. it's a new process. The connection index is reused;

str(plan("backend")$workers[[1]]$con)
#> 'sockconn' int 4
#> - attr(*, "conn_id")=<externalptr> 

showConnections()
#>   description         class      mode  text     isopen   can read can write
#> 4 "<-localhost:11825" "sockconn" "a+b" "binary" "opened" "yes"    "yes"    
#> Warning message:
#> In .Internal(gc(verbose, reset, full)) :
#>   closing unused connection 3 (<-localhost:11825)

... and there's that warning you saw, which is triggered by the garbage collector (*)

(*) I'm not sure why it reports on connection with index 3, and not index 4.

Yes, it's an annoying warning, but it should be harmless. I'll see if I can try to muffle it, e.g. by forcing the garbage collector to run whenever a new PSOCK worker is cloned.

Trying to track down the exact details, so thinking out loud here.

Gives warning

> cl <- parallelly::makeClusterPSOCK(1L)
> str(cl[[1]]$con)
 'sockconn' int 4
 - attr(*, "conn_id")=<externalptr> 
> cl[[1]] <- parallelly::cloneNode(cl[[1]])
> str(cl[[1]]$con)
 'sockconn' int 3
 - attr(*, "conn_id")=<externalptr> 
> void <- gc()
Warning message:
In .Internal(gc(verbose, reset, full)) :
  closing unused connection 4 (<-localhost:11861)

No warning

cl <- parallelly::makeClusterPSOCK(1L)
str(cl[[1]]$con)
#> 'sockconn' int 4
#> - attr(*, "conn_id")=<externalptr> 
void <- parallelly::killNode(cl[[1]])
cl[[1]] <- parallelly::cloneNode(cl[[1]])
str(cl[[1]]$con)
#> 'sockconn' int 3
#> - attr(*, "conn_id")=<externalptr> 
void <- gc()

Fix random warning on garbage collector closing connections when canc…

Small update: I've manage to track this down to cluster futures relaunching a terminated worker twice, which results in the first relaunch will be discarded immediately, resulting in that stray connection. I'm still puzzled by how this can happen, but I'll get to it.

Fix random warning on garbage collector closing connections when canc…

I've manage to track this down to cluster futures relaunching a terminated worker twice, which results in the first relaunch will be discarded immediately, resulting in that stray connection.

This took some time, but I found the mistake (commit 7e034af). Fixed now:

> library(future); plan(multisession, workers = I(1)); f <- future("a"); cancel(f)
> f <- future("b")
Warning in value[[3L]](cond) :
  [FUTURE INTERRUPTED]: Caught FutureCanceledError with error message: Future (<unnamed-1>) of class MultisessionFuture was canceled (pid 2668176) [future <unnamed-1> (061c24bc40616f0481e9e095a21f4732-1); on 061c24bc40616f0481e9e095a21f4732@hb-x1-2023<2668127> at 2026-02-07 22:33:35.674117]
Calls: future -> run -> run.Future -> tryCatch -> tryCatchList -> tryCatchOne -> doTryCatch -> tryCatchList -> tryCatchOne -> doTryCatch -> launchFuture -> launchFuture.ClusterFutureBackend -> requestNode -> await -> FutureRegistry -> collectValues -> tryCatch -> tryCatchList -> tryCatchOne -> doTryCatch -> tryCatchList -> tryCatchOne -> <Anonymous>
TRACKER: 1 connection opened => Now 1 open connection
> gc()
          used (Mb) gc trigger (Mb) max used (Mb)
Ncells  639092 34.2    1724565 92.2  1724565 92.2
Vcells 1410298 10.8    8388608 64.0  8388553 64.0

Awesome thank you!

Navigation Menu

Repository navigation

Intermittent leaked worker connections when a future errors #820

Gives warning

No warning

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Intermittent leaked worker connections when a future errors #820

Description

Activity

DavisVaughan commented on Jan 30, 2026

HenrikBengtsson commented on Feb 7, 2026

HenrikBengtsson commented on Feb 7, 2026

Gives warning

No warning

HenrikBengtsson commented on Feb 7, 2026

HenrikBengtsson commented on Feb 8, 2026

DavisVaughan commented on Feb 8, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions