TypeError with joblib while checking "__future__" in code

Environment data

debugpy version: 1.8.13
OS and version: Ubuntu 24.04
Python version (& distribution if applicable, e.g. Anaconda): 3.12
Using VS Code or Visual Studio: VS Code

Actual behavior

Using joblib (1.5.1) debugpy prints

Traceback (most recent call last):
  File "/root/.vscode-server/extensions/ms-python.debugpy-2025.9.2025052001/bundled/libs/debugpy/_vendored/pydevd/_pydev_bundle/pydev_monkey.py", line 543, in patch_args
    new_args.append(_get_python_c_args(host, port, code, unquoted_args, SetupHolder.setup))
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.vscode-server/extensions/ms-python.debugpy-2025.9.2025052001/bundled/libs/debugpy/_vendored/pydevd/_pydev_bundle/pydev_monkey.py", line 195, in _get_python_c_args
    if "__future__" in code:
       ^^^^^^^^^^^^^^^^^^^^
TypeError: a bytes-like object is required, not 'str'

Expected behavior

Checking if code is byte

Steps to reproduce:

launch.json

{
    "version": "0.2.0",
    "configurations": [
        {
            "name": "Worker",
            "type": "debugpy",
            "request": "launch",
            "module": "app.worker",
            "subProcess": true,  // with this to false we wont debug the subprocess
            "justMyCode": true
        }
    ]
}

app/worker.py

from joblib import Parallel, delayed

def test():
    return 42
Parallel(n_jobs=10)(delayed(test)() for _ in range(10))

Tested monkey patch:

@@ -190,7 +190,8 @@
     setup_repr = setup if setup is None else (sorted_dict_repr(setup))
 
     future_imports = ""
-    if "__future__" in code:
+    future_str = b'__future__' if isinstance(code, bytes) else '__future__'
+    if future_str in code:
         # If the code has a __future__ import, we need to be able to strip the __future__
         # imports from the code and add them to the start of our code snippet.
         future_imports, code = _separate_future_imports(code)

But with this I receive a deadlock

/tmp/venv/lib/python3.12/site-packages/joblib/externals/loky/backend/resource_tracker.py:120: UserWarning: resource_tracker: process died unexpectedly, relaunching.  Some folders/sempahores might leak.
  warnings.warn(

Can you try this with 3.11? joblib may be generating their own byte code, making debugpy not able to patch their source for debugging. In 3.11 there's a different method of debugging so that might work.

Rich Chiodo (@rchiodo)

Hi Rich Chiodo (@rchiodo)
Exactly same problem.
If I add a dummy print(code) the output is:

b'from joblib.externals.loky.backend.resource_tracker import main; main(4, False)'
0.61s - Error patching args (debugger not attached to subprocess).
Traceback (most recent call last):
  File "/root/.vscode-server/extensions/ms-python.debugpy-2025.8.0-linux-x64/bundled/libs/debugpy/_vendored/pydevd/_pydev_bundle/pydev_monkey.py", line 542, in patch_args
    new_args.append(_get_python_c_args(host, port, code, unquoted_args, SetupHolder.setup))
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.vscode-server/extensions/ms-python.debugpy-2025.8.0-linux-x64/bundled/libs/debugpy/_vendored/pydevd/_pydev_bundle/pydev_monkey.py", line 194, in _get_python_c_args
    if "__future__" in code:
       ^^^^^^^^^^^^^^^^^^^^
TypeError: a bytes-like object is required, not 'str'
from multiprocessing.resource_tracker import main;main(11)

After that, like before, most of the time goes in deadlock (you can see on the callstack panel a pid spawning and despawning fast).
But the debugging itself works

Rich Chiodo (@rchiodo)

Hello Rich Chiodo (@rchiodo),

I investigated a related scikit-learn callback failure from scikit-learn/scikit-learn#33710 and I tried to reduced the lower level failure to direct joblib/loky usage, without requiring scikit-learn

The smaller reproducer is directly the joblib.externals.loky.backend.get_context().Manager() on debugpy subprocess patching

Environment

OS: macOS arm64
Python: 3.14.3
debugpy: 1.8.20
joblib: 1.5.3
scikit-learn: local callbacks branch, only used as context for [Callbacks] VS Code debugger fails with EOFError during Callback instance creation scikit-learn/scikit-learn#33710

Minimal direct-loky reproducer

from joblib.externals.loky.backend import get_context

manager = get_context().Manager()
values = manager.list()
values.append("ok")
print(list(values))
manager.shutdown()

results matrix

From a clean rerun with fresh logs/output directories:

case	result
plain Python, direct loky `Manager()`	OK
debugpy with `--configure-subProcess false`, direct loky `Manager()`	OK
debugpy default subprocess behavior, direct loky `Manager()`	FAIL
debugpy with `--configure-subProcess true`, direct loky `Manager()`	FAIL
plain Python, `joblib.Parallel(...)` reproducer from this issue	OK
debugpy with `--configure-subProcess false`, `joblib.Parallel(...)`	OK
debugpy default subprocess behavior, `joblib.Parallel(...)`	FAIL
debugpy with `--configure-subProcess true`, `joblib.Parallel(...)`	FAIL
plain stdlib multiprocessing managers	OK
debugpy default stdlib multiprocessing managers	exit 0, but logs contain debugpy traceback evidence

So the hard failure was not specific to scikit-learn. The smallest failing surface I found that it was direct joblib.externals.loky.backend.get_context().Manager() under debugpy subprocess patching

Parent side

For the direct loky reproducer under default debugpy, the parent process fails with:

Traceback (most recent call last):
  File "repro_loky_manager.py", line 15, in main
    manager = get_context().Manager()
  File ".../python3.14/multiprocessing/context.py", line 57, in Manager
    m.start()
  File ".../python3.14/multiprocessing/managers.py", line 570, in start
    self._address = reader.recv()
  File ".../python3.14/multiprocessing/connection.py", line 256, in recv
    buf = self._recv_bytes()
  File ".../python3.14/multiprocessing/connection.py", line 447, in _recv_bytes
    buf = self._recv(4)
  File ".../python3.14/multiprocessing/connection.py", line 416, in _recv
    raise EOFError
EOFError

My interpretation is that the EOFError is a secondary symptom: the manager/helper child exits before sending its address back to the parent

debugpy log evidence

The debugpy logs contain the same kind of error as reported here:

File ".../debugpy/_vendored/pydevd/_pydev_bundle/pydev_monkey.py", line 193, in _get_python_c_args
    if "__future__" in code:
       ^^^^^^^^^^^^^^^^^^^^
TypeError: a bytes-like object is required, not 'str'

the same logs show subprocess patching for loky/joblib helper processes with bytes argv values. for instance:

Patching args: [b'.../bin/python', b'-c', b'from joblib.externals.loky.backend.resource_tracker import main; main(6, False)']

and loky worker bootstrap commands such as:

Patching args: [b'.../bin/python', b'-m', b'joblib.externals.loky.backend.popen_loky_posix', b'--process-name', b'LokyProcess-1', b'--pipe', b'23']

There is also a stdlib resource tracker patching line in the logs:

Patching args: [b'.../bin/python', '-c', 'from multiprocessing.resource_tracker import main;main(20)']

Is it debugpy centered?

The results matrix (above) suggests this is tied to debugpy subprocess patching rather than plain loky or plain multiprocessing:

direct loky Manager() passes in plain Python;
direct loky Manager() passes with --configure-subProcess false;
direct loky Manager() fails with default subprocess behavior;
direct loky Manager() fails with --configure-subProcess true;
the joblib.Parallel(...) reproducer from this issue follows the same pass/fail pattern;
the failing logs show pydev_monkey.py bytes/str errors while patching loky/joblib helper process arguments

Does this look like the same underlying issue as the pydev_monkey.patch_args bytes/str error already discussed here?

I’m not sure what the best fix would be from the debugpy side. Would it make sense for debugpy to handle these bytes based subprocess arguments, or is there something special about loky’s helper processes that should be treated differently?

I don't get how "code" in that call stack isn't a string. It must be None then. I think the simple thing would be check for None first and see if it fixes the problem. You can test it out with your reproduction.

Rich Chiodo (@rchiodo)

Hello Rich Chiodo (@rchiodo), I checked for None and unfortunately it does not fix the problem.

I tested it locally by temporarily patching the installed (vendored) pydev_monkey.py in the repro virtualenv to add diagnostics around _get_python_c_args() and this guard:

if code is None:
    code = ""

then I reran on the joblib/loky reproducer and restored the original file afterward, finally, I saw that:

in the failing loky resource tracker subprocess patching path the code was observed as bytes
the code is None guard was never taken.
the direct loky manager repro still failed after adding that guard
the parent process still saw the same EOFError from multiprocessing.managers.Manager.start() / connection.recv()

Just a short diagnotisc snippet:

type(code)=<class 'bytes'> repr(code)=b'from joblib.externals.loky.backend.resource_tracker import main; main(8, False)' code_is_none=False code_is_bytes=True code_is_str=False

obs.: using the same environment as before

OS: macOS arm64

Python: 3.14.3

debugpy: 1.8.20

joblib: 1.5.3

Okay thanks. I must have missed something when looking at the callers. We probably just need to handle the bytes case as well then.

Rich Chiodo (@rchiodo)

Hello Rich Chiodo (@rchiodo), I tested the bytes handling direction in the same enviromnent. So, I applied a small local patch to pydev_monkey.py to decode code when it is bytes, before the "__future__" in code check.

The patch I tested was essentially:

if isinstance(code, bytes):
    try:
        code = code.decode(sys.getfilesystemencoding(), errors="surrogateescape")
    except Exception:
        code = code.decode("utf-8", errors="surrogateescape")

Actually this does remove the original pydev_monkey.py bytes/str TypeError, but, it does not fix everything yet because I stlil got the some fails.

direct loky Manager() under default debugpy subprocess
direct loky Manager() with --configure-subProcess true
the joblib.Parallel() repro under default debugpy subprocess handling
the joblib.Parallel() repro with --configure-subProcess true

The other visible failure is still the parent EOFError from:

multiprocessing.managers.Manager.start()
self._address = reader.recv()

and the joblib.Parallel() repro still ends in worker termination

So it looks like handling bytes in _get_python_c_args() fixes the first visible error, but the problem is there is still another subprocess/bootstrap failure on Python 3.14.3

Jhonatan Ramos Felix (@JhonatanFelix)

Jhonatan Ramos Felix (@JhonatanFelix), can you try the fix outlined here?
#2021

That fixes it for me but wanted to make sure it works for you.

Rich Chiodo (@rchiodo)

Confirmed! This fixes the failing matrix on my original repro environment as well. Thank you a lot Rich Chiodo (@rchiodo)!

Navigation Menu

Repository navigation

TypeError with joblib while checking "future" in code #1905

Environment data

Actual behavior

Expected behavior

Steps to reproduce:

Environment

Minimal direct-loky reproducer

results matrix

Parent side

debugpy log evidence

Is it debugpy centered?

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

TypeError with joblib while checking "__future__" in code #1905

Description

Environment data

Actual behavior

Expected behavior

Steps to reproduce:

Activity

rchiodo commented on May 30, 2025

FedericoHeichou commented on Jun 3, 2025

JhonatanFelix commented on Apr 28, 2026

Environment

Minimal direct-loky reproducer

results matrix

Parent side

debugpy log evidence

Is it debugpy centered?

rchiodo commented on Apr 28, 2026

JhonatanFelix commented on Apr 29, 2026

rchiodo commented on Apr 29, 2026

JhonatanFelix commented on Apr 30, 2026

rchiodo commented on May 1, 2026

JhonatanFelix commented on May 2, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

TypeError with joblib while checking "future" in code #1905