Opened 2 years ago

Closed 2 years ago

Last modified 2 years ago

#33719 closed Bug (fixed)

Running tests in parallel throws exceptions.

Reported by: Pēteris Caune Owned by: Mariusz Felisiak
Component: Testing framework Version: 4.1
Severity: Release blocker Keywords:
Cc: David Smith, Nick Pope, Ahmad A. Hussein Triage Stage: Accepted
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

When I run manage.py test --parallel, I get:

Process ForkPoolWorker-387:
Traceback (most recent call last):
  File "/usr/lib/python3.10/multiprocessing/process.py", line 315, in _bootstrap
    self.run()
  File "/usr/lib/python3.10/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/lib/python3.10/multiprocessing/pool.py", line 109, in worker
    initializer(*initargs)
  File "/tmp/venv/lib/python3.10/site-packages/django/test/runner.py", line 415, in _init_worker
    with counter.get_lock():
AttributeError: 'ParallelTestSuite' object has no attribute 'get_lock'
Process ForkPoolWorker-388:
Traceback (most recent call last):
  File "/usr/lib/python3.10/multiprocessing/process.py", line 315, in _bootstrap
    self.run()
  File "/usr/lib/python3.10/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/lib/python3.10/multiprocessing/pool.py", line 109, in worker
    initializer(*initargs)
  File "/tmp/venv/lib/python3.10/site-packages/django/test/runner.py", line 415, in _init_worker
    with counter.get_lock():
AttributeError: 'ParallelTestSuite' object has no attribute 'get_lock'
(....)

How to reproduce:

  1. I created a new virtualenv and a new Django project: 
python3 -m venv venv
source venv/bin/activate
pip install --pre django
django-admin startproject woo
  1. I created *two* test files:

woo/test_something.py:

from django.test import TestCase

class SomethingTestCase(TestCase):
    def test_it_works(self):
        pass

woo/test_something_else.py:

from django.test import TestCase

class SomethingTestCase(TestCase):
    def test_it_works(self):
        pass
  1. I ran the tests with the --parallel flag: 
./manage.py test --parallel

Django version: Django==4.1a1
Python version: 3.10.4
OS: Ubuntu 22.04

Change History (9)

comment:1 by Pēteris Caune, 2 years ago

Type: Uncategorized → Bug

comment:2 by Pēteris Caune, 2 years ago

Version: 4.1 → dev

comment:3 by Mariusz Felisiak, 2 years ago

Cc: David Smith Nick Pope Ahmad A. Hussein added
Severity: Normal → Release blocker
Summary: Running tests in parallel throws exceptions in Django 4.1a1 → Running tests in parallel throws exceptions.
Triage Stage: Unreviewed → Accepted
Version: dev → 4.1

Thanks for the report.
Regression in 3b3f38b3b09b0f2373e51406ecb8c9c45d36aebc.
Reproduced at 2cec020f5bc462954d902bdad1a5955852cecff6.

comment:4 by Mariusz Felisiak, 2 years ago

It looks that initializer was change unintentionally, the following diff fixed it for me:

  • django/test/runner.py 

    diff --git a/django/test/runner.py b/django/test/runner.py
index fe30d2289b..77eb68fe65 100644
    a b class ParallelTestSuite(unittest.TestSuite):  
    496496        counter = multiprocessing.Value(ctypes.c_int, 0)
    497497        pool = multiprocessing.Pool(
    498498            processes=self.processes,
    499             initializer=self.init_worker,
     499            initializer=self.init_worker.__func__,
    500500            initargs=[
    501501                counter,
    502502                self.initial_settings,

comment:5 by Pēteris Caune, 2 years ago

The above change fixed the issue here for me as well.

comment:6 by Mariusz Felisiak, 2 years ago

Owner: changed from nobody to Mariusz Felisiak
Status: new → assigned

comment:7 by Mariusz Felisiak, 2 years ago

Has patch: set

​PR

comment:8 by GitHub <noreply@…>, 2 years ago

Resolution: → fixed
Status: assigned → closed

In 41c4cb2:

Fixed #33719 -- Fixed test command crash when running in parallel.

Thanks Pēteris Caune for the report.

Regression in 3b3f38b3b09b0f2373e51406ecb8c9c45d36aebc.

comment:9 by Mariusz Felisiak <felisiak.mariusz@…>, 2 years ago

In 2dd646e9:

[4.1.x] Fixed #33719 -- Fixed test command crash when running in parallel.

Thanks Pēteris Caune for the report.

Regression in 3b3f38b3b09b0f2373e51406ecb8c9c45d36aebc.
Backport of 41c4cb253c137edf5a96b7408ea55d57d6e0602a from main

Note: See TracTickets for help on using tickets.
Back to Top