Watchman timing out when loading server

[Jacob Green]

When starting the server on my project using ./manage.py runserver, I get the error message:

Error connecting to Watchman: timed out waiting for response, while executing ('watch-project', 'my_app')

I am using Watchman v4.9 on an Ubuntu 16.04 Docker image hosted on Mac OS X 10.14.4.

My research shows that there is a default 1 second timeout in pywatchman that Django is using. In my tests, for our particular project it takes between 1 and 2 seconds to start the initial root watchman query, and thus will always time out when started with Django.

My suggestion is that we increase the pywatchman client timeout higher than 1 second

[Jacob Green]

This is where I would consider setting a higher client timeout:
​https://github.com/django/django/blob/99cfb28e995388db109e2f2ed15307f09945dda8/django/utils/autoreload.py#L390

We set a lower client timeout here as an example:
​https://github.com/django/django/blob/99cfb28e995388db109e2f2ed15307f09945dda8/django/utils/autoreload.py#L390

[Daniel Hepper]

Jacob, just wanted to let you know that the two URLs you posted are identical. I assume that was not intentional.

[Tobias Kunze]

I had this issue occur on my machine, but I can't reproduce it. Is it reproducible for you, Jacob? If so, could you check if ​this solves the issue? If it does, I'll submit it as PR.

[Peter J. Farrell]

We are seeing the same issue especially the container starts (less often if it's just a server reload). It appears the timeout on pywatchman needs to be configurable from Django in this case as 1.0 seconds is too low in certain circumstances.

[Tobias Kunze]

A more generous default timeout might be a better choice than a configuration option, in my opinion. In any case it would be great if somebody who runs into this consistently could test ​my patch or provide one of their own.

[Jacob Green]

I run into the issue consistently on our codebase so I can repro it. Tobias, how would you recommend that I test against your patch? I know how to install Django from pypi but I don't know how to install it from a fork.

Also I may be misreading but in the commit that you linked it seems that you lowered the timeout to 0.2s instead of raising it above 1.0s. Perhaps that was unintentional?

[James Addison]

I've just found that adding a .watchmanconfig file to add the ignore_dirs directive seemed to help (I assume this would allow watchman to start more quickly). See ​https://facebook.github.io/watchman/docs/config.html#ignore_dirs

{
  "ignore_dirs": ["node_modules", ".envs", "bin", "lib", "include", ".git", ".idea", "templates", "static_assets"]
}

It certainly isn't a fix - but as mentioned in the Django docs, it's worth investigating as a bandaid.

[Jacob Green]

I tried out this change (increasing timeout to 5 seconds) and that fixed my issue:

​https://github.com/django/django/compare/stable/2.2.x...Jacobkg:Jacobkg-patch-1

I'm happy to submit this as a patch if there is interest

[Jacob Green]

I submitted a Pull Request, with the proposal that this be backported onto 2.2 since it improves the functionality of a feature released as part of 2.2

​PR

[Mariusz Felisiak <felisiak.mariusz@…>]

In ed3c590:

Fixed #30361 -- Increased the default timeout of watchman client to 5 seconds and made it customizable.

Made the default timeout of watchman client customizable via
DJANGO_WATCHMAN_TIMEOUT environment variable.

[Mariusz Felisiak <felisiak.mariusz@…>]

In e4576319:

[2.2.x] Fixed #30361 -- Increased the default timeout of watchman client to 5 seconds and made it customizable.

Made the default timeout of watchman client customizable via
DJANGO_WATCHMAN_TIMEOUT environment variable.

Backport of ed3c59097a01ed3f32f8a8bed95307fb5c181251 from master