Opened 7 years ago

Closed 7 years ago

Last modified 7 years ago

#29239 closed Bug (wontfix)

GEOSGeometry treats WKB and WKT input differently (former incorrectly)

Reported by: MatBurnham Owned by: Jani Tiainen
Component: GIS Version: 2.0
Severity: Normal Keywords:
Cc: Triage Stage: Unreviewed
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description (last modified by MatBurnham)

I have imported some data from Trimble's OpenStreetMap dataset into PostGIS using shp2pgsql. I tried to load this into a Django model generated using manage.py inspectdb. This worked fine for the points and lines tables, but not for polygons as I get an error GEOS_ERROR: IllegalArgumentException: Points of LinearRing do not form a closed linestring.

As far as I can tell, the data from OSM does not form a closed linestring and should not be interpreted as a LinearRing, rather it should be a MultiPolygon.

In trying to debug the issue, I created GEOSGeometry objects manually from the same data. I identified a difference depending on the format presented - hex-encoded WKB fails in the way described; the same shape pre-converted in PostGIS to WKT using SELECT ST_AsEWKT(...) works fine.

>>> from django.contrib.gis.geos import GEOSGeometry

>>> wkb = '01060000C00100000001030000C00100000005000000EF03EB93371B9E3F7477425486A349400000000000000000FFFFFFFFFFFFEFFF00AE64C746209E3FBFA6BD778AA349400000000000000000FFFFFFFFFFFFEFFF1224004922459E3F540A70308AA349400000000000000000FFFFFFFFFFFFEFFF027A861513409E3F9E9ACB0D86A349400000000000000000FFFFFFFFFFFFEFFFEF03EB93371B9E3F7477425486A349400000000000000000FFFFFFFFFFFFEFFF'

>>> wkt = 'MULTIPOLYGON(((0.0294007 51.2775369 0 -1.79769313486232e+308,0.02942 51.2776632 0 -1.79769313486232e+308,0.0295606 51.2776547 0 -1.79769313486232e+308,0.0295413 51.2775285 0 -1.79769313486232e+308,0.0294007 51.2775369 0 -1.79769313486232e+308)))'

>>> GEOSGeometry(wkb)
GEOS_ERROR: IllegalArgumentException: Points of LinearRing do not form a closed linestring

Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "/home/mat/virtualenvs/test/lib/python3.6/site-packages/django/contrib/gis/geos/geometry.py", line 690, in __init__
    g = wkb_r().read(force_bytes(geo_input))
  File "/home/mat/virtualenvs/test/lib/python3.6/site-packages/django/contrib/gis/geos/prototypes/io.py", line 151, in read
    return wkb_reader_read_hex(self.ptr, wkb, len(wkb))
  File "/home/mat/virtualenvs/test/lib/python3.6/site-packages/django/contrib/gis/geos/libgeos.py", line 155, in __call__
    return self.func(*args, **kwargs)
  File "/home/mat/virtualenvs/test/lib/python3.6/site-packages/django/contrib/gis/geos/prototypes/threadsafe.py", line 48, in __call__
    return self.cfunc(self.thread_context.handle.ptr, *args)
  File "/home/mat/virtualenvs/test/lib/python3.6/site-packages/django/contrib/gis/geos/prototypes/errcheck.py", line 32, in check_geom
    raise GEOSException('Error encountered checking Geometry returned from GEOS C function "%s".' % func.__name__)
django.contrib.gis.geos.error.GEOSException: Error encountered checking Geometry returned from GEOS C function "GEOSWKBReader_readHEX_r".

>>> GEOSGeometry(wkt)
<MultiPolygon object at 0x7f8d0f094f78>

The error traces through to the upstream C code, but I'm not sure whether that's the root cause or whether it's in the Python wrapper. osgeo.org are being somewhat difficult about creating an account to report the bug there.

I'm using stock Django 2.0.2 on an OSBoxes-derived Ubuntu 17.10 VM with Python 3.6.3. GEOS library version appears to be 3.5.1-CAPI-1.9.1 r4246.

Change History (6)

comment:1 by Jani Tiainen, 7 years ago

Resolution: invalid
Status: newclosed

Problem seems to originate from GEOS library itself (native library) which apparently treats different types of data slightly differently.

In any case, you geometry is not valid thus Django is not at fault. If problem persists with a valid geometry please reopen the ticket.

comment:2 by MatBurnham, 7 years ago

Resolution: invalid
Status: closednew

The geometry IS valid. Formatted as WKT it is accepted. The same geometry formatted as hex encoded WKB is misinterpreted as LinearRing when it should be a MultiPolygon.

comment:3 by Jani Tiainen, 7 years ago

Owner: changed from nobody to Jani Tiainen
Status: newassigned

Django passes geometries to GEOS library so error originates from there.

Now few notes about WKT geometry is it XYZM geometry since it seems to contain 4 values per point?

Django supports only 3D geometries so you may lose data here. Also tried to decode WKB but from the quick look it looked like WKB from postgis (because AFAIK it uses geos as well as underlying library).

Few points from geometry.

-0.45747735963738   51.4742768635629 0 -1.79769313486232e+308, 
-0.457470821752906 51.474364454451   0 -1.79769313486232e+308,

Version of installed GEOS libraries would be useful information here.

comment:4 by MatBurnham, 7 years ago

Description: modified (diff)
Summary: GEOSGeometry treats WKB and WKT input differently (former fails)GEOSGeometry treats WKB and WKT input differently (former incorrectly)

I've tried to find a more succinct example and amended the bug report to reflect where the data has come from.

comment:5 by Jani Tiainen, 7 years ago

Resolution: wontfix
Status: assignedclosed

From WKT you lose M data:

>>> g1 = GEOSGeometry(wkt)
>>> g1.wkt
'MULTIPOLYGON Z (((0.0294007 51.2775369 0, 0.02942 51.2776632 0, 0.0295606 51.2776547 0, 0.0295413 51.2775285 0, 0.0294007 51.2775369 0)))'

As you see, fourth value is gone and you have lost some of your data.

Since Django supports only 3D (XYZ) coordinates and GEOS library is at the fault closing as won't fix.

comment:6 by Jani Tiainen, 7 years ago

Note that there exists issue in GEOS tracker for WKB M-support:
https://trac.osgeo.org/geos/ticket/460

Note: See TracTickets for help on using tickets.
Back to Top