Tryton - Issues

 

Issue2919

Title Support geographic types
Priority feature Status resolved
Superseder Nosy List albertca, ced, katr, pilou, pokoli, reviewbot, roundup-bot, udono
Type feature request Components
Assigned To ced Keywords review
Reviews 43011002
View: 43011002

Created on 2012-12-08.00:40:48 by pilou, last changed by roundup-bot.

Messages
New changeset 2a07d81dbdc9 by Cédric Krier in branch 'default':
Initial commit
http://hg.tryton.org/trytond-gis/rev/2a07d81dbdc9
review43011002 updated at https://codereview.tryton.org/43011002/#ps1
msg37493 (view) Author: [hidden] (ced) (Tryton committer) (Tryton translator) Date: 2017-12-31.17:39:01
It was decided that the client should not try to manage GIS fields because it is too specialized requirement. But indeed the server should be able to define and manipulate GIS columns (which may be filled by GIS software).
So thanks to recent rework on the backend, it is possible to register backend from external package. So as GIS is quite specialized requirement, it is the best option.
Here is review43011002 which define such backend for the development series. It is ready to run on the Tryton CI and run standard test + some custom.
It uses geomet as GeoJSON library.
msg15277 (view) Author: [hidden] (udono) Date: 2014-01-22.18:02:39
Found a nice showcase for switching form/list/maps view:

    * http://demos.jquerymobile.com/1.4.0/map-list-toggle/#map-page
msg13274 (view) Author: [hidden] (ced) (Tryton committer) (Tryton translator) Date: 2013-04-27.15:56:33
On 27/04/13 14:54 +0200, Pierre-Louis Bonicoli wrote:
> Missing for inclusion:
> 
> * First: remove dependency to dbgis
>  1) Performance problem
> We encounter performance problems with the current patch set (review210002 patch
> set 17 & review208001 patch set 18). When a geometry is processed by the tryton
> client, the following operations occur:
>  - trytond fetch the geometry from the database using EWKB format
>  - from the EWKB representation trytond create an instance of dbgis.Geometry

We need a Python structure to manipulate such object.

>  - then trytond serialize dbgis.Geometry into JSON
>  - serialization is sent to the client
> The last two operations don't scale at all when the geometry has a large size:

Did you try with simplejson?

>  - a lot of memory are used for serialization (size of EWKB representation <<<
> size of serialization)
>  - sending over the network this serialization took a lot of time

After that any way the protocol used, sending large size of data takes
times.
And the goal is not to make a full GIS application but to manage
simple/small data.

>  2) dbgis doesn't support pyspatialite at all (ie only reading Point is
> currently implemented)

So dbgis needs to be improved.

>  3) instead of using EWKB format, dbgis use the internal PostgreSQL format [1]:
> nonsense

Don't understand. dbgis use a tool to read raw EWKB and convert into
Python object.

> * Second: allow to create PostgreSQL geographic database using tryton client.
> This could be easily implemented when at least PostgreSQL 9.1 are PostGIS 2.0
> are used [2], previous versions of PostGIS don't allow that.

It is not a blocking point.
msg13273 (view) Author: [hidden] (pilou) (Tryton committer) Date: 2013-04-27.14:54:22
Missing for inclusion:

* First: remove dependency to dbgis
 1) Performance problem
We encounter performance problems with the current patch set (review210002 patch
set 17 & review208001 patch set 18). When a geometry is processed by the tryton
client, the following operations occur:
 - trytond fetch the geometry from the database using EWKB format
 - from the EWKB representation trytond create an instance of dbgis.Geometry
 - then trytond serialize dbgis.Geometry into JSON
 - serialization is sent to the client
The last two operations don't scale at all when the geometry has a large size:
 - a lot of memory are used for serialization (size of EWKB representation <<<
size of serialization)
 - sending over the network this serialization took a lot of time

 2) dbgis doesn't support pyspatialite at all (ie only reading Point is
currently implemented)
 3) instead of using EWKB format, dbgis use the internal PostgreSQL format [1]:
nonsense
 
* Second: allow to create PostgreSQL geographic database using tryton client.
This could be easily implemented when at least PostgreSQL 9.1 are PostGIS 2.0
are used [2], previous versions of PostGIS don't allow that.

[1]
https://projects.bioecoforests.com/projects/python-dbgis/repository/revisions/master/entry/dbgis.py#L37
[2]
http://postgis.net/docs/manual-2.0/postgis_installation.html#create_new_db_extensions
msg13157 (view) Author: [hidden] (pilou) (Tryton committer) Date: 2013-04-19.13:09:48
For references: patch set 17 of review210002 honors the state argument of
geographic fields for Geometry.
msg12464 (view) Author: [hidden] (ced) (Tryton committer) (Tryton translator) Date: 2013-02-07.18:23:46
Missing for inclusion:

- unittest on the new fields
- public release of dbgis
msg12103 (view) Author: [hidden] (ced) (Tryton committer) (Tryton translator) Date: 2012-12-08.00:54:47
On 08/12/12 00:40 +0100, Pierre-Louis Bonicoli wrote:
> When installing a model with GIS fields (trytond -i geo -d db):
> 
> Following exceptions are encountered when trying to install a module with a
> geometric field and GIS features not available:
> 
> *trytond: -patch -dbgis -postgis*
> --> call to register fails:
> INFO:modules:foret:registering classes
> [...]
> File "trytond/trytond/modules/foret/foret.py", line 14, in Foret
>   geom = fields.MultiPolygon(u"Géométrie", srid=2154, required=True,
> AttributeError: 'module' object has no attribute 'MultiPolygon'
> 
> 
> *trytond +patch -dbgis -postgis*
> --> call to register fails:
> INFO:modules:foret:register foret.foret
> [...]
> File "trytond/trytond/model/modelsql.py", line 132, in __register__
>   FIELDS[field._type].sql_type(field), field.srid,
> KeyError: 'multipolygon'

It will be good to have a more explicit error here, because this is the
case that will often happens.

> *trytond +patch +dbgis -postgis*
> -->
> ERROR:sql:Wrong SQL: SELECT AddGeometryColumn('foret_foret', 'geom', 2154,
> 'MULTIPOLYGON', 2)
> [...]
> File "trytond/backend/postgresql/database.py", line 323, in execute
>   res = self.cursor.execute(sql, params)
> psycopg2.ProgrammingError: ERREUR:  function addgeometrycolumn(unknown, unknown,
> integer, unknown, integer) doesn't exist
> LINE 1: SELECT AddGeometryColumn('foret_foret', 'geom', 2154, 'MULTI...
> 
> 
> 
> Is there a point blocking the inclusion of these patches?
> 
> 
> NB: in order to enable PostGIS, execute SQL commands from postgis.sql &
> spatial_ref_sys.sql files:
>     psql -d db_name -f /usr/share/postgresql/9.1/contrib/postgis-1.5/postgis.sql
>     psql -d db_name -f
> /usr/share/postgresql/9.1/contrib/postgis-1.5/spatial_ref_sys.sql
> and the following rights need to be granted:
>     grant select on spatial_ref_sys to db_user;
>     grant all on geometry_columns to db_user;
>     grant all on geography_columns to db_user;

Could we add this to the doc? Maybe using a link to the documentation of
each project.
msg12102 (view) Author: [hidden] (pilou) (Tryton committer) Date: 2012-12-08.00:40:47
Hi,

At PyconFR Cedric told me that GIS patches needed more testing before inclusion
in repositories.

This is a summary of my tests with GIS patches. These patches have been
developed by Nicolas Evrard for B2CK and financed by Bio Eco Forests.

Patches are available in these codereviews:
- Tryton client: review210002 (patch set 14)
- Tryton server: review208001 (patch set 18)

python-dbgis is a dependency of these patches and is available here:
https://projects.bioecoforests.com/projects/python-dbgis

Below, "+patch -dbgis -postgis" means "patch set applied, python module dbgis
not available, postgis extension not installed on current database".


*GIS features enabled on tryton but disabled on trytond*
trytond: -patch -dbgis -postgis
tryton: +patch -dbgis
=> No problem found.

trytond: -patch -dbgis -postgis
tryton: +patch, +dbgis
=>No problem found.


*GIS features disabled on tryton but enabled on trytond*
trytond: +patch -dbgis -postgis
tryton: -patch -dbgis
=>No problem found.

trytond: +patch +dbgis -postgis
tryton: -patch -dbgis
=>No problem found (was NOK with trytond patch set 16).

trytond: +patch +dbgis +postgis, test module 'forest' installed
tryton: -patch -dbgis
=> Views related to the 'forest' module can not be displayed: an exception occurs.

trytond: +patch +dbgis +postgis
tryton: +patch -dbgis
=> Views related to the 'forest' module can not be displayed: an exception occurs.


Regarding the behavior of trytond with pyspatialite, it is the same as with postgis.


When installing a model with GIS fields (trytond -i geo -d db):

Following exceptions are encountered when trying to install a module with a
geometric field and GIS features not available:

*trytond: -patch -dbgis -postgis*
--> call to register fails:
INFO:modules:foret:registering classes
[...]
File "trytond/trytond/modules/foret/foret.py", line 14, in Foret
  geom = fields.MultiPolygon(u"Géométrie", srid=2154, required=True,
AttributeError: 'module' object has no attribute 'MultiPolygon'


*trytond +patch -dbgis -postgis*
--> call to register fails:
INFO:modules:foret:register foret.foret
[...]
File "trytond/trytond/model/modelsql.py", line 132, in __register__
  FIELDS[field._type].sql_type(field), field.srid,
KeyError: 'multipolygon'


*trytond +patch +dbgis -postgis*
-->
ERROR:sql:Wrong SQL: SELECT AddGeometryColumn('foret_foret', 'geom', 2154,
'MULTIPOLYGON', 2)
[...]
File "trytond/backend/postgresql/database.py", line 323, in execute
  res = self.cursor.execute(sql, params)
psycopg2.ProgrammingError: ERREUR:  function addgeometrycolumn(unknown, unknown,
integer, unknown, integer) doesn't exist
LINE 1: SELECT AddGeometryColumn('foret_foret', 'geom', 2154, 'MULTI...



Is there a point blocking the inclusion of these patches?


NB: in order to enable PostGIS, execute SQL commands from postgis.sql &
spatial_ref_sys.sql files:
    psql -d db_name -f /usr/share/postgresql/9.1/contrib/postgis-1.5/postgis.sql
    psql -d db_name -f
/usr/share/postgresql/9.1/contrib/postgis-1.5/spatial_ref_sys.sql
and the following rights need to be granted:
    grant select on spatial_ref_sys to db_user;
    grant all on geometry_columns to db_user;
    grant all on geography_columns to db_user;
History
Date User Action Args
2018-01-05 17:48:04roundup-botsetstatus: testing -> resolved
nosy: + roundup-bot
messages: + msg37516
2017-12-31 17:54:47reviewbotsetnosy: + reviewbot
messages: + msg37494
2017-12-31 17:39:16cedsetreviews: geomet -> 43011002
2017-12-31 17:39:02cedsetstatus: deferred -> testing
reviews: geomet
messages: + msg37493
keyword: + review
assignedto: pilou -> ced
2014-01-22 18:02:39udonosetnosy: + udono
messages: + msg15277
2014-01-14 12:25:38pokolisetnosy: + pokoli
2014-01-09 11:52:02katrsetnosy: + katr
2013-10-20 14:55:32albertcasetnosy: + albertca
2013-05-02 08:12:14udonosetassignedto: pilou
2013-04-27 15:56:34cedsetmessages: + msg13274

Showing 10 items. Show all history (warning: this could be VERY long)