Issue 8952

Title
Executing the import_zip.py script in country module, it returns error 404
Priority
feature
Status
chatting
Nosy list
Marina, ced, pokoli, yangoon
Assigned to
Keywords

Created on 2019-12-23.15:39:26 by Marina, last changed 8 months ago by pokoli.

Messages

Author: [hidden] (pokoli) Tryton committer Tryton translator
Date: 2020-02-24.13:44:51
From the export readme [1] I see that the work is available under Creative Commons Attribution 4.0 License [2], which allows us to copy (and even modify) if the only need of giving the apropiate credit to it's authors. 


[1] https://download.geonames.org/export/dump/readme.txt
[2] https://creativecommons.org/licenses/by/4.0/
Author: [hidden] (ced) Tryton committer Tryton translator
Date: 2020-02-24.10:27:12
I re-open it because this week-end the service was again down (and prevented the publication of the docker images as the tests failed).

I think we should publish a mirror of this site on our CDN and refresh it once a week. But for that someone needs to check if we are allowed to do that.
The import_zip script should have an extra parameter to allow specify the base URL to fetch with the default value pointing to Tryton's CDN.
Author: [hidden] (ced) Tryton committer Tryton translator
Date: 2020-02-16.18:53:49
I mark it as closed as the service is now working and we got no answer about alternative path.
Author: [hidden] (pokoli) Tryton committer Tryton translator
Date: 2020-01-29.14:38:26
El 29/1/20 a les 13:53, C├ędric Krier ha escrit:
> Does it contain the same data?
Yes, it's exactly the same file and can be imported.
>
> What is the policy of geonames.org about that?
> For me, we need to have a statement from them to use any fallback URL.

I haven't seen any statement from geonames.org but I've added a comment on [1] to see if the proposed direction is also sopported.

The comment is still pending to be approves, so lets see if we have any reply.

[1] https://groups.google.com/forum/#!topic/geonames/54PjqEBeSvk
Author: [hidden] (ced) Tryton committer Tryton translator
Date: 2020-01-29.13:53:34
Does it contain the same data?
What is the policy of geonames.org about that?
For me, we need to have a statement from them to use any fallback URL.
Author: [hidden] (pokoli) Tryton committer Tryton translator
Date: 2020-01-29.13:33:29
Currently the downloads server is not avialable. 

http://download.geonames.org/export/zip/ES.zip -> Returns Not found

but: 

http://www.geonames.org/export/zip/ES.zip -> Works as expected. 

I'm wondering if we can use the second url as fallback on scripts.
Author: [hidden] (Marina)
Date: 2019-12-24.11:52:06
It was a temporally issue, the url works again
Author: [hidden] (ced) Tryton committer Tryton translator
Date: 2019-12-23.16:06:09
For me, the dump URL does not contain the same data. I guess they have some issue on their server which prevents them to publish the zip files. It would be better to contact them.
Author: [hidden] (Marina)
Date: 2019-12-23.15:39:26
> It seems that the url has changed, the new url is 'http://download.geonames.org/export/dump/'

FetchingTraceback (most recent call last):
  File "./modules/country/scripts/import_zip.py", line 122, in <module>
    run()
  File "./modules/country/scripts/import_zip.py", line 118, in run
    main(args.database, args.codes, args.config_file)
  File "./modules/country/scripts/import_zip.py", line 97, in main
    do_import(codes)
  File "./modules/country/scripts/import_zip.py", line 105, in do_import
    import_(fetch(code))
  File "./modules/country/scripts/import_zip.py", line 40, in fetch
    responce = urlopen(url)
  File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/lib/python3.5/urllib/request.py", line 472, in open
    response = meth(req, response)
  File "/usr/lib/python3.5/urllib/request.py", line 582, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib/python3.5/urllib/request.py", line 510, in error
    return self._call_chain(*args)
  File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.5/urllib/request.py", line 590, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 404: Not Found


> The structure of the import file also seems to have changed.

Traceback (most recent call last):
  File "./modules/country/scripts/import_zip.py", line 122, in <module>
    run()
  File "./modules/country/scripts/import_zip.py", line 118, in run
    main(args.database, args.codes, args.config_file)
  File "./modules/country/scripts/import_zip.py", line 97, in main
    do_import(codes)
  File "./modules/country/scripts/import_zip.py", line 105, in do_import
    import_(fetch(code))
  File "./modules/country/scripts/import_zip.py", line 83, in import_
    country = get_country(row['country'])
  File "./modules/country/scripts/import_zip.py", line 57, in get_country
    country, = Country.find([('code', '=', code)])
ValueError: not enough values to unpack (expected 1, got 0)
History
Date User Action Args
2020-09-09 13:22:29cedlinkissue9586 superseder
2020-02-24 13:44:52pokolisetmessages: + msg55560
2020-02-24 10:27:12cedsetstatus: closed -> chatting
priority: bug -> feature
type: behavior -> feature request
messages: + msg55555
2020-02-16 18:53:49cedsetstatus: chatting -> closed
messages: + msg55410
2020-01-29 16:47:03yangoonsetnosy: + yangoon
2020-01-29 14:38:27pokolisetmessages: + msg55103
2020-01-29 13:53:34cedsetmessages: + msg55101
2020-01-29 13:33:29pokolisetstatus: closed -> chatting
nosy: + pokoli
messages: + msg55099
2019-12-24 11:52:07Marinasetstatus: deferred -> closed
messages: + msg54635
2019-12-23 16:06:10cedsetstatus: chatting -> deferred
nosy: + ced, - pokoli
messages: + msg54627
2019-12-23 16:02:43pokolisetnosy: + pokoli
2019-12-23 15:39:26Marinacreate