Tryton - Issues

 

Issue8952

Title Executing the import_zip.py script in country module, it returns error 404
Priority feature Status chatting
Superseder Nosy List Marina, ced, pokoli, yangoon
Type feature request Components country
Assigned To Keywords
Reviews

Created on 2019-12-23.15:39:26 by Marina, last changed by pokoli.

Messages
msg55560 (view) Author: [hidden] (pokoli) (Tryton committer) (Tryton translator) Date: 2020-02-24.13:44:51
From the export readme [1] I see that the work is available under Creative Commons Attribution 4.0 License [2], which allows us to copy (and even modify) if the only need of giving the apropiate credit to it's authors. 


[1] https://download.geonames.org/export/dump/readme.txt
[2] https://creativecommons.org/licenses/by/4.0/
msg55555 (view) Author: [hidden] (ced) (Tryton committer) (Tryton translator) Date: 2020-02-24.10:27:12
I re-open it because this week-end the service was again down (and prevented the publication of the docker images as the tests failed).

I think we should publish a mirror of this site on our CDN and refresh it once a week. But for that someone needs to check if we are allowed to do that.
The import_zip script should have an extra parameter to allow specify the base URL to fetch with the default value pointing to Tryton's CDN.
msg55410 (view) Author: [hidden] (ced) (Tryton committer) (Tryton translator) Date: 2020-02-16.18:53:49
I mark it as closed as the service is now working and we got no answer about alternative path.
msg55103 (view) Author: [hidden] (pokoli) (Tryton committer) (Tryton translator) Date: 2020-01-29.14:38:26
El 29/1/20 a les 13:53, C├ędric Krier ha escrit:
> Does it contain the same data?
Yes, it's exactly the same file and can be imported.
>
> What is the policy of geonames.org about that?
> For me, we need to have a statement from them to use any fallback URL.

I haven't seen any statement from geonames.org but I've added a comment on [1] to see if the proposed direction is also sopported.

The comment is still pending to be approves, so lets see if we have any reply.

[1] https://groups.google.com/forum/#!topic/geonames/54PjqEBeSvk
msg55101 (view) Author: [hidden] (ced) (Tryton committer) (Tryton translator) Date: 2020-01-29.13:53:34
Does it contain the same data?
What is the policy of geonames.org about that?
For me, we need to have a statement from them to use any fallback URL.
msg55099 (view) Author: [hidden] (pokoli) (Tryton committer) (Tryton translator) Date: 2020-01-29.13:33:29
Currently the downloads server is not avialable. 

http://download.geonames.org/export/zip/ES.zip -> Returns Not found

but: 

http://www.geonames.org/export/zip/ES.zip -> Works as expected. 

I'm wondering if we can use the second url as fallback on scripts.
msg54635 (view) Author: [hidden] (Marina) Date: 2019-12-24.11:52:06
It was a temporally issue, the url works again
msg54627 (view) Author: [hidden] (ced) (Tryton committer) (Tryton translator) Date: 2019-12-23.16:06:09
For me, the dump URL does not contain the same data. I guess they have some issue on their server which prevents them to publish the zip files. It would be better to contact them.
msg54624 (view) Author: [hidden] (Marina) Date: 2019-12-23.15:39:26
> It seems that the url has changed, the new url is 'http://download.geonames.org/export/dump/'

FetchingTraceback (most recent call last):
  File "./modules/country/scripts/import_zip.py", line 122, in <module>
    run()
  File "./modules/country/scripts/import_zip.py", line 118, in run
    main(args.database, args.codes, args.config_file)
  File "./modules/country/scripts/import_zip.py", line 97, in main
    do_import(codes)
  File "./modules/country/scripts/import_zip.py", line 105, in do_import
    import_(fetch(code))
  File "./modules/country/scripts/import_zip.py", line 40, in fetch
    responce = urlopen(url)
  File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/lib/python3.5/urllib/request.py", line 472, in open
    response = meth(req, response)
  File "/usr/lib/python3.5/urllib/request.py", line 582, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib/python3.5/urllib/request.py", line 510, in error
    return self._call_chain(*args)
  File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.5/urllib/request.py", line 590, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 404: Not Found


> The structure of the import file also seems to have changed.

Traceback (most recent call last):
  File "./modules/country/scripts/import_zip.py", line 122, in <module>
    run()
  File "./modules/country/scripts/import_zip.py", line 118, in run
    main(args.database, args.codes, args.config_file)
  File "./modules/country/scripts/import_zip.py", line 97, in main
    do_import(codes)
  File "./modules/country/scripts/import_zip.py", line 105, in do_import
    import_(fetch(code))
  File "./modules/country/scripts/import_zip.py", line 83, in import_
    country = get_country(row['country'])
  File "./modules/country/scripts/import_zip.py", line 57, in get_country
    country, = Country.find([('code', '=', code)])
ValueError: not enough values to unpack (expected 1, got 0)
History
Date User Action Args
2020-09-09 13:22:29cedlinkissue9586 superseder
2020-02-24 13:44:52pokolisetmessages: + msg55560
2020-02-24 10:27:12cedsetstatus: closed -> chatting
priority: bug -> feature
type: behavior -> feature request
messages: + msg55555
2020-02-16 18:53:49cedsetstatus: chatting -> closed
messages: + msg55410
2020-01-29 16:47:03yangoonsetnosy: + yangoon
2020-01-29 14:38:27pokolisetmessages: + msg55103
2020-01-29 13:53:34cedsetmessages: + msg55101
2020-01-29 13:33:29pokolisetstatus: closed -> chatting
nosy: + pokoli
messages: + msg55099
2019-12-24 11:52:07Marinasetstatus: deferred -> closed
messages: + msg54635
2019-12-23 16:06:10cedsetstatus: chatting -> deferred
nosy: + ced, - pokoli
messages: + msg54627

Showing 10 items. Show all history (warning: this could be VERY long)