Issue 9561

Title
Improve account type amount performance
Priority
feature
Status
testing
Superseder
Ir groups should be defined on res module (issue 9633)
Nosy list
ced, pokoli, reviewbot
Assigned to
pokoli
Keywords
review

Created on 2020-08-31.18:17:54 by pokoli, last changed 2 days ago by reviewbot.

Messages

Author: [hidden] (pokoli) Tryton committer Tryton translator
Date: 2020-09-02.18:19:01
El 2/9/20 a les 15:21, C├ędric Krier ha escrit:
> The groups stuffs should be defined in res module.
Thanks, I've updated the review following your suggestion and test are working now!
Author: [hidden] (ced) Tryton committer Tryton translator
Date: 2020-09-02.15:21:19
The groups stuffs should be defined in res module.
Author: [hidden] (pokoli) Tryton committer Tryton translator
Date: 2020-09-02.14:41:46
I have the following traceback when creating a new database with this review:

Traceback (most recent call last):
  File "/home/pokoli/projectes/nclone/trytond/trytond/tests/test_field_function.py", line 15, in setUpClass
    activate_module('tests')
  File "/home/pokoli/projectes/nclone/trytond/trytond/tests/test_tryton.py", line 59, in activate_module
    create_db(lang=lang)
  File "/home/pokoli/projectes/nclone/trytond/trytond/tests/test_tryton.py", line 792, in create_db
    pool.init(update=['res', 'ir'], lang=[lang])
  File "/home/pokoli/projectes/nclone/trytond/trytond/pool.py", line 162, in init
    restart = not load_modules(
  File "/home/pokoli/projectes/nclone/trytond/trytond/modules/__init__.py", line 434, in load_modules
    _load_modules(update)
  File "/home/pokoli/projectes/nclone/trytond/trytond/modules/__init__.py", line 401, in _load_modules
    load_module_graph(graph, pool, update, lang)
  File "/home/pokoli/projectes/nclone/trytond/trytond/modules/__init__.py", line 252, in load_module_graph
    tryton_parser.parse_xmlstream(fp)
  File "/home/pokoli/projectes/nclone/trytond/trytond/convert.py", line 456, in parse_xmlstream
    self.sax_parser.parse(source)
  File "/usr/lib64/python3.8/xml/sax/expatreader.py", line 111, in parse
    xmlreader.IncrementalParser.parse(self, source)
  File "/usr/lib64/python3.8/xml/sax/xmlreader.py", line 125, in parse
    self.feed(buffer)
  File "/usr/lib64/python3.8/xml/sax/expatreader.py", line 217, in feed
    self._parser.Parse(data, isFinal)
  File "/build/python/src/Python-3.8.5/Modules/pyexpat.c", line 459, in EndElement
  File "/usr/lib64/python3.8/xml/sax/expatreader.py", line 336, in end_element
    self._cont_handler.endElement(name)
  File "/home/pokoli/projectes/nclone/trytond/trytond/convert.py", line 511, in endElement
    self.taghandler = self.taghandler.endElement(name)
  File "/home/pokoli/projectes/nclone/trytond/trytond/convert.py", line 294, in endElement
    self.mh.import_record(
  File "/home/pokoli/projectes/nclone/trytond/trytond/convert.py", line 687, in import_record
    self.create_records(model, [values], [fs_id])
  File "/home/pokoli/projectes/nclone/trytond/trytond/convert.py", line 698, in create_records
    values[key] = self._clean_value(key, record)
  File "/home/pokoli/projectes/nclone/trytond/trytond/convert.py", line 554, in _clean_value
    return getattr(record, key)
  File "/home/pokoli/projectes/nclone/trytond/trytond/model/fields/function.py", line 130, in __get__
    return super().__get__(inst, cls)
  File "/home/pokoli/projectes/nclone/trytond/trytond/model/fields/field.py", line 336, in __get__
    return inst.__getattr__(self.name)
  File "/home/pokoli/projectes/nclone/trytond/trytond/model/modelstorage.py", line 1587, in __getattr__
    read_data = self.read(list(index.keys()), list(ffields.keys()))
  File "/home/pokoli/projectes/nclone/trytond/trytond/model/modelsql.py", line 837, in read
    getter_results = field.get(ids, cls, field_list, values=result)
  File "/home/pokoli/projectes/nclone/trytond/trytond/model/fields/function.py", line 105, in get
    return call(names)
  File "/home/pokoli/projectes/nclone/trytond/trytond/model/fields/function.py", line 99, in call
    return method(records, name)
  File "/home/pokoli/projectes/nclone/trytond/trytond/ir/action.py", line 315, in get_action
    value = getattr(value, name)
  File "/home/pokoli/projectes/nclone/trytond/trytond/model/fields/field.py", line 336, in __get__
    return inst.__getattr__(self.name)
  File "/home/pokoli/projectes/nclone/trytond/trytond/model/modelstorage.py", line 1587, in __getattr__
    read_data = self.read(list(index.keys()), list(ffields.keys()))
  File "/home/pokoli/projectes/nclone/trytond/trytond/model/modelsql.py", line 818, in read
    getter_result = field.get(ids, cls, fname, values=result)
  File "/home/pokoli/projectes/nclone/trytond/trytond/model/fields/many2many.py", line 129, in get
    Relation = self.get_relation()
  File "/home/pokoli/projectes/nclone/trytond/trytond/model/fields/many2many.py", line 274, in get_relation
    return Pool().get(self.relation_name)
  File "/home/pokoli/projectes/nclone/trytond/trytond/pool.py", line 184, in get
    return self._pool[self.database_name][type][name]
KeyError: 'ir.action-res.group'


The problem, is that the groups field of the Action which is loaded when loading the name (the name is a function field) but the module is not loaded in the pool because it is loaded on the res module. 

I do not see how to solve this. IIRC there was some issue to merge ir and res into base module (which I can not find on the tracker). This may solve the issue but I'm wondering if there is a simpler solution.

What do you think?
Author: [hidden] (pokoli) Tryton committer Tryton translator
Date: 2020-09-02.13:31:53
I've added review314201002 which implements msg59891
So the performance optimitzation will be applied for all functional fields of same fucntion name.
Author: [hidden] (ced) Tryton committer Tryton translator
Date: 2020-08-31.18:49:13
I'm wondering if it will not be better to improve ModelStorage.__getattr__ to be smarter with lazy loading fields.
Maybe we could add if we are loading a lazy field which is a Function field, all other fields that have the same classmethod getter.
Author: [hidden] (pokoli) Tryton committer Tryton translator
Date: 2020-08-31.18:17:54
The computation of the account.account.type amount reads the debit and credit debits of the related accounts. This fields can computed using a single function (using single query) but as the code call them in diferent lines the function is called one time for each field. 

On big database the computation of this query can spend long time because a big number of values should be summed (We mesured 700ms per query on our of our customers database). Computing both fields in a single call reduces the time to compute by a half as the data needs to be scanned only once. 

In this case I think we should use search_read to read both values into a single call, which means computing both values into a single query and reducing the time to compute the amount of a type.
History
Date User Action Args
2020-11-30 16:00:20reviewbotsetmessages: + msg62315
2020-11-21 18:50:59reviewbotsetmessages: + msg62095
2020-11-07 20:08:52reviewbotsetmessages: + msg61748
2020-10-12 18:11:13reviewbotsetmessages: + msg60866
2020-10-12 00:32:54reviewbotsetmessages: + msg60822
2020-10-09 22:32:01reviewbotsetmessages: + msg60752
2020-10-09 13:31:03reviewbotsetmessages: + msg60701
2020-09-25 00:25:14reviewbotsetmessages: + msg60361
2020-09-24 23:55:00pokolisetsuperseder: + Ir groups should be defined on res module
2020-09-02 18:41:44reviewbotsetmessages: + msg59921

Showing 10 items. Show all history (warning: this could be VERY long)