Created on 2020-08-31.18:17:54 by pokoli, last changed 2 months ago by roundup-bot.
New changeset 427291d4e374 by Sergi Almacellas Abellana in branch 'default': Eager load Function field with same multiple getter https://hg.tryton.org/tryton-env/rev/427291d4e374
New changeset 28cf90b8bc4e by Sergi Almacellas Abellana in branch 'default': Eager load Function field with same multiple getter https://hg.tryton.org/trytond/rev/28cf90b8bc4e
El 2/9/20 a les 15:21, Cédric Krier ha escrit: > The groups stuffs should be defined in res module. Thanks, I've updated the review following your suggestion and test are working now!
The groups stuffs should be defined in res module.
I have the following traceback when creating a new database with this review: Traceback (most recent call last): File "/home/pokoli/projectes/nclone/trytond/trytond/tests/test_field_function.py", line 15, in setUpClass activate_module('tests') File "/home/pokoli/projectes/nclone/trytond/trytond/tests/test_tryton.py", line 59, in activate_module create_db(lang=lang) File "/home/pokoli/projectes/nclone/trytond/trytond/tests/test_tryton.py", line 792, in create_db pool.init(update=['res', 'ir'], lang=[lang]) File "/home/pokoli/projectes/nclone/trytond/trytond/pool.py", line 162, in init restart = not load_modules( File "/home/pokoli/projectes/nclone/trytond/trytond/modules/__init__.py", line 434, in load_modules _load_modules(update) File "/home/pokoli/projectes/nclone/trytond/trytond/modules/__init__.py", line 401, in _load_modules load_module_graph(graph, pool, update, lang) File "/home/pokoli/projectes/nclone/trytond/trytond/modules/__init__.py", line 252, in load_module_graph tryton_parser.parse_xmlstream(fp) File "/home/pokoli/projectes/nclone/trytond/trytond/convert.py", line 456, in parse_xmlstream self.sax_parser.parse(source) File "/usr/lib64/python3.8/xml/sax/expatreader.py", line 111, in parse xmlreader.IncrementalParser.parse(self, source) File "/usr/lib64/python3.8/xml/sax/xmlreader.py", line 125, in parse self.feed(buffer) File "/usr/lib64/python3.8/xml/sax/expatreader.py", line 217, in feed self._parser.Parse(data, isFinal) File "/build/python/src/Python-3.8.5/Modules/pyexpat.c", line 459, in EndElement File "/usr/lib64/python3.8/xml/sax/expatreader.py", line 336, in end_element self._cont_handler.endElement(name) File "/home/pokoli/projectes/nclone/trytond/trytond/convert.py", line 511, in endElement self.taghandler = self.taghandler.endElement(name) File "/home/pokoli/projectes/nclone/trytond/trytond/convert.py", line 294, in endElement self.mh.import_record( File "/home/pokoli/projectes/nclone/trytond/trytond/convert.py", line 687, in import_record self.create_records(model, [values], [fs_id]) File "/home/pokoli/projectes/nclone/trytond/trytond/convert.py", line 698, in create_records values[key] = self._clean_value(key, record) File "/home/pokoli/projectes/nclone/trytond/trytond/convert.py", line 554, in _clean_value return getattr(record, key) File "/home/pokoli/projectes/nclone/trytond/trytond/model/fields/function.py", line 130, in __get__ return super().__get__(inst, cls) File "/home/pokoli/projectes/nclone/trytond/trytond/model/fields/field.py", line 336, in __get__ return inst.__getattr__(self.name) File "/home/pokoli/projectes/nclone/trytond/trytond/model/modelstorage.py", line 1587, in __getattr__ read_data = self.read(list(index.keys()), list(ffields.keys())) File "/home/pokoli/projectes/nclone/trytond/trytond/model/modelsql.py", line 837, in read getter_results = field.get(ids, cls, field_list, values=result) File "/home/pokoli/projectes/nclone/trytond/trytond/model/fields/function.py", line 105, in get return call(names) File "/home/pokoli/projectes/nclone/trytond/trytond/model/fields/function.py", line 99, in call return method(records, name) File "/home/pokoli/projectes/nclone/trytond/trytond/ir/action.py", line 315, in get_action value = getattr(value, name) File "/home/pokoli/projectes/nclone/trytond/trytond/model/fields/field.py", line 336, in __get__ return inst.__getattr__(self.name) File "/home/pokoli/projectes/nclone/trytond/trytond/model/modelstorage.py", line 1587, in __getattr__ read_data = self.read(list(index.keys()), list(ffields.keys())) File "/home/pokoli/projectes/nclone/trytond/trytond/model/modelsql.py", line 818, in read getter_result = field.get(ids, cls, fname, values=result) File "/home/pokoli/projectes/nclone/trytond/trytond/model/fields/many2many.py", line 129, in get Relation = self.get_relation() File "/home/pokoli/projectes/nclone/trytond/trytond/model/fields/many2many.py", line 274, in get_relation return Pool().get(self.relation_name) File "/home/pokoli/projectes/nclone/trytond/trytond/pool.py", line 184, in get return self._pool[self.database_name][type][name] KeyError: 'ir.action-res.group' The problem, is that the groups field of the Action which is loaded when loading the name (the name is a function field) but the module is not loaded in the pool because it is loaded on the res module. I do not see how to solve this. IIRC there was some issue to merge ir and res into base module (which I can not find on the tracker). This may solve the issue but I'm wondering if there is a simpler solution. What do you think?
I've added review314201002 which implements msg59891 So the performance optimitzation will be applied for all functional fields of same fucntion name.
I'm wondering if it will not be better to improve ModelStorage.__getattr__ to be smarter with lazy loading fields. Maybe we could add if we are loading a lazy field which is a Function field, all other fields that have the same classmethod getter.
The computation of the account.account.type amount reads the debit and credit debits of the related accounts. This fields can computed using a single function (using single query) but as the code call them in diferent lines the function is called one time for each field. On big database the computation of this query can spend long time because a big number of values should be summed (We mesured 700ms per query on our of our customers database). Computing both fields in a single call reduces the time to compute by a half as the data needs to be scanned only once. In this case I think we should use search_read to read both values into a single call, which means computing both values into a single query and reducing the time to compute the amount of a type.
History | |||
---|---|---|---|
Date | User | Action | Args |
2020-12-06 18:34:56 | roundup-bot | set | messages: + msg62480 |
2020-12-06 18:34:54 | roundup-bot | set | messages:
+ msg62479 nosy: + roundup-bot status: testing -> resolved |
2020-11-30 16:00:20 | reviewbot | set | messages: + msg62315 |
2020-11-21 18:50:59 | reviewbot | set | messages: + msg62095 |
2020-11-07 20:08:52 | reviewbot | set | messages: + msg61748 |
2020-10-12 18:11:13 | reviewbot | set | messages: + msg60866 |
2020-10-12 00:32:54 | reviewbot | set | messages: + msg60822 |
2020-10-09 22:32:01 | reviewbot | set | messages: + msg60752 |
2020-10-09 13:31:03 | reviewbot | set | messages: + msg60701 |
2020-09-25 00:25:14 | reviewbot | set | messages: + msg60361 |
Showing 10 items. Show all history (warning: this could be VERY long)