Issue 11129

Title
Process task by batch
Priority
feature
Status
testing
Nosy list
ced, pokoli, resteve, reviewbot
Assigned to
ced
Keywords
review

Created on 2022-01-11.17:01:46 by ced, last changed 1 week ago by resteve.

Messages

Author: [hidden] (ced) Tryton committer Tryton translator
Date: 2022-01-11.17:17:17

I activate by default the batch on modules where it makes sense.

Author: [hidden] (pokoli) Tryton committer Tryton translator
Date: 2022-01-11.17:10:01

Is there any plan to activate the feature by default on some modules?

Author: [hidden] (ced) Tryton committer Tryton translator
Date: 2022-01-11.17:01:46

We often put in the queue a method call with a list of records. But often there is no strict need that all the records are being processed in the same transaction nor at the same time.
We can have performance issue when the list of records is very large (and if the method take locks on table or many records) and it takes time to process all of them. So often the system will be more reactive if if we process small batch of records instead of all of them.
So I propose to add a new contextual parameters to __queue__ calls which allow the queue to create many tasks (batches) for one call. The parameter will allow to just activate the feature (with a default batch size) or define the size for the batch. Of course the division will only happen if there are workers otherwise there is no benefit.

History
Date User Action Args
2022-01-11 19:20:16restevesetnosy: + resteve
2022-01-11 17:22:38reviewbotsetmessages: + msg73200
nosy: + reviewbot
2022-01-11 17:17:17cedsetkeyword: + review
messages: + msg73199
reviews: 381661002
status: in-progress -> testing
2022-01-11 17:10:01pokolisetmessages: + msg73198
nosy: + pokoli
2022-01-11 17:01:46cedcreate

Showing 10 items. Show all history (warning: this could be VERY long)