Tryton - Issues

 

Issue7406

Title Transactional queue
Priority feature Status resolved
Superseder Allow expression in index_action
View: 7580
Nosy List Timitos, ced, pokoli, resteve, reviewbot, roundup-bot
Type feature request Components proteus, trytond
Assigned To ced Keywords review
Reviews 45461002
View: 45461002

Created on 2018-04-30.18:44:11 by ced, last changed by roundup-bot.

Messages
New changeset dcb8be191b25 by Cédric Krier in branch 'default':
Add transactional queue and workers
https://hg.tryton.org/tryton-env/rev/dcb8be191b25
New changeset 8b8103041310 by Cédric Krier in branch 'default':
Add transactional queue and workers
https://hg.tryton.org/proteus/rev/8b8103041310
New changeset b436891a6754 by Cédric Krier in branch 'default':
Add transactional queue and workers
https://hg.tryton.org/trytond/rev/b436891a6754
review45461002 updated at https://codereview.tryton.org/45461002/#ps300002
review45461002 updated at https://codereview.tryton.org/45461002/#ps300001
review45461002 updated at https://codereview.tryton.org/45461002/#ps280001
msg42335 (view) Author: [hidden] (ced) (Tryton committer) (Tryton translator) Date: 2018-07-25.00:21:59
Last review include the creation of the index.
review45461002 updated at https://codereview.tryton.org/45461002/#ps260001
review45461002 updated at https://codereview.tryton.org/45461002/#ps240001
review45461002 updated at https://codereview.tryton.org/45461002/#ps220001
review45461002 updated at https://codereview.tryton.org/45461002/#ps200001
msg42159 (view) Author: [hidden] (ced) (Tryton committer) (Tryton translator) Date: 2018-07-13.13:03:18
I do not think we should show the queue. It is a technical thing which may even contain sensitive data.
About the cleaning, I think it is good to have the history, just like we keep all the _history records, the ir.trigger.log or the notification.email.log
If it is about the performance, there is a TODO about creating an index with nulls first. But it can be manually created (as it is our trends to minimize database customization by default).
msg42158 (view) Author: [hidden] (pokoli) (Tryton committer) (Tryton translator) Date: 2018-07-13.12:46:31
I'm wondering if it will be great to add an Action on the user Interface to see the ir.queue module with the following domains: Pending (dequed is null) / Running (Dequed is not null but finished is null) and All. 

As the queue is an append only model, maybe we should also add a wizard or cron job to clean older entries.
msg42141 (view) Author: [hidden] (ced) (Tryton committer) (Tryton translator) Date: 2018-07-11.17:16:49
I plan to commit this in the next week if I get no new comments.
review45461002 updated at https://codereview.tryton.org/45461002/#ps180001
review45461002 updated at https://codereview.tryton.org/45461002/#ps160001
review45461002 updated at https://codereview.tryton.org/45461002/#ps140001
review45461002 updated at https://codereview.tryton.org/45461002/#ps120001
review45461002 updated at https://codereview.tryton.org/45461002/#ps120001
review45461002 updated at https://codereview.tryton.org/45461002/#ps120001
review45461002 updated at https://codereview.tryton.org/45461002/#ps120001
review45461002 updated at https://codereview.tryton.org/45461002/#ps100001
review49371002 updated at https://codereview.tryton.org/49371002/#ps1
msg41002 (view) Author: [hidden] (ced) (Tryton committer) (Tryton translator) Date: 2018-05-24.13:09:12
proteus needs also to run tasks: review49371002
review45461002 updated at https://codereview.tryton.org/45461002/#ps80001
review45461002 updated at https://codereview.tryton.org/45461002/#ps60001
review45461002 updated at https://codereview.tryton.org/45461002/#ps40001
review45461002 updated at https://codereview.tryton.org/45461002/#ps20001
New review45461002 at https://codereview.tryton.org/45461002/#ps1
msg40489 (view) Author: [hidden] (ced) (Tryton committer) (Tryton translator) Date: 2018-04-30.18:53:29
Here is an WIP of the implementation: review45461002
It is already working. I publish it to get feedback's on the API and some choices.
Principal choices are:

- Usage of advisory lock to pull tasks
- Usage of listen/notify when available
- Launch task at the end of requests if no worker is configured (but reuse as much as possible worker code)
- Use multiprocess as pool of worker
- Model.__queue__ API which is restricted to have record(s) as first argument (instance method or class method)
- Use scheduled and expected timestamp as priority of the queue
- Allow to run worker for a specific queue name


The next step will be to update modules to use it when possible.
msg40488 (view) Author: [hidden] (ced) (Tryton committer) (Tryton translator) Date: 2018-04-30.18:44:10
From https://discuss.tryton.org/t/transactional-queue/461
History
Date User Action Args
2018-07-27 09:33:51roundup-botsetmessages: + msg42385
2018-07-27 09:33:31roundup-botsetmessages: + msg42378
2018-07-27 09:33:22roundup-botsetstatus: testing -> resolved
nosy: + roundup-bot
messages: + msg42377
2018-07-25 16:35:36reviewbotsetmessages: + msg42346
2018-07-25 13:40:50reviewbotsetmessages: + msg42340
2018-07-25 11:01:26reviewbotsetmessages: + msg42338
2018-07-25 00:21:59cedsetmessages: + msg42335
2018-07-24 22:38:18reviewbotsetmessages: + msg42334
2018-07-19 15:51:22cedsetsuperseder: + Allow expression in index_action
2018-07-19 10:00:44reviewbotsetmessages: + msg42209

Showing 10 items. Show all history (warning: this could be VERY long)