I am not able to build the code, but from the initial scans, it looks like you are using Lock-free constructs for the queuing.
Lock-free constructs may help in heavy throughput scenarios, where continuous streams are available for hand-off. But most of the times, in production, load comes sporadically. In such cases (where load is less 80% of the time), the threads will be ticking unnecessarily (due to their lock free nature).
A more reasonable approach in such cases would be, to gracefully degrade from lock-free waiting to locked-waiting (if no new work-item or message appears for few iterations). Please give it a thought.
Thanks.
I am not able to build the code, but from the initial scans, it looks like you are using Lock-free constructs for the queuing.
Lock-free constructs may help in heavy throughput scenarios, where continuous streams are available for hand-off. But most of the times, in production, load comes sporadically. In such cases (where load is less 80% of the time), the threads will be ticking unnecessarily (due to their lock free nature).
A more reasonable approach in such cases would be, to gracefully degrade from lock-free waiting to locked-waiting (if no new work-item or message appears for few iterations). Please give it a thought.
Thanks.