Yarn capacity-scheduler queue and app sorting policy

capacity-scheduler scheduling

daemon thread to pick up the candidates nodemanagers. see: #Attachment#scheduling entry point
and then use the single nodemanager to find out the candidate queue + apps
- find out the priority children queues. see #Attachment#queue getting after sorting
  1. first factor: priority
  2. second factor: abs used capacity (The queue with less relative used-capacity goes first)
- find out the sorted apps submitted to upper queue. see: #Attachment#app getting after sorting in leaf queue
  1. getting the apps from the specified sorting policies

Question

queue selection is selected by the abs used capacity, when to sub/add used capacity

Only when the container is allocated successfully in the async scheduling commit process, the allocated resource will be added into the queue's usage resources. Please see Attachment#Resource deductions associated code

BTW, if the container is allocated, it will notify the app's ordering policy to reorder the associated app. Detailed code is leaf queue.

orderingPolicy.containerAllocated(  
schedulerContainer.getSchedulerApplicationAttempt(),  
schedulerContainer.getRmContainer());

Is it possible that race condition happens when multi thread to propose commit for the same apps or queue

I think yes, multi threads will choose the same queue and then to get the same apps to allocate.
From the sorting policy, I don't see any avoidance strategies.