I'd like a clarification on CS planner algorythms

[akrasnov-drv]

Hi,
I have quite a weird problem.
I want each new VM to start on another host.
I tried using random planner, then switched to userdispersing, added the setting also to storage, everything to no avail.
VMs are created on the same host in blocks, e.g. 5 VMs on the same host in a raw, then another host, but keeping use just of this 2. And just after some time it continues to additional host.
I have no problem to start VM on particular host by specifying it in VM deployment. But if I allow CS to choose, it does the above.
If it changes anything, I have single cluster and single pod, and I use local storage for VMs.

Here are settings I tried:

• vm.allocation.algorithm (random, userdispersing)
• vm.deployment.planner (userdispersing)
• deployment.planners.exclude (removed some defalut values keeping just SkipHeuresticsPlanner)
• deployment.planners.order (moved UserDispersingPlanner to be the 1st)
• image.store.allocation.algorithm (random)

Please explain/advise.

[rajujith]

@akrasnov-drv vm.allocation.algorithm = userdispersing should work. No need to change the vm.deployment.planner. You also try anti-host affinity in affinity groups.

[akrasnov-drv]

Unfortunately I can't use anti-affinity groups as jCloud plugin does not support using that.

I already have vm.allocation.algorithm = userdispersing and just now got 9 of 10 new VMs on the same host (cluster had 5 hosts and just 1 VM running)

What other parameters influence on host choice?
Maybe there is a bug?

[akrasnov-drv]

Oh, there is new version.
I'm still on 4.20.1.0. It has just random for volume, and it did not help.
I tried setting volume to random, and VM to userspreading.
Then 4 VMs I started within some 10 minutes were placed to the same host 😢

[akrasnov-drv]

I

• upgraded to 4.21
• set vm.allocation.algorithm to userdispersing
• set volume.allocation.algorithm to userdispersing
• set vm.deployment.planner to UserDispersingPlanner
• recreated service offerings with Deployment planner set to UserDispersingPlanner

and nothing helped
I got 20 VMs started on the same host while I have 12 hosts now
Looks like it tries to fill a host before moving to the next one.

[akrasnov-drv]

I did several more tests. Seems the issue is in timing. If I start 1 VM in 3 minutes, then spreading works.
But if I ask to start additional VM while the previous are in "Starting" state, all are started on the same host.
Not sure though, if host usage is updated by some timer, or when VM switches to "Running" state.
I believe "Starting" case should be considered by algorithms the same way as " Running".

[akrasnov-drv]

Finally I moved back to random, it's not perfect but works a lot better in my case, and I see a change comparing to 4.20.1 here.
Previously random did not help. Likely option to place volume randomly fixed it.
Nevertheless it would be great to have 'userdispersing' considering all nodes assigned to a host, not just running ones.

[sudo87]

vm.allocation.algorithm = userdispersing algorithm takes only Running Instances into consideration for allocation.

Query used in the allocation logic indicates the same:

SELECT host.id, SUM(IF(vm.state='Running' AND vm.account_id = ?, 1, 0)) FROM `cloud`.`host` host LEFT JOIN `cloud`.`vm_instance` vm ON host.id = vm.host_id WHERE host.data_center_id = ? AND host.type = 'Routing' AND host.removed is null 

It explains the scenario and outcome of uneven distribution of multiple instances if created in short span of time.

I am not sure about considering "Starting" state instances for allocation decision, as Host allocation is not finalized until Instance is in Running State. However feel free to open an improvement issue.