We have WAP deployment with 800+ subscriptions. It's public cloud, all tenants subscribed to one plan. This plan have 60 templates and 16 hardware profiles allowed.
We're experiencing performance issues with subscription sync - when we change anything in plan or new tenant subscribed.
First of all, subcriptions synchronized sequentially, not parallel. One sync ("Change properties of user role" job in VMM) takes ~1.5min. So full synchonization process of 800 subscriptions will take ~1day, which is totally unacceptable.
And second and more serious problem. Subscription sync process causes large CPU and network load for VMM and VMM DB. Network can be loaded up to 300Mbps (send+receive) on VMM and DB when only one subscription synchronizing. Oh, our VMM cluster nodes have 8CPU and 16GB memory, and 12CPU-32GB for DB nodes.
As I understand, the problem is related to large read-write operations in TR_Task*, TR_Subtasks* tables in VMM DB.
Screenshot from VMM jobs:
![]()
That's how it looks in the SQL Profiler at the same time:
![]()
It's filtered by only one of calls, of course there are other calls. (updates to Tasks for example)
And screenshot from SQL Studio:
![]()
In this query first "SubtaskParamsXml" have ~12MB of data and ~200KB in other rows. So one procedure call causes ~25MB transfer.
This procedure called for each subtask update twice ~= 160 calls. Totally we have ~4GB data transfer for just one subscription sync (!). And all this data processed by VMM and DB...
I hope for some workaround or fix.
We're experiencing performance issues with subscription sync - when we change anything in plan or new tenant subscribed.
First of all, subcriptions synchronized sequentially, not parallel. One sync ("Change properties of user role" job in VMM) takes ~1.5min. So full synchonization process of 800 subscriptions will take ~1day, which is totally unacceptable.
And second and more serious problem. Subscription sync process causes large CPU and network load for VMM and VMM DB. Network can be loaded up to 300Mbps (send+receive) on VMM and DB when only one subscription synchronizing. Oh, our VMM cluster nodes have 8CPU and 16GB memory, and 12CPU-32GB for DB nodes.
As I understand, the problem is related to large read-write operations in TR_Task*, TR_Subtasks* tables in VMM DB.
Screenshot from VMM jobs:
That's how it looks in the SQL Profiler at the same time:
It's filtered by only one of calls, of course there are other calls. (updates to Tasks for example)
And screenshot from SQL Studio:
In this query first "SubtaskParamsXml" have ~12MB of data and ~200KB in other rows. So one procedure call causes ~25MB transfer.
This procedure called for each subtask update twice ~= 160 calls. Totally we have ~4GB data transfer for just one subscription sync (!). And all this data processed by VMM and DB...
I hope for some workaround or fix.