The ability to run tests in parallel is one of the key features of using OTS.
Test Managers in production environments are keen to ensure that the throughput of the Testruns is optimal.
The Workers perform three primary functions
So time spent on a Testrun is:
dT = Flashing_t + Booting_t + SUM(Testing_t)
When OTS receives a Testrun request it distributes it to a Worker. The Worker runs through the sequence above on all Test Packages.
This is the simplest and in many respects the most efficient way of running the Testrun.
If we have a Testrun with several packages and we wanted it processed as quickly as possible we can exploit the parallelism in a different way. By running the Packages on different Workers this Testrun *might* be processed more quickly.
It *might* because there are other factors at play:-
If more than one Testrun takes the same approach there is a good chance it will slow things down. Flashing is time consuming so the maximising the TestPackages that get run per flash is a good policy if all the Testruns are to be treated fairly.
So what works quickest is dependent on many factors. For example:
If a Testrun with many packages is known to take a long time and a single instance of this is run alongside a few Testruns that take a short amount of time then the total time for all the Testruns *might* be reduced by splitting the packages (see next section).
So this approach scales badly, splitting the packages for all Testruns will reduce throughput on a busy Worker farm.
Consider a Testrun made of a n of Packages:
dT_total_testrun = testrun_t1 + testrun_t2 + ..... + testrun_tn
We might consider dividing up the Testrun up into a number of Tasks of even length so that the no_of_tasks = no_of_workers
So the Task time is now approximately:
dT_task = flash_time + boot_time + (dT_total_testrun / no_of_workers)
There is no guarantee that this will give the speed-up that we are looking for. In many cases when a Testrun is added to the Queue there will already be Tasks in process on some of the Workers.
If the time remaining for the Tasks on the queues is significant (i.e. > dT_task ), then a proportion of the split tasks will run in serial rather than parallel. Running in serial means that flash_time and boot_time must now be repeated! Potentially we could be slowing down the very run that we want to speed up by taking this approach.
The parameters that affect throughput are:
Looking at the parameters...
Considering the examples of the different scenarios that might arise as sketched out above it is evident that the optimal system will depend on the usage model and the needs of the Test Manager.
At this point OTS will provide an API that allows Plugins for alternative Distribution Systems for your particular needs based on these Parameters