Optimal Design of Experiments

(Version: September 2021)

Application Example

The following application example illustrates the benefits of experimental design as part of sample size determination and thus preparation of data analysis. In a study, the manufacturer of an ointment wants to find out what influence the drug has on the reduction of age spots, taking into account other factors. On the one hand, the study should deliver statistically reliable results, on the other hand, the number of test subjects should be minimal.

In addition to the ointment it is assumed that age, gender, skin type and dietary supplements are also influencing factors. The success is defined by a desired equalisation of the skin's colour tone within a predetermined period of treatment. The task is to plan this study with an adequate number of subjects in different combinations of factors and corresponding group sizes.

In the following, the methodological background for solving this purely data-based task is briefly presented and subsequently used for the application example.

Methodological Background

In many areas of research, development and operation (production and administration), there is not only a lack of sufficient knowledge about the behaviour of a system. There is also a constant desire to optimise products and processes. In these cases, one possibility is to carry out experiments or trials to obtain measurement data for a data analysis and thus describe these systems.

Statistical design of experiments comprises a variety of procedures so that such experiments can be planned in a methodical way for the analysis of certain relationships or for optimisation, [1,2,3,4]. This also means that design of experiments is influenced by assumptions about the system and thus by the subsequent data analysis of the experiments. At the same time, another task is to optimise the experimental design itself in order to reduce the number of experiments for various reasons. Basically, within the experiments, the assumed input variables, also called "factors", are set to certain "steps" (also: "levels") and the resulting output variable(s) are measured.

The actual method used depends on the respective state of knowledge and can also have an iterative character. Depending on the information available and the objective, the following small selection of method classes can be mentioned: (1)  "Factor Screening" (i.e. cause-effect relationships) with the sub-methods (i)  "one factor at a time", (ii) full factorial design and (iii) fractional factorial design, (2) more complex models as well as optimisation of the target variables (interactions, block formation and non-linearities, especially "response surface methods") as well as (3) the reduction of variance (especially with methods according to Taguchi).

For the application example mentioned above, an experimental design is to be carried out with the methods mentioned under (1). Similar to its non-methodological alternative "trial and error", "one factor at a time" has the disadvantage of imbalance and ineffectiveness (more individual trials with the same statistical properties or more replicates with an apparently smaller number of trials). In addition, the effects of interactions between inputs on outputs are not taken into account.

Full factorial design, i.e. combinations of all levels of all factors, usually leads to an unfeasible large number of experiments. Therefore, in the continued application example, the frequently used fractional factorial design will be presented. This can considerably reduce the number of necessary experiments (with the disadvantage of mixing main effects and interactions).

Continuation Application Example

For the application example introduced above, the factor "age" is to be excluded in the course of the experimental planning: on the one hand, it is not obvious in this case to transform this continuous variable into a discrete one with a few levels, and on the other hand, it may be advantageous under certain circumstances for the recruitment of the study participants. This leaves 4 more factors for which the client suggests the following levels to be investigated in this scenario: ointment (yes/no), gender (m/f), 6 different skin types (according to Thomas Fitzpatrick, 1975) as well as food supplements (none, selenium, vitamin E). The full combinatorics thus results in a number of experiments $N=2\times2\times6\times3=72$ (view here).

These specifications already allow a reduction to a fractional factorial plan in the form of an orthogonal array, as published in a good overview by Kuhfeld and available in the R package DoE.base, [5, 6, 7]. This reduces the number of necessary experiments from 72 to 36 (view here). However, another effect of this method can be used here, which probably has its origin in number-theoretical properties: a certain increase in the number of levels leads to an increased full combinatorics but to a lower number of experiments in the fractional factorial plan. Thus, in the present scenario, one more food supplement can be added: Astaxanthin. This full combinatorics would have $N=2\times2\times6\times4=96$ experiments (view here). In the reduced fractional factorial plan, on the other hand, the following 24 experiments are sufficient:

# treatment sex skintype nutrsuppl
1 FALSE m I 0
2 FALSE m II selenium
3 FALSE m III astaxanthin
4 FALSE m IV 0
5 FALSE m V vitE
6 FALSE m VI vitE
7 FALSE f I astaxanthin
8 FALSE f II 0
9 FALSE f III vitE
10 FALSE f IV selenium
11 FALSE f V astaxanthin
12 FALSE f VI selenium
13 TRUE m I selenium
14 TRUE m II vitE
15 TRUE m III 0
16 TRUE m IV astaxanthin
17 TRUE m V selenium
18 TRUE m VI astaxanthin
19 TRUE f I vitE
20 TRUE f II astaxanthin
21 TRUE f III selenium
22 TRUE f IV vitE
23 TRUE f V 0
24 TRUE f VI 0

Even with the experiments of this orthogonal array, all information is collected: all levels occur in the same number and all pairwise information of the levels exists. With this experimental design, the sample size determination can be continued with replicates while subsequently an event time analysis can be performed to evaluate the ointment.


  1. [1] R. A. Fisher: The Design of Experiments. 9. Auflage. New York: Hafner Press, Macmillan Publishers, 1971. ISBN: 978-0028446905.
  2. [2] D. Rasch, J. Pilz, R. Verdooren, A. Gebhardt: Optimal experimental design with R. Boca Raton: Chapman und Hall, 2011. ISBN: 978-1439816974.
  3. [3] D. Montgomery: Design and Analysis of Experiments. 8. Auflage. Hoboken: John Wiley & Sons, 2013. ISBN: 978-1118146927.
  4. [4] W. Kleppmann: Versuchsplanung. Produkte und Prozesse optimieren. 10. Auflage. München: Carl Hanser Verlag, 2020. ISBN: 978-3446461468.
  5. [5] W. F. Kuhfeld: Orthogonal Array Lists. SAS Institute. url: https://support.sas.com/techsup/technote/ts723.pdf
  6. [6] W. F. Kuhfeld: Orthogonal Arrays. SAS Institute. 2019. url: https://support.sas.com/techsup/technote/ts723b.pdf
  7. [7] U. Grömping: R Package DoE.base for Factorial Designs. In: Journal of Statistical Software, 85(5):1–41, 2018. doi: 10.18637/jss.v085.i05