WINKS Manual -- Chapter 5, Part 5

WINKS Online Manual

Chapter 5 Part 5

Multiple Comparisons Analysis

When you perform an analysis of variance test, the F-test (main effects) usually is testing for the presence of a difference between factor means. However, if the test is significant, you do not necessarily know which of the several means can be considered significantly different. Some people have tended to perform multiple t-tests to examine the differences between a number of means; however, this approach is incorrect since the p-values associated with multiple tests are no longer appropriate. Multiple comparison tests are designed to allow you to perform differences between all possible pairs of means using an appropriate and controlled significance level.

WINKS allows you to perform four different kinds of multiple comparison tests. They are:

Newman-Keuls (Newman, 1939, Keuls, 1952)
Tukey (Tukey, 1953)
Scheffé (Scheffé, 1953; 1959)
Dunnett (Dunnett, 1955)

The Newman-Keuls and Tukey tests are commonly used multiple comparison procedures. The Tukey multiple comparison procedure is also known as the "honestly significant difference test" or HSD test. The Scheffé test is a conservative test that is often used for multiple contrasts. Although not covered directly in WINKS, you can use the information in the Scheffé tests to perform multiple contrasts. Dunnett's test is a specialized multiple comparison test that allows you to compare a single control group to all other groups.

The test you use depends, in part, on your discipline. Some areas of research prefer one multiple comparison test over another. You should consult your literature to see which of these tests is most often used. There are other multiple comparison tests not provided in WINKS, but the ones included are widely known and accepted in most disciplines. During the setup procedure, WINKS allows you to choose between the Newman-Keuls, Tukey and Scheffé tests as the default test for multiple comparisons following analysis of variance procedures. For some procedures, such as the non-parametric comparison tests, the Tukey comparisons are used always.

However, if you want to perform other multiple comparison tests than those provided in other WINKS procedures, you can access the Multiple Comparison tests directly in this module.

Note: When you begin the Multiple Comparison and request one of the comparison types, if you have recently performed an analysis of variance, the data from the last multiple comparison will appear in a table. This allows you to easily perform a multiple comparison test using a different comparison procedure than the one used by default in WINKS.

Example Multiple Comparison - Tukey Analysis

Suppose you have the following data:

Group   Mean       N
1         61.03       4
2         70.13       4
3         89.07       3
4         85.78       4

MSE = 14.74 and D.F. = 11

and you want to perform a Tukey Multiple Comparison analysis. Follow these steps:

Step 1: From the Analyze menu select "Multiple Comparisons." You will be prompted to enter the table size. For this example, enter 4.

Step 2: Enter the data as listed above, including, the MSE and D.F. Select Tukey as the analysis type. See figure 1.

Figure 1

Step 3: Choose Calculate, then Exit to display the results. The results will be displayed in the viewer. A portion of the output is shown below.


Error term used for comparisons = 14.74 with 11 d.f.

							   Critical q
Tukey Multiple Comp. 		Difference 	Q 	   (.05)
------------------------------------------------------------------------
Mean(3)-Mean(1) = 		28.04 		13.523 	   4.256 *
Mean(3)-Mean(2) = 		18.94 		 9.135 	   4.256 *
Mean(3)-Mean(4) = 		 3.29 		 1.587 	   4.256
Mean(4)-Mean(1) = 		24.75 		12.893 	   4.256 *
Mean(4)-Mean(2) = 		15.65 		 8.153 	   4.256 *
Mean(2)-Mean(1) = 		  9.1 		  4.74 	   4.256 *

Homogeneous Populations, groups ranked 

Gp Gp Gp Gp
1   2  4  3
      ------
--- 
   --- 

This is a graphical representation of the Tukey multiple comparisons
test. At the 0.05 significance level, the means of any two groups
underscored by the same line are not significantly different.

The results are similar to those described in the One-Way Analysis of Variance example Since the techniques are different, the results of the Newman-Keuls, Tukey and Scheffé tests may vary.

Example Multiple Comparison - Dunnett's Analysis

Using the same data as above, suppose group 1 is actually a control group, and you want to perform all comparisons with that group using the Dunnett's test. Follow these steps:

Step 1: From the Analyze menu select "Multiple Comparisons." You will be prompted to enter the table size. For this example, enter 4.

Step 2: Enter the data as listed on page 5-50, including, the MSE and D.F. Select Dunnett's as the analysis type.

Step 3: Choose Calculate. A list of the four groups will appear on the screen. Select group 1 and OK. Then choose Exit to display the results.

Step 4: The results will appear in the WINKS viewer where you can examine them or print them.

Note: When you do two or more analyses in the same "run" each analysis will be appended to the previous analysis in the WINKS viewer.

Continue to References