ANOVA

We will discuss about testing of hypothesis where a specific  hypothesis is stated about a population or universe parameter, and sample statistics are used to assess the likelihood that the hypothesis is true. The hypothesis is based on available information and the investigator's belief about the population parameters.

The specific test considered here is called ANOVA or Analysis of Variance. It is a test of hypothesis or a statement of researcher that is appropriate to compare means of a continuous variable in two or more independent comparison groups. It may be comparision of two comparision groups one being given a new drug for treatment of malaria medicated group a to a placebo and to a standard treatment. A study may compare mean blood pressure or a mean cholosteral level in persons of different weight groups. Under weight, normal weight and over weight.

ANOVA technique may be applied when there are two or more than two comparision groups. 

The test statistic must take into account the sample sizes, sample means and sample standard deviations in each of the comparison groups.

The fundamental strategy of ANOVA is to systematically examine variability within groups being compared and also examine variability among the groups being compared.

The ANOVA approach

Suppose we wish to compare the mean systolic blood pressure according to the Body Mass Index (BMI) we devide these groups in to four  categories (Under weight Normal, Over weight and Obese) let us denote these groups by A, B, C and D.

we wish to test whether there is a statistically significant difference in mean systolic blood pressures among the four groups.

The sample data are organized as follows:

           Group 1 Group 2              Group 3             Group 4

Sample Size n1             n2                   n3                           n4

Sample Mean  X1 X2                   X3                            X4

Sample Standard 

Deviation           s1                 s2                  s3                            s4

The hypotheses of interest in an ANOVA are as follows:

• H₀: μ₁ = μ₂ = μ₃ ... = μ_k
• H₁: Means are not all equal.

where k = the number of independent comparison groups.

In this example, the hypotheses are:

• H₀: μ₁ = μ₂ = μ₃ = μ₄
• H₁: The means are not all equal.
• The null hypothesis in ANOVA is always that there is no difference in means. The research or alternative hypothesis is always that the means are not all equal and is usually written in words rather than in mathematical symbols. The research hypothesis captures any difference in means and includes, for example, the situation where all four means are unequal, where one is different from the other three, where two are different, and so on. The alternative hypotheses, as shown above, capture all possible situations other than equality of all means specified in the null hypothesis.
  Test Statistic for ANOVA
  The test statistic for testing H₀: μ₁ = μ₂ = ... =   μ_k is:
  and the critical value is found in a table of probability values for the F distribution with (degrees of freedom) df₁ = k-1, df₂=N-k.
  In the test statistic, n_j = the sample size in the j^th group (e.g., j =1, 2, 3, and 4 when there are 4 comparison groups),   is the sample mean in the j^th group, and   is the overall mean.  
  
  
  k represents the number of independent groups (in this example, k=4), and
  N represents the total number of observations in the analysis.
  
  
  Note that N does not refer to a population size, but instead to the total sample size in the analysis (the sum of the sample sizes in the comparison groups, e.g., N=n₁+n₂+n₃+n₄).
  
  
  The test statistic is complicated because it incorporates all of the sample data. While it is not easy to see the extension, the F statistic shown above is a generalization of the test statistic used for testing the equality of exactly two means.  
• Note:
• The test statistic F assumes equal variability in the k populations (i.e., the population variances are equal, or s₁² = s₂² = ... = s_k² ). This means that the outcome is equally variable in each of the comparison populations.
• The F statistic is computed by taking the ratio of what is called the "between treatment" variability to the "residual or error" variability.
• In analysis of variance we are testing for a difference in means (H₀: means are all equal versus H₁: means are not all equal) by evaluating variability in the data.
• The numerator captures between treatment variability (i.e., differences among the sample means) and the denominator contains an estimate of the variability in the outcome.
• The test statistic is a measure that allows us to assess whether the differences among the sample means (numerator) are more than would be expected by chance if the null hypothesis is true. 
• The decision rule for the F test in ANOVA is set up in a similar way to decision rules we established for t tests. The decision rule again depends on the level of significance and the degrees of freedom.

 The F statistic has two degrees of freedom. These are denoted df₁ and df₂, and called the numerator and denominator degrees of freedom, respectively. The degrees of freedom are defined as follows:

df₁ = k-1 and df₂=N-k,

where k is the number of comparison groups and N is the total number of observations in the analysis.  

If the null hypothesis is true, the between treatment variation (numerator) will not exceed the residual or error variation (denominator) and the F statistic will small.

The rejection region for the F test is always in the upper (right-hand) tail of the distribution as shown below.

Rejection Region for F   Test with a =0.05, df₁=3 and df₂=36 (k=4, N=40)

The decision rule is: Reject H₀ if F > 2.87.

For solved example see my another post.