Semi-Summary of Week 1 discussion

In our group discussion of the week 1 statistical test (two-sample t-test with unequal variances), we discussed normality tests and a few other things:

 

We decided that for our purposes, the Shapiro-Wilk test was a good test for normality. The test spits out a statistic W, which in words I can describe as follows: W is the weighted sum of the numbers in our data set, squared, divided by the sum of the squared differences between the numbers in out data set and the mean (essentially the variance). The weighting coefficient takes into account the expected value for the i-th smallest number in a randomly chosen sample from the normal distribution and the covariance between two such numbers. 

We discussed Q-Q plots as a useful visual representation of normality. The logic of a Q-Q plot is that it compares the quantiles of two distributions against each other. In our case, you would plot the 99th percentile of our data against the 99th percentile of the standard normal distribution (I presume with the same mean...) etc. If the data are normally distributed, your plot should be a straight line. One advantage of this information is you can easily spot outliers. 

A question we had is: is there a p-value for the two-sample t-test that is so low that it is completely unreasonable, such that we should take it as a signal that we used the wrong test? To put this in context, the p-values we obtained were on the order of 10^-9 and 10^-16 (in Mathematica and R, respectively), and we're not sure if these seem right. 

Regarding our actual test of whether the means were different in our two data sets: we used a two-tailed t- test (or at least we should have!) because we considered it significant if one class differed from the other in either direction. Had we been interested in whether the mean score of class B was greater than the mean score of class A, we would only consider deviation in the positive direction and thus we should use a one-tailed t-test.