A Monte Carlo sampling experiment is carried out with 200 replications for J = 5, n = 13, and parameter values of µ1 = µ2 = µ3 = 64, µ4 = µ5 = 49, and σ = 10.
On each replication of the sampling experiment, .05-level t-tests and .05-level q* tests (Tukey tests) of H0: µj – µj’ (j ≠ j’) have been conducted for each comparison, along with 95% individual-t confidence intervals and 95% Tukey simultaneous confidence intervals.
The data for this sampling experiment can be found in the file PSYC3001 Assignment 1 2021.sav, where each row refers to a single trial (replication) consisting of the following 105 variables:
|1||Trial||Replication (0001 to 2000|
|2-6||M1, M2, M3, M4, M5||Sample means|
|7-16||M1_M2, M1_M3, … M4_M5||Sample comparisons|
|17-21||SS1, SS2, SS3, SS4, SS5||Within group sums of squares|
|22-31||sigt1_2, sigt1_3, sigt1_4, sigt1_5, sigt2_3, sigt2_4, sigt2_5, sigt3_4, sigt3_5, sigt4_5||Significant (sigt = 1) or nonsignificant (sigt = 0) .05-level t-test for each comparison|
|32||T1errors_t||Number of Type I errors (for that trial) for the t-test method|
|33-52||ll_t1_2, ul_t1_2, ll_t1_3, ul_t1_3, ll_t1_4, ul_t1_4, … etc … ll_t4_5, ul_t4_5||Lower (ll) and upper (ul) 95% tbased CI limits for each comparison|
|53-62||noncovert1_2, noncovert1_3, noncovert1_4, noncovert1_5,
… etc … noncovert4_5
|Noncoverage error (noncovert =
1) OR no noncoverage error (noncovert = 0) for each comparison t-based CI
|63||NonCovErrors_t||Number of noncoverage errors
(for that trial) for t-based Cis
|64-73||sigq1_2, sigq1_3, sigq1_4, sigq1_5, sigq2_3, sigq2_4, sigq2_5, sigq3_4, sigq3_5, sigq4_5||Significant (sigq = 1) or nonsignificant (sigq = 0) .05-level Tukey Test for each comparison|
|74||T1errors_q||Number of Type I errors (for that trial) for Tukey method|
|75-94||ll_Tuk1_2, ul_Tuk1_2, ll_Tuk1_3, ul_Tuk1_3, ll_Tuk1_4, ul_Tuk1_4, … etc … ll_Tuk4_5, ul_Tuk4_5||Lower (ll) and upper (ul) Tukey
95% CI limits for each comparison
|95-104||noncoverTuk1_2, noncoverTuk1_3, noncoverTuk1_4, noncoverTuk1_5, noncoverTuk2_3, … etc … noncoverTuk4_5||noncoverTuk has value =1
(noncoverage error) or value = 0
(no noncoverage error) for each comparison Tukey 95% CI
|105||NonCovErrors_Tuk||Number of noncoverage errors (for that trial) for Tukey CIs|
For all parts of this question, consider only the comparisons with a true H0 (i.e., comparisons where there is zero difference between the two population means). Include all relevant SPSS output (i.e., tables) with your answer.
- Use SPSS to calculate the average value (across all 2000 trials) for each sample comparison. Comment on these values.
- Use SPSS to select a subset of 200 out of the 2000 total trials. Use the last three digits of your student number and then add 199 to set the first and last cases. [Hint: go to Data – Select Cases
– Based on time or case range – Range – First Case = (last 3 digits of student number); Last Case = (First Case + 199) – Continue – OK].
For this subset of replications, calculate the average value for each comparison. Comment on the difference between these values and the values obtained in part A-i.
Parts B and C refer to all 2000 trials in the sampling experiment [Hint: go to Data – Select Cases – All cases – OK]
- Use SPSS to calculate the FWER and the FWNCER for the t-based method. Provide a general definition for each of these terms and comment these values in relation to alpha and each other.
- Use SPSS to calculate the FWER and the FWNCER for the Tukey method. Comment on these values in relation to each other and the values obtained in part B.
Refer to Generating Trial Numbers for PSYC3001 Assignment 1 posted in the Assignments section on Moodle. You will use the first of the two generated trial numbers for this question – make sure you state the trial number at the beginning of your answer to this question.
Within the trial that you arrived at via the above, pick two comparisons where:
- It must be possible for a Type I error to occur for one of these comparisons.
- It must be possible for a Type II error to occur for the other comparison.
For each of the above comparisons within your selected trial:
- State the comparison you have chosen and the t-based confidence interval limits for this comparison.
- Do these confidence interval limits reflect the occurrence of a noncoverage error? Explain why or why not. Do these confidence interval limits reflect the occurrence of any other inferential error? Explain why or why not.
- State whether it is possible for there to be a confidence interval for the comparison you have picked where you would give the same answer for (i) but a different answer for (ii). If it is possible, provide an example of such a confidence interval (you can make up the limits or find one from the provided dataset) along with a brief explanation. If it is not possible, explain why such a confidence interval cannot exist.
Note: your answer to the above should have 8 parts – i, ii, iii and iv for each of A and B.
Refer to Generating Trial Numbers for PSYC3001 Assignment 1. You will use the second of the two generated trial numbers for this question – make sure you state the trial number at the beginning of your answer to this question.
- Provide a brief substantive context for the J = 5 experiment. That is, describe the experimental factor (IV) and the 5 levels (say what they are) as well as the dependent measure (DV).
- Identify the maximal comparison in this trial and show the calculation for the limits of the Tukey 95% simultaneous confidence interval.
- Provide a substantive conclusion for the confidence interval in part a.
- From the same trial, select a comparison (other than the maximal comparison) for which the Tukey confidence interval contains 0. State the selected comparison and the limits of this confidence interval. Provide an appropriate conclusion for this confidence interval.
- Suppose that the Tukey 95% simultaneous confidence interval for the maximal comparison of a given trial contains 0. What does this imply about the confidence intervals for the other comparisons in the same trial? Explain your answer.
- There is no word limit for this assignment. However, you should find that you are able to answer the questions in full with about 4-5 typed pages (1.5 spacing) of written answers and calculations. This excludes tables and SPSS output.
- You should create a single word or pdf document for upload to the Turnitin submission link on Moodle.
- Remember to include all relevant SPSS output in your assignment (copy and paste SPSS tables into the body of your assignment).
- For calculations, you can type your calculations into your document (e.g., using the
Equations function in Microsoft Word) or insert an image of pen & paper calculations (please make sure that the calculations are clearly visible and legible).
- For information about School policies on assessment including penalties for late submission of work, see the current School of Psychology Student Guide (under the Course Orientation section).