Evaluative Summary of article on
Randomized Block Design
Submitted by Tsai-Feng Wang

1. Background Information
Author: Gupta, S.
Title: Impulsivity/sociability and reinforcement in verbal operant conditioning: a replication.
Source: Personality and Individual Differences.
Year: 1990, 11(6), 585-589.

2. Abstract
The purpose of this study was to examine the verbal operant conditioning (dependent variable) in relation to impulsivity/sociability (three levels: high, moderate and low) and reinforcement conditions (two levels: good (rewarding reinforcer) and poor (punishing reinforcer)). The author stated that this is a 3 x 2 randomized block design (however, I think this a 3 x 2 fixed-effect generalized randomized blocking design because the author tested interaction effects in the ANOVA tables). Previous similar study (Gupta and Nagpal, 1978) was conducted by using different reinforcers (one verbal and one nonverbal reinforcers for rewarding condition, and one nonverbal reinforcer for punishing condition). In this study, the author used only verbal reinforcers for both rewarding and punishing conditions). Participants were 240 female students from undergraduate and postgraduate schools. 120 subjects joined in the one experiment with impulsivity variable, and 120 subjects joined another one with sociability variable. Dependent variable (verbal operant conditioning) was measured by one instrument: stimulus material using Taffel's sentence construction technique. Two separate ANOVA tables were completed on each of the two research questions (one is related to the impulsivity, and another is sociability). The results of ANOVA with impulsivity variable indicated that the effect of impulsivity (p = .02), reinforcement (p = .05), and interaction between impulsivity and reinforcement (p < .01) were found significant. The results of ANOVA with sociability variable indicated that the effect of sociability (p < .01) and interaction between sociability and reinforcement (p < .01) were found significant. However, the effect of reinforcement (p = ?) was not found to be significant.

3. Null hypothesis, (or p) - level and sample size per group.
In this study, two null hypotheses for this 3 x 2 randomized block design (RB-2) in two separate experiments are as follows:
(A). For the study with the impulsivity variable:
(1) Test for the effect of reinforcement treatment:
Ho1: j = 0 for all j. There is no effect caused by the two types of reinforcement, which means that there is no verbal operant conditioning scores difference for subjects either with rewarding condition or punishing one.
H11: j 0 for some j. There is verbal operant conditioning scores difference caused by the different types of reinforcement.
(2) Test for the blocking variable (impulsivity):
Ho1 : z = 0 for all z. There is no effect caused by the three levels of impulsivity, which means that there is no verbal operant conditioning scores difference for subjects with high, moderate, or low level of impulsivity.
H11 : z 0 for some z. There is verbal operant conditioning scores difference caused by the different levels of impulsivity.
(B). For the study with the sociability variable:
(1). Test for the effect of reinforcement treatment:
Ho1: j = 0 for all j. There is no effect caused by the two types of reinforcement, which means that there is no verbal operant conditioning scores difference for subjects either with rewarding condition or punishing one.
H11: j 0 for some j. There is verbal operant conditioning scores difference caused by the different types of reinforcement.
(2) Test for the blocking variable (sociability):
Ho1 : z = 0 for all z. There is no effect caused by the three levels of sociability, which means that there is no verbal operant conditioning scores difference for subjects with high, moderate, or low level of sociability.
H11 : z 0 for some z. There is verbal operant conditioning scores difference caused by the different levels of sociability.
The level was not stated throughout the article. The p level was reported in the ANOVA table. In the experiment with impulsivity variable, a significant F-ratio was report in the reinforcement effect at p = .05 level, in the impulsivity effect at p = .02 levels, and in the interaction effect between reinforcement and impulsivity at p < .01. In the experiment with sociability variable, a significant F-ratio was report in the reinforcement effect at p < .01 levels, in the sociability effect at p < .01 levels.
The total number of participants was 240, which was divided into two parts of experiments with 120 subjects in each one. Six groups were formed in this 3 x 2 randomized block design (RB-2). Therefore, there were 20 subjects in each group. The author used equal sample size (n=20 subjects) in each group and in each experiment.

4. Independent and dependent variables.
There were two fixed independent variables, the blocking and treatment variables, and one dependent variable in the 3 x 2 randomized block design (RB-2).
The blocking variable:
In experiment 1: impulsivity, with three levels of scores: high, moderate, and low. In experiment 2: sociability, with three levels of scores: high, moderate, and low.
The treatment variable: Reinforcement, with two levels: good (rewarding condition) and poor (punishing condition).
The dependent variable: Verbal operant conditioning scores measured by the stimulus material using Taffel's sentence construction technique.

5. Instrument, briefly comment on its reliability and validity.
Three instruments were used in this study. For the purpose of sample selection, Hindi version of the Eysenck Personality Inventory (EPI) was used. In order to measure the scores of the dependent variable (verbal operant conditioning), the author used the stimulus material using Taffel's sentence construction technique. Also, there is a questionnaire used for testing the awareness of the connection between subjects' behavior and the reinforcement shortly after the treatment experiment. None of these three instruments were given the evidence of validity and reliability. We don't know what these instruments are, and how readers can be convinced about their consistency and accuracy related with the author's intention on different purposes in this study.

6. Experimental Procedure
240 subjects were selected from the preliminary testing (Hindi version of the Eysenck Personality Inventory (EPI) in order to block subjects according to three levels of impulsivity and sociability) among 2000 female volunteers. Two experiments were conducted with 120 subjects in each experiment. Every subject was tested individually by the stimulus material using Taffel's sentence construction technique, which presenting a card at a time and the total card number is 100. In the stage of the fist 20 cards, the experimenter simply wrote down what the subject's choice in pronounce in the nature condition. In the stage of the next 60 cards, it is a treatment condition that the experimenter spoke "good" or "poor" where "I' or "WE" was reinforced. In the stage of last 20 cards, the reinforcement was removed. As a result, frequency in using different pronouns from each card in these three stages will be recorded and calculated toward the final conditioning scores. Shortly after the experiment, a questionnaire used for testing the awareness of the connection between subjects' behavior and the reinforcement was conducted for each subject. If any subject were aware of the connection, another new subject would replace her. There were 15 replacements happened in this study.

7. Statistical analysis and conclusion.
Two ANOVA analyses were conducted separately on two experimental interests, one is related with impulsivity variable and another one is related with sociability variable. In the experiment with impulsivity variable, the effect of impulsivity and reinforcement were revealed to have significant effects, where F(2, 114) = 4.58, p = .02 and F(1, 114) = 4.38, p = .05, respectively. Also, the interaction effect between impulsivity and reinforcement was found to be significant (F(2, 114) = 56.39, p < .01). It was found that subjects in the high level of impulsivity had larger conditioning scores with "good"(rewarding reinforcer) than with "poor"(punishing reinforcer) whereas subjects in the low level of impulsivity had larger conditioning scores with "poor" than with "good". In the experiment with sociability, the effect of sociability was revealed to have a significant effect (F(2,114) = 5.57, p < .01). Also, the interaction effect between sociability and reinforcement was found to be significant (F(2, 114) = 19.69, p <.01). However, the effect of reinforcement was not found to be significant. It was found that subjects in the high level of sociability had larger conditioning scores with "good"(rewarding reinforcer) than with "poor"(punishing reinforcer) whereas subjects in the low level of sociability had larger conditioning scores with "poor" than with "good".

8. If you were the researcher, how would you improve the study?
1. Although the author stated that this is a 3 x 2 randomized block design; however, I think this is a 3 x 2 fixed-effect generalized randomized blocking design because the author tested interaction effects (which were also the author's intention) in the ANOVA tables. Therefore, this is a GRB-2 rather RB-2 design. The null hypothesis about interaction effect in the GRB-2 design should be added after previous GB-2 design in Part 3 as follows:
(A). For the study with the impulsivity variable:
(3) Test for the interaction effect between impulsivity and reinforcement:
Ho1: ( ) jz = 0 for all j and z. There is no interaction effect between the two types of reinforcement and three levels of impulsivity. That is, a special combination of the treatment and blocking factors, e.g. rewarding condition and low level of impulsivity, does not affect the consistency of verbal operant conditioning scores when compared to other combinations of the treatment and blocking factors.
H11: ( ) jz 0 for some j and z. Interaction effect exists between the two types of reinforcement and three levels of impulsivity in the verbal operant conditioning scores difference.
(B). For the study with the sociability variable:
(3) Test for the interaction effect between sociability and reinforcement:
Ho1: ( ) jz = 0 for all j and z. There is no interaction effect between the two types of reinforcement and three levels of sociability. That is, a special combination of the treatment and blocking factors, e.g. rewarding condition and high level of sociability, does not affect the consistency of verbal operant conditioning scores when compared to other combinations of the treatment and blocking factors.
H11: ( ) jz 0 for some j and z. Interaction effect exists between the two types of reinforcement and three levels of sociability in the verbal operant conditioning scores difference.
2. The way the author conducted the interaction comparison is quiet doubtful. I don't really understand how he found his conclusions about the interaction effects between reinforcement and impulsivity/sociability. He just stated that he used Newman- Keuls procedure to test the interaction variance. How come? This method is supposed to test the differences among the levels of main effects. However, there were just two levels of treatment (reinforcement), and blocking variable (impulsivity/sociability) is not our interests (although it has three levels), we don't need to conduct a post-hoc procedure in this study. When interaction effect is significant, and we are interested in the interaction effect, contrast-contrast interaction (in this case, three interaction contrasts in each experiment) comparison procedure should be used and presented. It is even clearer for readers if the author could show interaction graphs in the study.
3. Usually when we use RB or GRB design, we treat the blocking variables are nuisance variables. Blocking is used to remove the effects of nuisance variables. Also, in GRB design, we are not really interested in the interaction effect between blocking variables and treatment variables. In this study, the author did care about the significant results of the blocking variable and the interaction effect of the blocking variable and treatment variable and treated these results as important findings. From the viewpoint of GRB design issue, we should ignore the significant results of the blocking variable and the interaction effect of the blocking variable and treatment variable, because they are not our interests. However, if we are interested in the interaction effect, we should use another factorial design for your research intention and purpose.
According to author's purpose (author is both interested in impulsivity/sociability and reinforcement) for this study, and the author also treated impulsivity and sociability as main effects (see Results part in the article, p587), it is really make me feel confused why he described that he used RB design in his study. The variables in this study were not defined well. If I were the researcher, I would clarify the independent variables as a treatment variable or blocking one, and how these variables are related to the dependent variable in order to make sure that I use right experimental design and statistical analysis.
4. We can find that all sample are from female. The author didn't explain why he only chose female as samples (because of convenient sample or his intention?). It is problematic issue, when we generalize these results to the public without mentioning they are suitable for female population. Therefore, I think the title of this study should be either changed as Impulsivity/sociability and reinforcement in verbal operant conditioning for female, or discuss and explain it in the conclusion part of the paper when talked about the generalization of the research results.
5. Statistical assumption should be stated in the paper in order to make sure that statistical results are unbiased. For example, for the ANOVA method in this study, there are three important assumptions: (1). Normality- the samples for each of the groups are assumed to come from normal populations. (2). Homogeneity of Variance- this is the assumption that the populations from which the data came all have the same variance, regardless of whether or not their means are equal. (3). Independence- this is the assumption that the errors associated with each observation are independent. Because this study is a fixed model (model 1), we don't need to use MSinteraction as the error term (homogeneity of population within-cell variation is required when we use MSwithin as a error term). Therefore, the sphericity assumption is not required in this study (see Kirk in p.306).
6. ANOVA table (in Table 4) didn't provide what is the p-value in the effect of reinforcement. We cannot understand how author judged that it was non-significant in the reinforcement variable. Was the p-value too high or just slightly past .05? I would present the p-value no matter what the value is in order to inform readers to judge with the real raw data.
7. The evidence of instrument validity and reliability should be provided. The author used three different instruments for different purposes. They were Hindi version of the Eysenck Personality Inventory (EPI), the stimulus material using Taffel's sentence construction technique and the questionnaire used for testing the awareness of the connection between subjects' behavior and the reinforcement. The construct validity of evidence of EPI should be provided in order to make sure EPI was exactly used to measure the construct of impulsivity/sociability. Also, the test-retest reliability coefficient of EPI should be proved its consistency. I will also provide the evidence of the predictive-criterion validity for the stimulus material and the questionnaire in order to prove the accuracy of these two instruments for the purposes of this study. The evidence of the test-retest and split-half reliability should be stated for the stimulus material and questionnaire, respectively, in order to inform readers their consistency when using these two instruments.
8. Although the ANOVA table was given, and the effects of impulsivity, reinforcement, and interaction were tested to be significant, the information is not sufficient to the readers. In order to understand the differences were large enough to be useful in the real world, effect size (and 2) should provide to give the evidence practical significance. After I did the calculations (an example from the experiment with impulsivity variable):
2 =(2-1)(4.38-1) / (2-1)(4.38-1)+ (20*2*30) = .027 (large association)
2 = (3-1)(4.58-1) / (3-1)(4.58-1) + (20*2*30) = .056 (small association)
( )2 = (2-1)(3-1)(56.39-1) / (2-1)(3-1)(56.39-1) + (20*2*30) = .480 (large association)
f = squr(.027/1-.027) = .167 (small effect size)
f = squr( .056-/1-.056) = .251 (small effect size)
f ( ) = aqur(.480/1-.480) = 1.332 (large effect size)
I found that only interaction effect had a large effect size in relation to the verbal operant conditioning scores. Although effects of reinforcement and impulsivity reached significance, its small effect size can't be counted on any usefulness in relation to the verbal operant conditioning scores.
9. I found that the conclusion part in this study was quite weak. No limitation, recommendation or further explanation described in this study. Although the author stated the different finding comparing the previous study, he didn't make any suggestion or application for this new finding. What is the possible explanation for this different result? The caffeine example in the discussion part was not adequate and totally was not related with the purpose of the study. If I were the researcher, I would like to provide a clear discussion in the final section about why the interaction between impulsivity and reinforcement would potentially affect one's verbal operant conditioning scores? How can this finding be used in a practical way? Also, I would present what the limitations in this study were and give some suggestions for further studies.