The extent to which a study's results (regardless of whether the study is descriptive or experimental) can be generalized/applied to other people or settings reflects its external validity. Typically, group research employing randomization will initially possess higher external validity than will studies (e.g., case studies and single-subject experimental research) that do not use random selection/assignment. Campbell and Stanley (cited in Isaac & Michael, 1971) have identified 4 factors that adversely affect a study's external validity.
An interaction between how the subjects were selected and the treatment (e.g., the independent variable) can occur. If subjects are not randomly selected from a population, then their particular demographic/organismic characteristics may bias their performance and the study's results may not be applicable to the population or to another group that more accurately represents the characteristics of the population.
Pretesting subjects in a study may cause them to react more/less strongly to the treatment than they would have had they not experienced the pretest. In such situations the researcher(s) cannot conclude that members of the population who were not pretested would perform in a similar manner to the participants in the study. Restated, to generalize the results of the study the researcher would have to specify that a particular type of pretesting also be done because the pretesting could be serving as an extra, unintentional independent variable.
The performance of subjects in some studies is more a product or reaction to the experimental setting (e.g., the situation where the study is conducted) than it is to the independent variable. For example, subjects who know they are participants in a study, or who are aware of being observed, etc., may react differently to the treatment than a subject who experienced the treatment but was not aware of being observed, etc.
Studies that use multiple treatments/interventions may have limited generalizability because the early treatments may have a cumulative effect on the subjects' performance. If a group experienced treatment X1, and the first treatment was followed by a second (X2), their measured performance after X2 will be affected by both treatments not just X2's because the effects of X1 are not erasable.
