Stats-Threats to validity

In an experiment, we identify cause-effect relationships:

~we control the cause and observe the effect.

~changes in the DV caused by the manipulation of the IV.

Internal Validity

=>The extent to which the results obtained are a function of the variables that were systematically manipulated. Are changes in the IV responsible for the observed variation in the DV? Might the variation in the DV be attributable to other causes (confounds)?

=>Why is it important?? High internal validity=strong evidence of causality

=>To maximise internal validity:

• must be able to rule out the possibility of other factors producing the change(confounds)
• must control everything and eliminate possible extraneous influences
• easient in highly controlled, laboratory settings

=>Threats to internal validity that compromise our confidence in saying that a relationship exists between the IV and DV:

• History effects-events occurring during the experiment that are not part of the treatment; can be solve by holding experiences constant except for IV/randomlyy assign conditions to time
• Maturation effects-bilogical or psychological processes (e.g. aging, fatique, hunger...) within partcipants that may change due to the passing of time
• Mortality-differential loss of individuals from treatment and/or control groups due to nonrandom reasons (those who drop out of a study could be qualitatively different from those who remain)
• Instrument decay-equipement becomes iaccurate with age/experimenters become more skilled or bored; can be solve by randomisation condition to time, check reliability of instrument and staff.
• Participant selection-different types of participants placed at different levels of the IV; use random assignment or matching method
• Statistical regression to the mean-going back to mean after extreme behaviour
• Participants communicate-diffucion of treatment effects (control group learns about the manipulation), compensatory rivalry (participants in different conditions start competing), compensatory equalisatoin (experimenters know the condition of participants are in and provide enhance services that go beyond the routine), resentful demoralisation (control group leans that they are in the control and not try as hard)

External Validity

=>Does the IV represent the concept we intend? A measure is externally calid if it truly measures the hypothetical construct intended. An experiment is externally valid if it is similarto phenomenon in the real world.

=>Having high external validity often means having a lack of contrl of confounds

=>Population validity: the extent to which the results can be generalised from the experiemntal sample to a defined population

=>Ecological validity: the extent to which the results can be generalised from the set of experimental conditions in the experiment to other conditions.

=>Threts to External Validity compromises our confidence in stating whether the study's result are generalisable:

• reactive effects of testing: when a pre-test increase/decrease the repondents' sensitivity to the treatment (especially in seld-eport measures of attitude and interest)
• reactive effects of experimental setting: when the conditions of the study are such that the results are not likely to be replicated in the non-experimental situation
• selection-treatment interaction: the possibility that some characteristic of the participants selected for the study enteracts with some aspect of the treatment (prior experiences, learning, personality factors...)
• multiple-treatment interference: participants receive more than one treatment, the effects of previous treatment may influence subsequent ones (sequence effects/carry-over effects)

=>Improving External Validity:

• Replication-an additional scientific study conducted in exactly the same manner as the original research project. When we replicate an experimental finding, we are able to place more confidence in that result.
• Replication with extension-seeks to replicate a previous finding but does so in a different setting /with different participants/under different conditions

Statistical Validity

=>Making type 1 error: rejecting the null hypothesis when the null hypothesis is true (false positive)

• possible causes: fishing

=>Making type 2 erroe: failing to reject the null hypothesis when the null hypothesis is false (false negative)

• possible causes: power, reliability of measures/ =treatments, random irrelevance, random heterogeneity of respondents...