| Experimental design | “A planned interference in the natural order of events.” |
| Subject(s) | Means the subject under study, which can be people, technology and natural phenomena. |
| Sample | Typically you can't study all the individuals of a particular subject pool, so in your experiment you use a sample and hope that the results generalize to the rest of the subjects. |
| Between subjects vs. within subjects design | Between subjects: Two separate groups of subject/phenomena measured. Within subjects: Same subjects/phenomena measured twice, on different occasions |
| Quasi-Experimental | When conditions do not permit an ideal design to be used and a controlled experiement is impossible, there are other techniques that can be used. These are called quasi-experimental designs. |
| Internal validity | How likely is it that the independent variables caused the dependent variables? |
| External validity | How likely is it that the results generalize to other instances of the phenomenon under study? |
| Correlation | Some factors/variables co-vary when changes in one variable are related with changes in the other, negative or positive. |
| Correlation: Powerful tool | Any variables in the world can be measured for correlation. Only two variables are needed (independent and dependent) for doing correlation studies. |
| Main operating principle behind correlation | There is no causation without correlation. |
| Correlation: Pitfall | Correlation does not imply causation between the variables measured! |
| Quasi-experimental designs | Purpose: Where true experimental design is not possible, approximate it. If direct control over dependent/independent variables is not possible. |
| How it works | 1. One-shot case study (no control group). 2. Single group pre- and post-test (minimal control) 3. ABAB: Single-group repeated measures (slightly less minimal control). |
| Limitations | Much greater uncertainty as to the internal and external validity of the quasi-experiments than true experimental designs. |
| What is it? | A more loose, pre-study using the intended experimental design to tune it. A pre-study intended to gauge the nature, scales or other factors of the variables to be measured, or the subject to be measured. |
| Why and when | Pilots are much more useful than you might think. Yes, it will increase the duration and effort of your experiment BUT: It can significantly improve the quality of the subsequent experiment in many cases. It will certainly clarify and sharpen the experimenter's understanding of one or more of: the experiment, experimental procedure, variables and subjects. |
| Bottom line | Do not try to “save time” by skipping a pilot if a pilot study seems to makes sense. |