| Next revision | Previous revision |
| rem4:experimental_designs_ii [2008/10/13 11:47] – created thorisson | rem4:experimental_designs_ii [2024/04/29 13:33] (current) – external edit 127.0.0.1 |
|---|
| |
| ===Index=== | ===Overview=== |
| |
| | True Experimental Designs: Procedure | | | True Experimental Designs: Procedure | |
| | Levels | Relating to an independent variable: The number of levels of an independent variable is equal to the number of variations of that variable used in an experiment. | | | Levels | Relating to an independent variable: The number of levels of an independent variable is equal to the number of variations of that variable used in an experiment. | |
| | Dependent variables | Values are measured during and/or after the experiment. | | | Dependent variables | Values are measured during and/or after the experiment. | |
| | Sample: subject selection from a "population" A representative subset, drawn from a population, of the phenomenon we are studying. | Examples: \\ | | Sample: subject selection from a "population" A representative subset, drawn from a population, of the phenomenon we are studying. | Examples: \\ a. Siggi, Maggi and Biggi representing human males. \\ b. 10 lakes representing all freshwater on the Earth's surface. \\ c. rust on bottom of doors representing the overall state of an automobile. \\ A sample should be randomly chosen to (1) minimize spurious correlations and thus (2) maximize the generalizability of the results of measuring only a small subset of the phenomenon. | |
| - Siggi, Maggi and Biggi representing human males. \\ | |
| - 10 lakes representing all freshwater on the Earth's surface. \\ | |
| - rust on bottom of doors representing the overall state of an automobile. \\ | |
| A sample should be randomly chosen to (a) minimize spurious correlations and thus (b) maximize the generalizability of the results of measuring only a small subset of the phenomenon. | | |
| | Spurious correlation | "false" correlation - correlation that implies a connection between things measured, where there is no causal relationship between them, in and of themselves. | | | Spurious correlation | "false" correlation - correlation that implies a connection between things measured, where there is no causal relationship between them, in and of themselves. | |
| | Between-subjects design | If our control group in an experiment contains different instances than the experimental group. | | | Between-subjects design | If our control group in an experiment contains different instances than the experimental group. | |
| === Some Statistical Methods for Experimental Designs: What to Use When === | === Some Statistical Methods for Experimental Designs: What to Use When === |
| |
| | Selecting between hypotheses | Statistical tests help you figure out whether the difference (in means) observed in a dependent variable (as measured between two samples) is large enough to indicate a **non-coincidence**. To make this judgement, the "natural" variation in each group is used as a "baseline". \\ Significance level is a measure that tells you how non-coincidental you want your measure to be, to be considered as "significant". p<0.05 and p<0.01 are most common (less than 5%, 1% probability of the result being random). | | | Selecting between hypotheses | Statistical tests help you figure out whether the difference (in means) observed in a dependent variable (as measured between two samples) is large enough to indicate a **non-coincidence**. \\ To make this judgement, the "natural" variation in each group is used as a "baseline". \\ Significance level is a measure that tells you how non-coincidental you want your measure to be, to be considered as "significant". p<0.05 and p<0.01 are most common (less than 5%, 1% probability of the result being random). | |
| | **What you study** | **What you use** | | | **What you study** | **What you use** | |
| | Two factors varying along a continuum | Correlation/regression measures | | | Two factors varying along a continuum | Correlation/regression measures | |
| === Using Models to Validate and Measure - a.k.a. Simulation === | === Using Models to Validate and Measure - a.k.a. Simulation === |
| |
| | What simulation is | A simplified model of subject under study. \\ Sometimes only the environment is simulated. | | | What simulation is | A simplified model of subject under study - that is, a simplification not of the key causal factors in the phenomenon, which must remain in our model for it to be useful, but rather a reduction (sometimes a radical one) of the "extra stuff that really doesn't matter". | |
| | What it does | Simplifies! Makes it easier to \\ (A) set up testing conditions, \\ (B) control independent variables, \\ (C) make changes to the independent variables,(D) measure the results. | | | What it does | Simplifies! Makes it easier to \\ (A) set up testing conditions, \\ (B) control independent variables, \\ (C) make changes to the independent variables,(D) measure the results. | |
| | When to use | When the complexity of that which is to be modeled/understood becomes so great that mathematical models are intractable and hypothesis falsification would take decades, centuries or millennia. | | | When to use | When the complexity of that which is to be modeled/understood becomes so great that mathematical models are intractable and hypothesis falsification would take decades, centuries or millennia, or is simply out of the question (as in e.g. astrophysics). | |
| | Kinds of simulation methodologies | Continuous time and state: E.g. differential equations. \\ Discrete time/state: E.g. automata. \\ | | | Kinds of simulation methodologies | Continuous time and state: E.g. differential equations. \\ Discrete time/state: E.g. automata. \\ | |
| | Relation between scientific theories and simulations | To build a simulation we need a theory that tells us how things relate to each other. | | | Relation between scientific theories and simulations | To build a simulation we need a theory that tells us how things relate to each other. | |
| | Procedure | Pick methodology. \\ Decide which kinds of questions to answer. \\ Model major states/transitions or input/output/functional properties of system. \\ Run simulations with variations in independent variables. \\ Note outcome. \\ Fix model. \\ Repeat. | | | Procedure | Pick methodology. \\ Decide which kinds of questions to answer. \\ Model major states/transitions or input/output/functional properties of system. \\ Run simulations with variations in independent variables. \\ Note outcome. \\ Fix model. \\ Repeat. | |
| |
| | \\ |
| | \\ |
| | \\ |
| | \\ |
| |
| === Levels of System Knowledge in Simulation === | === Levels of System Knowledge in Simulation === |
| | 1 Generative | Means to generate data in the system | | | 1 Generative | Means to generate data in the system | |
| | 1 Structure | Components (at lower levels) coupled together to form a generative system | | | 1 Structure | Components (at lower levels) coupled together to form a generative system | |
| | <sub>Source: G. Klir 1985</sub> | | | <sub>Source: G. Klir 1985</sub> | | |
| |
| | \\ |
| | \\ |
| | \\ |
| | \\ |
| |
| === Using Models to Validate and Measure: The Model Human Processor === | === Using Models to Validate and Measure: The Model Human Processor === |
