Example: Inter-rater reliabilityIn an observational study where a team of researchers collect data on classroom behaviour, interrater reliability is important: all the researchers should agree on how to categorise or rate different types of behaviour. You use it when data is collected by researchers assigning ratings, scores or categories to one or more variables. Inter-rater reliability (also called inter-observer reliability) measures the degree of agreement between different people observing or assessing the same thing. Remember that changes can be expected to occur in the participants over time, and take these into account.

When planning your methods of data collection, try to minimise the influence of external factors, and make sure all samples are tested under the same conditions.When designing tests or questionnaires, try to formulate questions, statements, and tasks in a way that won’t be influenced by the mood or concentration of participants.You administer the test two months apart to the same group of people, but the results are significantly different, so the test-retest reliability of the IQ questionnaire is low. Example: Test-retest reliabilityYou devise a questionnaire to measure the IQ of a group of participants (a property that is unlikely to change significantly over time). Then you calculate the correlation between the two sets of results. To measure test-retest reliability, you conduct the same test on the same group of people at two different points in time. The smaller the difference between the two sets of results, the higher the test-retest reliability. Test-retest reliability can be used to assess how well a method resists these factors over time. Many factors can influence your results at different points in time: for example, respondents might experience different moods, or external conditions might affect their ability to respond accurately. Example: Test-retest reliabilityA test of colour blindness for trainee pilot applicants should have high test-retest reliability, because colour blindness is a trait that does not change over time. You use it when you are measuring something that you expect to stay constant in your sample. Test-retest reliability measures the consistency of results when you repeat the same test on the same sample at a different point in time. Which type of reliability applies to my research?.Probability vs non-probability sampling.

Test retest reliability psychology