Reliability and Validity

How would you define reliability?

• The consistency of data

• The accuracy of data

• The levels of data

• The validity of data

Organize these options into the right categories. You must get the correct definitions and what studies lack this type of validity, and what studies have high levels of this type of validity.

Groups:

(a) Internal Validity

,

(b) Temporal Validity

,

(c) Population Validity

,

(d) Ecological Validity

• Van IJzendoorn and Kroonenberg did a meta-analysis looking at 32 studies across 8 countries

• Moscovici used all female ppts- this is an example of beta bias and reduces __ validity

• Baddeley and Godden artificial task of reading word lists under the sea and on land

• Asch's results can be said to be due to America being very conformist in the 1950s

• When it measures what is intended. Know the IV is the cause.

• Observes natural infant-caregiver behaviours in a structured but realistic setting- Ainsworth

• Lab studies like Asch, should have a high level of this type of validity

• Studies should be valid despite progressions with time

• Do the results and conclusions apply to everyday life?

• Demand characteristics reduce this type of validity, like in Zimbardos study

• Asch and Zimbardo only used while, male ppts. This is androcentric and alpha bias,

• The extent to which the sample can be generalised to similar and wider populations

• Bowlbys, Milgrams and Ainsworth study have been replicated today and still apply

• Freud generally lacks this validity due to androcentric views

One way to assess reliability is by _. This involves re-administering the same test/questionnaire to the same people, under the same conditions, but on separate occasions.

Different researchers may interpret the same situation in a different way to another researcher, and this is called subjectivity bias. _____ must be established to ensure that more than one researcher interprets a situation in the same way. Behaviour categorises can be assignments, and a pilot study can take place to help test this.

• Inter-rater reliability

• Content Analysis

• Interviews

• +.80 correlation co-efficient

You can improve reliability of experiments by

• Standardising procedures

• Establishing cause and effect

• Changing the temperature to help suit the ppts

• Train the ppts to be more reliable

​ Observations can be improved by ​ (a)   behaviour categories. This means to make variable more​ (b)   . These categories should not overlap (e.g., hugging and cuddling).

• Operationalising
• measurable
• reliable 
• accurate 

Match the definitions for ways of assessing validity.

Groups:

(a) Face validity

,

(b) Concurrent Validity

,

(c) Predictive Validity

• E.g., Asking the same sample of employees to fill in both an existing survey, and your new one

• E.g., observing high IQ scores, score higher in exams

• Measure of whether it look subjectively promising that a tool measure what it's supposed too

• E.g., Using your GCSE grades to get into college

• It compares two assessments - one that has already been tested, and one that needs to be valid

• This refers to the ability of a test or other measurement to predict a future outcome

You can improve the validity of experiments by...

• Using a control group to see if IV does influence the DV

• Standardising methods and procedures

• Using Single-blind, or double blind studies.

• All options are correct.

Improving Validity of...

• Should be clear and operationalised

• Behaviour categories for observations

• Interpretations must match ppts reality

• Qualitative methods

• A lie scale and anonymity

• Qualitative Method sources

• Using Covert method

• Observations

• Using multiple sources - triangulation

• Questionnaires

What is one way to increase inter-rater reliability in observational studies?

• Providing clear and detailed coding schemes

• Changing the research question

• Reducing the number of observers

• Allowing observers to use their own judgment