The 5 Stages of Test Development

1. Test Conceptualization
2. Test Construction
3. Test Tryout
4. Item Analysis
4. Test Revision

Test Conceptualization

• The objective of the test (what will it measure?)
• The need for the test (what advantages will it have?)
• The test population (who will take it?)
• The content (what will the test cover?)
• The style of administration (how will it be given)
• The item format (will it be forced choice? Multiple choice?)
• The inclusion of alternate forms use (is it necessary for this test?)
• The training requirements (what professionals are allowed to give the test?)

Test Construction - Scaling

• Scaling or the process of assigning numbers in measurement
• Thurstone was the first to push the issue, stating that scales should be developed based on the performance of multiple test takers...Allows you to say someone has more or less of the characteristic you are measuring
• Summative scale - when the item scores are totaled (e.g., a questionnaire composed of Likert Scales)
• Comparative scaling - a subject must select one of two responses, called paired comparisons
• Categorical scaling - placing items into different categories which are then scored differently.
• Guttman scales - items range sequentially from a weak to strong expression of an attitude or trait

Test Construction - Item Writing

• Dichotomous format - offers two possible responses to each item (True-False is most common)
• Polytomous format - More than one choice for each item, credit given for only one. The others items are called "distractors." (e.g., such as multiple choice)

Test Tryout

• Pilot the test with a small number of subjects to determine the correct population, and number/quality of items

Item Analysis
Two Types:

• Item difficulty (or easiness) - The proportion of test takers getting any specific item correct
• Item discrimination - Measures how individual items correlate with the entire test for a group of test takers

Test Revision

• Involves cross validation or giving the test to another group of test-takers to see if results are similar to the original group (remember that the validity coefficient of the test will be smaller due to shrinkage)

Where to Find Tests in Print

• Tests in Print - gives title, for whom the test is designed, copywriter, author, publisher, whether or not a manual exists
• Mental Measurements Yearbook (Buros)- Includes reviews of each test, and comments about reliability and validity
• Tests: A Comprehensive Reference for Assessment in Psychology, Education, and Business - Descriptions of tests broken down into these subheadings