|
TSQS supports two programs that may be used for test
(item) analysis:
These programs are designed to be used with the GPSCOR scanner
program and the General Purpose Answer sheets that can be
processed by GPSCOR. In addition, the Itemanal program
can be used with data files produced by the MRSCOR scanner
program by treating each response to each item as a True/False
item. Processing parameters (such as the number of items
in the test and the number of choices per item) are automatically
picked up from information supplied on the Request for Service
sheet. Appropriate output will be returned along with your
processed answer sheets if you indicate that you want one
of these programs run by indicating your choice under the
Data Processing Required: option on the Request for
Service sheet. A popular practice in a number of departments
is to create multiple forms of a test for purposes of administration
in large (crowded) classes. This is usually done by reordering
the items in the test and, sometimes, by reordering the
choices within each item. In order to analyse all forms
of a test as a single set, mapping
instructions must be provided so that the items
can be aligned to correspond from one form to another before
proceeding with the item analysis. In order to analyse multiple
forms of a test using Itemanal, you must sort the answer
sheets into separate jobs according to test form and have
them scored separately, i.e., the combining of the forms
is done after the answer sheets have been scored. Item analysis
programs are also available for personal use on your PC
or Mac. Information about such packages can be obtained
from vendors such as
If you desire assistance in using a data file produced
by us with software on your own computer, please contact
Dan Precht.
Itemanal:
- Of the packages available at TSQS, Itemanal
produces the most comprehensive test and item analysis.
It begins by displaying the key that was used to score
the test. A value of * is used to indicate that no correct
answer was supplied for an item on the key that was
used.
KEY
* D A D D B B B A C
A A D A B D B D B B
A C B A B D A D D B
C D B B D D B A C B
The following pages display the FREQuency of occurrence
of each SCORE, the Z-score that would be associated with
that raw SCORE if the distribution was normally distributed,
the PERCENTILE associated with the SCORE (which is the
CUMulative PERCENTAGE of scores occurring below the current
score plus half of the current scores) and the CUMulative
PERCENTAGE of examinees obtaining scores at or below the
current value. Finally, the cutting points are reported
which will be used to divide the class into high, middle
and low groups. By default, the program attempts to place
27% of the class into each of the high and low groups
with the remainder in the middle. The success in identifying
27% will depend on the size of the class and the presence
of tied scores at the cutting points.
*** FREQUENCY DISTRIBUTION AND HISTOGRAM OF SUM OF CORRECT RESPONSES ***
SCORE FREQ Z-NORML %-ILE CUM-%
2 1 -2.673 .38 .75
3 0 -2.432 .75 .75
4 0 -2.432 .75 .75
5 0 -2.432 .75 .75
6 1 -2.281 1.13 1.50
7 2 -2.004 2.26 3.01
8 2 -1.780 3.76 4.51
9 1 -1.656 4.89 5.26
10 4 -1.494 6.77 8.27
11 5 -1.273 10.15 12.03
12 8 -1.035 15.04 18.05
13 6 -.831 20.30 22.56
14 10 -.633 26.32 30.08
15 10 -.417 33.83 37.59
16 6 -.257 39.85 42.11
17 7 -.132 44.74 47.37
18 11 .038 51.50 55.64
19 7 .208 58.27 60.90
20 1 .286 61.28 61.65
21 4 .336 63.16 64.66
22 4 .417 66.17 67.67
23 6 .522 69.92 72.18
24 8 .680 75.19 78.20
25 4 .831 79.70 81.20
26 6 .972 83.46 85.71
27 3 1.119 86.84 87.97
28 3 1.232 89.10 90.23
29 4 1.387 91.73 93.23
30 1 1.523 93.61 93.98
31 1 1.586 94.36 94.74
32 5 1.828 96.62 98.50
33 0 2.170 98.50 98.50
34 1 2.281 98.87 99.25
35 0 2.432 99.25 99.25
36 0 2.432 99.25 99.25
37 0 2.432 99.25 99.25
38 1 2.673 99.62 100.00
The cutting points used to divide the class into HIGH, MIDDLE and LOW groups
for the item statistics presented below are: 14.5 and 23.5
HISTOGRAM 1
FREQUENCY 1 0 1 4 5 13 16 16 18 8 8 14 10 6 5 6 1 0 1
-------------------------------------------------------------------------------------------------------
18 *
17 *
16 * * *
15 * * *
14 * * * *
13 * * * * *
12 * * * * *
11 * * * * *
10 * * * * * *
9 * * * * * *
8 * * * * * * * *
7 * * * * * * * *
6 * * * * * * * * * *
5 * * * * * * * * * * * *
4 * * * * * * * * * * * * *
3 * * * * * * * * * * * * *
2 * * * * * * * * * * * * *
1 * * * * * * * * * * * * * * * * *
--------------------------------------------------------------------------------------------------------
CLASS
INTERVAL 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38
After reviewing the distribution of scores in the frequency table and histogram, above, and the overall test statistics that appear in the sample output, you may find it useful to start at the back of the report where a scattergram is displayed which summarizes the biserial correlations and difficulty levels of all the items. Items having low values of either (say, below .3) and, particularly, items having negative correlations should then be given special attention in examining the item-wise reports. If the class-size is large enough, plots are also provided that display the success rates (difficulty levels) of quintile groups for each item. These, too, can alert you to items that are behaving strangely. In the examples below, the plot for Item 34 illustrates a nicely behaving item while those for Items 13 and 37 indicate a need for closer examination. In the item-wise reports, items for which the HIGH group chooses a particular incorrect response more often than they choose the correct response should, especially, be examined more closely. Also, items for which the LOW group apparently gives the correct response more frequently than the HIGH group or for which the HIGH group chooses a particular incorrect response more often than does the LOW group should also be closely examined.
Examtab:
Examtab produces significantly less information
than Itemanal. It has
none of the graphical output, the frequency distribution
does not include the percentile column, and the item-wise
report consists of only one line per item. As will be
discovered by comparing the output from the two programs,
the Point Biserial reported by Examtab is the correlation
between the score on the item and the score on the remainder
of the test (rather than the total test score). In addition
to its brevity, a significant feature of the Examtab
program is that it provides a facility to process multiple
forms of a test, i.e., different versions obtained by
reordering the items in the test or by reordering choices
within each item.
The answer sheets from the different forms of the test
do not need to be sorted into separate groups. During
scanning, the answer sheets are scored against the keys
for all forms of the test. If Examtab is run without
providing information about the mapping of items from
one form to another, a separate item analysis is produced
for each version of the test. If the mapping
information is provided, a single, combined
item analysis is produced.
Main Parameters 'E' and 'F' on the Request
for Service sheet should both be zero if you
intend to use this program. If more than one form of
the test is to be processed, the Special Code field
on the answer sheets must be used to indicate the form
number. This means that Main Parameter 'C' on the Request
for Service sheet must be 1 or 2. This last requirement
also means that the letter-sized answer sheet (Form
#16412) cannot be used with the Examtab program.
SCORE FREQ. PERCENT CUM-%
2 1 .75 .75
3 0 .00 .75
4 0 .00 .75
5 0 .00 .75
6 1 .75 1.50
7 2 1.50 3.01
8 2 1.50 4.51
9 1 .75 5.26
10 4 3.01 8.27
11 5 3.76 12.03
12 8 6.02 18.05
13 6 4.51 22.56
14 10 7.52 30.08
15 10 7.52 37.59
16 6 4.51 42.11
17 7 5.26 47.37
18 11 8.27 55.64
19 7 5.26 60.90
20 1 .75 61.65
21 4 3.01 64.66
22 4 3.01 67.67
23 6 4.51 72.18
24 8 6.02 78.20
25 4 3.01 81.20
26 6 4.51 85.71
27 3 2.26 87.97
28 3 2.26 90.23
29 4 3.01 93.23
30 1 .75 93.98
31 1 .75 94.74
32 5 3.76 98.50
33 0 .00 98.50
34 1 .75 99.25
35 0 .00 99.25
36 0 .00 99.25
37 0 .00 99.25
38 1 .75 100.00
TEST MEAN= 18.83
TEST VARIANCE= 46.29
TEST STANDARD DEVIATION= 6.804
ITEM POINT ALTERNATIVES ANSWERED
BISERIAL A B C D CORRECT
8 .302 22.6 26.3* 13.5 36.8 35
9 .201 5.3* 19.5 17.3 45.9 7
12 .309 51.1* 13.5 18.0 16.5 68
13 .034 13.5 17.3 29.3 27.8* 7
34 .408 12.0 45.9* 24.1 17.3 61
35 .367 26.3 9.8 24.8 37.6* 50
36 .415 14.3 10.5 11.3 63.9* 85
37 -.031 21.1 39.1* 21.8 16.5 52
(PERCENT) (FREQUENCY)
This page was last updated
at
|
