29

Chapter 3. The screening process

3.1 Methods and design of the screening process

3.1.1 Description of the study population sample

Four schools were randomly chosen in both of the two Berlin districts Mitte and Tiergarten from twelve basic-level schools. The two districts constitute the city center of Berlin. While Mitte is situated in the Eastern, Tiergarten is in the Western part of the inner city. With the choice of an equal number of schools in the East and the West we intended to control for any effect of different teaching methods or socialization in the formerly politically separate parts. Berlin, the German capital, is a major city with a population of 3.4 million people. 433,000 of its inhabitants are of foreign descent. The district Mitte has a population of 75,000 people, 9,500 of which are were not born in Germany. In the district Tiergarten 89,000 people are registered, with 24,000 inhabitants of foreign descent (Statistisches Landesamt, 1998).

In February of 1996, eight regular third-grade classes were included in the study. The sample included 182 third-graders, 100 male and 82 female. To minimize the effect of particular teachers on academic performance, all classes were from different schools. Special education schools were excluded. We obtained written permission for the study from the Office for School Affairs of the Berlin Senate, the school‘s principal and the teacher‘s and parent‘s of the included classes. We were not denied permissions for the study in any one student of the erstwhile chosen classes. The data were obtained in an anonymous fashion: the classes‘ teachers assigned a number to each proband. On the test sheets only those numbers were noted, and the teachers did not see the completed tests.

The age of the probands ranged from 8-11 years, with a mean of 8.73 years, the standard deviation (SD) being 0.69. There were 19.8% of probands (n=36), whose first language was not German. This likens the average population in Berlin, which has a quota of 23.6% for these pupils.

3.1.2 Applied screening instruments

The aim of the screening process was to identify probands with a likely diagnosis of a Specific disorder of arithmetical skills. Thus, the sensitivity of the method had relatively more importance than its specificity. The existing standardized German academic achievement tests for children are grade dependent and designed for use in either the last four months of the designated academic year or the first two months of the next one.

In spite of careful, long-term planning, starting in July of 1995, we obtained the written permission from the authorities cited above only in January 1996. The permission was limited to a period of two months, such as to minimize interference of the study with the academic schedule of the classes. The Berlin school year lasts until mid-July. Hence we were faced with the predicament that in the defined time it was either four months too late for the use of the standardized achievement tests for grade 2 or one month too early for the use of those for grade 3.

30

3.1.2.1 Diagnostic calculation test for third grades (Diagnostischer Rechentest für 3.Klassen - DRE 3)

The DRE 3 can be used as an achievement test as well as a diagnostic test for arithmetical abilities and understanding. It is intended for use in the last four months of the third grade or at the beginning of the fourth grade. The test was standardized in 1971 with 2,975 probands. Its validity was ensured by using items that correspond to the academic curriculum for the third grade. In addition, the standardization sample was compared to the marks of the probands with good correlation. The test‘s reliability was measured at r=0.91 Samstag, Sander, &Schmidt, 1992 .

The DRE 3 contains 44 arithmetical problems, 40 of which are in arabic number format and 4 in story format. To maintain the proband‘s motivation the problems are first relatively simple and then gradually increase in their difficulty. The results for boys and girls differed significantly in the standardization sample. Accordingly, there are different test standards for both sexes. As the DRE 3 was designed as an achievement tests, its distribution has a right shift. Measured are correctly solved problems, with a mean of 28.79 for boys, SD 10.55 and a mean of 27.24 for girls, SD 11.08. The test can be evaluated by the quantity of solved problems. In addition, the character of errors can be assessed in a qualitative evaluation. There are two parallel test forms, such that adjoining probands can not copy results from each other. In order to avoid neglectful errors, the test is not timed. In all eight classes the probands were instructed and handed out the DRE 3 by the same person with the assistance of the teacher. The written instructions for the test‘s application were strictly followed. For further objectivity, congruity and comparability of the results, all classes were tested at similar morning hours. Probands who had completed the test were asked to quietly read or draw. After 70 minutes 179 of the probands (98.4 %) had completed the test. For three probands (1.6%) the test had to be finished after 75 minutes since the allotted time had run out.

3.1.2.2 Diagnostic spelling test for third grades (Diagnostischer Rechtschreibtest für 3.Klassen - DRT 3)

To distinguish low scores of arithmetical achievement representing a Specific disorder of arithmetical skills from those representing overall academic underachievement or low motivation, we assessed the spelling abilities with the DRT 3.

Analogous to the DRE 3 the former is an academic achievement test for third-graders. It is intended for use in the last four months of the third or the first three months of the fourth grade. A first version of the DRT 3 was standardized in 1965.

31

The DRT 3 is constituted of 44 sentences with one word missing, respectively. The probands are handed out the test sheet. Then, the missing word in the first sentence is read by the examiner. Hereafter, the complete sentence is read aloud and finally the missing word is repeated. The test measures committed errors. As the DRT 3 is designed as an academic achievement test in accordance with the academic curriculum for the third grade, the score distribution is skewed to the left, with a mean error-score of 17.43, SD 10.08, found in 3,298 probands with German as their first language. For 392 probands, for whom German is not the first language, a mean error-score of 27.35, SD 9.18 was found. The test can be evaluated by quantity of committed errors and qualitatively, as evaluated by the character of errors Müller, 1991 . The test lasts between 25 and 30 minutes. In our study it was performed by the same person with assistance of the classes‘ teacher in all eight classes. The written instructions for the test‘s application were strictly followed. As with the DRE 3, the DRT 3 was carried out in all classes at a cognate time of the day.

3.1.2.3 Teacher questionnaire

Our study group developed a simple questionnaire for teachers. It was given to the teachers of each class. Since it is unlawful in Germany to submit student‘s marks to third parties, the teachers were asked to judge each proband‘s capacity in mathematics and spelling, respectively. A judgement of ’1‘ meant the highest capacity in the academic field, and a ’4‘ the lowest. With the questionnaire we strove to compare our data to the teacher‘s evaluation and to control for momentary difficulties of the probands. In one of the classes with 18 probands the teacher felt unable to fill out the questionnaire, as she had only recently begun to teach that class.

32

3.2 Screening Results

3.2.1 DRE 3 (Diagnostic calculation test for third grades)

The probands‘ scores in the arithmetical achievement test, separated by gender, are shown in figures 3 and 4.

Fig.3: Arithmetical achievement scores for male probands

Fig.4: Arithmetical achievement scores for female probands

From these scores, the following distributions can be drawn (Table 1):

Tab.1: Average arithmetical achievement scores

	Total			Mitte district (East)			Tiergarten district (West)
Sex	n	M	SD	n	M	SD	n	M	SD
male	100	14.6	10.14	47	14.57	10.51	53	14.62	9.2
female	82	10.38	7.24	44	10.34	6.92	38	10.42	7.67

(n: number of probands; M: mean DRE 3 score; SD: standard deviation)

33

3.2.2 DRT 3 (Diagnostic spelling test for third grades)

The probands‘ error-scores in the arithmetical achievement test, separated by German or foreign descent, are shown in figures 5 and 6.

Fig.5: Spelling raw scores for probands with German as their first language

Fig.6: Spelling raw scores for probands with German not as their first language

The resulting distributions of are shown in Table 2:

Tab.2: Average spelling raw scores

	Total			Mitte district (East)			Tiergarten district (West)
First language	n	M	SD	n	M	SD	n	M	SD
German	146	26.31	10.03	84	24.32	10.13	62	29.01	9.31
Not German	36	32.69	6.82	7	31.00	10.92	29	33.10	5.63

n: number of probands; M: mean of committed DRT 3 errors; SD: standard deviation

34

3.2.3 3.2.3. Teacher questionnaires

3.2.3.1 Mathematics

Figure 7 shows the correlation of the score in the DRE 3 with the teacher‘s evaluation of the student‘s potential in mathematics.

Fig.7: Teacher‘s evaluation of the mathematical potential of a student compared to this student‘s arithmetical achievement test result.

The figure shows some correlation between the teacher‘s evaluation of their student‘s mathematical ability and the corresponding test results in the DRE-3. However, test results in the lowest quartile of the tested probands can be found in the group of students judged by their teachers to be the most capable in mathematics (evaluation score 1) as mathematics scores twice as high and above the average can be found in the group with the lowest estimated potential for mathematics achievement (evaluation score 4). The correlation between the test results and the teacher‘s evaluation was calculated. Pearson‘s correlation coefficient was p = -0.5788.

35

3.2.3.2 Spelling

The correlation of the DRT 3 score with the teacher‘s evaluation of the students spelling potential is shown in Figure 8:

Fig.8: Teacher‘s evaluation of the spelling potential of a student compared to this student‘s spelling test result.

The figure shows some correlation between the teacher‘s evaluation of their student‘s mathematical ability and the corresponding test results in the DRT-3. However, there are test results just below the average of the tested probands in both the group of students judged by their teachers to be the most capable in spelling (evaluation score 1) as well as in the group with the lowest estimated potential for spelling achievement (evaluation score 4). The correlation between the test results and the teacher‘s evaluation was calculated. Pearson‘s correlation coefficient was found to be p= 0.6039.

36

3.3 Discussion of the screening results

In chapter 3.1.2. we explained our choice of screening instruments. Consequently, we expected the probands to score below the norms of the achievement tests. On the other hand, these tests are designed to assess classes‘ academic potential nationwide and they are oriented on the standardized academic curriculum for third grades. Only four to six weeks after our study the temporal criteria for the application of both the DRE 3 and the DRT 3 would have been met. Therefore we were surprised by the degree of low test achievement displayed by the our results.

The average value for boys in the DRE 3 is 28.79 points (SD 10.55). In our sample we found the mean score for boys to be 14.6 points (SD 10.14), a score 49.5% below that of the standardization sample. The results for the girls were analogous: in our study the girls reached a mean score of 10.38 (SD 7.24) in the DRE 3. This is 61.9% lower than the normal mean score of 27.24 points (SD 11.08) for girls. Although the probands reached significantly lower scores than the standardization sample, the standard deviations of the samples were similar. In addition, only 7 (7%) of the boys, and 2 (2.4%) of the girls scored above the 50^th percentile of the norm.

Comparable to that were the results of the test for spelling achievement. Our probands with German as their first language made an average of 26.31 (SD 10.03) errors in the DRT 3. This means that compared to the normal values from the DRT 3 of an average 17.43 (SD 10.08) errors for that group, our probands committed 50.9% more errors. The probands with German not as their first language committed an average of 32.69 (SD 6.82) errors, 19.5% more than the normal value for this group of 27.35 (SD 9.18). The probands with German as their first language in our sample thus reached an average score that resembles the score for probands for whom German is not their first language in the standardization sample of 1982. As for the DRE 3 scores, the standard deviation of both groups were comparable to those of the norm. Only 23 (15%) of probands with German as their first language, and 6 (16%) of probands for whom German is not the first language, scored above the 50^th percentile of the norm. Consequently, the results of the probands in our study for both tests seem to be distributed homogeneously and thus display their factual academic achievement.

As explained in chapter 3.1.2. we carefully avoided achievement tests that were relatively too easy since we intended to discern differences between arithmetical and spelling achievement. We were concerned that with the use of achievement tests for the second grade, the typical data shift towards high achievement scores in such tests would be too pronounced as to make out discrepancies in academic performance. Furthermore, the applied achievement tests have the advantage to test knowledge that is constantly accumulated over schooling time. To ensure good motivation of the probands, both test begin with more basic tasks and gradually increase in difficulty. However, our expectations were not met. The test results of all students were skewed towards the low end of achievement in both the arithmetical and the spelling domain. Since the obtained test scores had a homogenous distribution that was comparable to that of the standardization sample, our data seem to represent a realistic picture of the proband‘s academic abilities.

One explanation of our data would be a lack of motivation. But we could find no

37

What then is responsible for the shift of data towards low achievement scores? The slightly early test administration alone does not seem to be a sufficient explanation. One contributing influence appear to be the relatively old standardization data. The DRT 3 had been last standardized 16 years, and the DRE 3 even 25 years prior to our application of the test. It is known that academic achievement scores in the Western world worsen over generations as IQ-scores increase, as it was shown of the USA in contrast to China by Geary et al. While the younger generation in both samples had higher IQ-scores, the arithmetic abilities of the American sixth-grade students were below those of American adults, in contrast to their Chinese peers Gearyet.al, 1997 . Pfüller and Zerahn-Hartung showed that the orthographic achievement of German students significantly deteriorated in 27 years. They used a spelling test that was standardized in 1968. When they administered the same test in 1995, 48% of the probands scored below the 10th percentile compared to the probands that were tested 27 years earlier Pfüller &Zerahn-Hartung, 1996 . These data are quite analogous to our own findings.

© Die inhaltliche Zusammenstellung und Aufmachung dieser Publikation sowie die elektronische Verarbeitung sind urheberrechtlich geschützt. Jede Verwertung, die nicht ausdrücklich vom Urheberrechtsgesetz zugelassen ist, bedarf der vorherigen Zustimmung. Das gilt insbesondere für die Vervielfältigung, die Bearbeitung und Einspeicherung und Verarbeitung in elektronische Systeme.