GIRLS OUTPERFORMING BOYS IN LISTENING AND READING SKILLS TRUE GAP OR PSYCHOMETRIC ERROR? Koen aesaert

PROBLEM STATEMENT

INCONCLUSIVE RESULTS ON RELATIONSHIP GENDER-reading/listening

OnderzoeksvraGEN Kenmerken huidige klaspraktijk voor Nederlands? Beheersing van (een deel van) de eindtermen Nederlands lezen/luisteren/(schrijven)? Systematische verschillen in de beheersing tussen specifieke groepen van leerlingen/scholen? Evolutie in die beheersing tussen 2013 en 2018? Bijkomende kenmerken gerelateerd aan de prestaties Nederlands?

OnderzoeksvraGEN Kenmerken huidige klaspraktijk voor Nederlands? Beheersing van (een deel van) de eindtermen Nederlands lezen/luisteren/(schrijven)? Systematische verschillen in de beheersing tussen specifieke groepen van leerlingen/scholen? Evolutie in die beheersing tussen 2013 en 2018? Bijkomende kenmerken gerelateerd aan de prestaties Nederlands?

NATIONAL assessment FRENCH flanders 2017 READING GIRLS BOYS LISTENING

Females > males French (Burstall, 1975; Murphy, 2010) English (Boyle, 1987; Carr & Pauwels, 2006) German (Murphy 2010)

Females ≤ males Spanish (Burstall, 1975; Murphy, 2010) English (Boyle, 1987)  Inconclusive evidence!

ITEM BIAS – DIFFERENTIAL ITEM FUNCTIONING

Item functioning Language proficiency (theta) ITEM CHARACTERISTIC CURVE (ICC) Item functioning Language proficiency (theta)

DIFFERENTIAL ITEM FUNCTIONING SAME PROFICIENCY ‡ SAME CHANCE boys girls Language proficiency (theta)

DIFFERENTIAL ITEM FUNCT PSYCHOMETRIC VALIDITY CONTENT VALIDITY

Research objective

INVESTIGATE THE RELATIONSHIP BETWEEN GENDER AND STUDENTS’ LISTENING AND READING SKILLS (FL) WHEN DIF IS TAKING INTO ACCOUNT

METHODS

Case: NATIONAL assessment French primary education 2017

AIM To: Investigate students’ mastery of attainment targets French: reading comprehension and listening comprehension Identify differences in mastery between specific groups of students Identify relationships with background characteristics

TESTS Reading comprehension: - 21 items - MC - binary coding Listening comprehension: - 22 items

TESTS What’s this poem about? A about the parts of the body B about the seasons C about an old man who’s sick D About the weather in the winter

sample 2098 students 78 schools 114 classes May 30, 2017 w

sample Boy Girl

sample Exclusively Dutch Exclusively French Dutch and French Dutch and other (no French) French and other (no Dutch) Exclusively other

MAIN RESULT READING GIRLS BOYS LISTENING

analysis

ANALYSIS DIF ANALYSIS: - Lord’s chisquare (difR-package) - Creating DIF variables RELATIONSHIP GENDER AND READING/LISTENING PROFICIENCY: - IRT based GLMM - two models: model 1: baseline model model 2: DIF as predictor

RESULTS

Dif ANALYSIS

DIF items favoring girls Dif ANALYSIS DIF items favoring girls reading listening Item5: LE_07_T02 Item3: LU_01_T31 Item9: Le2_2_38_H Item5: Lu1_2_14 Item13: Le2_2_80_H Item16: LE_08_T42 Item18: LE_05_T49

Dif ANALYSIS

Relationship gender and listening/reading proficiency

reading .59 chance .086 higher chance .066 higher chance

LISTENING .70 chance .040 higher chance .043 higher chance

So…

So… GENDER DIFFERENCES IN FOREIGN LANGUAGE SKILLS EXIST… … AND REMAIN WHEN PSYCHOMETRIC ERROR (for gender!!!) IS TAKING INTO ACCOUNT.

GIRLS OUTPERFORMING BOYS IN LISTENING AND READING SKILLS TRUE GAP OR PSYCHOMETRIC ERROR? Koen aesaert