Assessment of teaching OiO college 6 (groep 4). Tussenevaluatie Goed Inzicht in onderzoeksresultaten/methoden en inzicht in bruikbaarheid daarvan voor.

Presentatie over: "Assessment of teaching OiO college 6 (groep 4). Tussenevaluatie Goed Inzicht in onderzoeksresultaten/methoden en inzicht in bruikbaarheid daarvan voor."— Transcript van de presentatie:

1 Assessment of teaching OiO college 6 (groep 4)

2 Tussenevaluatie Goed Inzicht in onderzoeksresultaten/methoden en inzicht in bruikbaarheid daarvan voor eigen onderwijs Enthousiaste docent ( ) Niet goed Teveel leeswerk, Vreselijke artikelen, Ratio inzet/opbrengst Te lang over onderzoekend leren Aandachtspunten Duidelijker opdrachten en cursusdoelen Werkdruk gelijk aan andere OiO-groepen Onderzoek naar docent (PCK)

3 Identify Purpose of Assessment Diagnosis (to clarify the type and extent of learners’ learning difficulties in light of well-established criteria, for intervention); Screening (to identify learners who differ significantly from their peers, for further assessment); Qualification (to decide whether learners are sufficiently qualified for a job, course or role in life – that is, whether they are equipped to succeed in it – and whether to enroll them or to appoint them to it); Licensing (to provide legal evidence – the license – of minimum competence to practice a specialist activity, to warrant stakeholder trust in the practitioner); Programme evaluation (to evaluate the success of educational programmes or initiatives, nationally or locally); Comparability (to guide decisions on comparability of examination standards for later assessments on the basis of cohort performance in earlier ones) Therefore, the first step in evaluating your assessment practices is to identify the purpose(s) of each assessment practice.

4 CE, SE – Qualification, maar ook… Goede scholen hebben jaar in jaar uit kleine verschillen tussen de twee soorten examens en vaker relatief hoge cijfers voor beiden. Scholen met weinig toegevoegde waarde hebben veel grotere verschillen tussen de twee examentypen. Het grootste verschil tussen de twee examensoorten doet zich voor bij vwo-scholen. Gemiddeld geeft een vwo-afdeling 0,57 meer voor het schoolexamen dan het centraal schriftelijk, op de havo is dat maar 0,27. Vakken als Duits, Nederlands en maatschappijleer springen er uit met gemiddeld een schoolexamen-cijfer dat gemiddeld 0,7 punt hoger ligt dan het centraal schriftelijk. Het kleinst (minder dan 0,2 punt) zijn de verschillen bij biologie, scheikunde en wiskunde. Bij bijna alle vakken zijn er vwo-scholen met een verschil van meer dan twee punten. Een groot en langdurig verschil is volgens Dronkers een goede indicator voor lage kwaliteit bij een vak of zelfs een school. Er is in de discussie opmerkelijk genoeg weinig aandacht is voor de CORRELATIE tussen SE-cijfers en CE cijfers…. Bron: http://wp.digischool.nl/docentvo/2012/04/28/verschillen-in-cijfers-se-en-ce/http://wp.digischool.nl/docentvo/2012/04/28/verschillen-in-cijfers-se-en-ce/

5 Rekentoets: qualification and (programme?) evaluation

6 PISA: evaluating national educational systems

7 Threats to validity & societal acceptance Framework (Relevancy, Acceptance, Curriculum coverage) Items (implementation of framework) Coding of items (scoring, categorisation) Cultural bias/translation issues Participants (Sampling, Effort put in) Analysis (Multilevel, Item response ….) Van Halen

8 Door bemoeienis van het Freudenthal Instituut is rekenen vervangen door raadseltjes, vaak slecht geformuleerd – een ramp voor de wiskunde, waar juist alles in het werk wordt gesteld om dubbelzinnigheid uit te sluiten. Op andere vakgebieden worden mechanismen vervangen door verhalen, feiten door meningen. (Vincent Icke, NRC, maart 2015)

9 Is Pisa fundamentally flawed? But what if there are “serious problems” with the Pisa data? What if the statistical techniques used to compile it are “utterly wrong” and based on a “profound conceptual error”? Suppose the whole idea of being able to accurately rank such diverse education systems is “meaningless”, “madness”? What if you learned that Pisa’s comparisons are not based on a common test, but on different students answering different questions? And what if switching these questions around leads to huge variations in the all- important Pisa rankings, with the UK finishing anywhere between 14th and 30th and Denmark between fifth and 37th? What if these rankings - that so many reputations and billions of pounds depend on, that have so much impact on students and teachers around the world - are in fact “useless”? Published in TES magazine on 26 July, 2013

10 Item response curves Treshold vs. determination

11 Incomplete block design The items were presented to students in thirteen standard test booklets, with each booklet being composed of four clusters, hence two hours of test time. Clusters labelled PM1, PM2, PM3, PM4, PM5, PM6A and PM7A denote the seven paper-based standard mathematics clusters, PR1 to PR3 denote the paper-based reading clusters, and PS1 to PS3 denote the paper- based science clusters. PM1, PM2 and PM3 were the same three mathematics clusters as those administered in 2009, and the remaining clusters would comprise new material. Two of the three reading clusters were intact clusters used in 2009. The remaining reading cluster was based on a cluster used in 2009 but with one unit substituted. The substitution was made after the 2010 oil spill in the Gulf of Mexico rendered a unit about the idyllic nature of the Gulf unusable. The three science clusters were intact clusters used in PISA 2009. The cluster rotation design for the standard booklets in the Main Survey corresponds to designs used in previous PISA surveys and is shown in Figure 2.1. This is a balanced incomplete block design. Each cluster (and therefore each test item) appears in four of the four-cluster test booklets, once in each of the four possible positions within a booklet, and each pair of clusters appears in one (and only one) booklet. An additional feature of the PISA 2012 test design is that one booklet (booklet 12) is a complete link, being identical to a booklet administered in PISA 2009. Each sampled student was randomly assigned to one of the thirteen booklets administered in each country, which meant each student undertook two hours of testing.

12 Non-significance explicitly reported in PISA

13 Maar meten de items wat ze willen meten? The claim put forward here is that the survey test situation in point of fact has important problem- setting requirement characteristics that modify the problem determination and possible solution demarcation decisively; and that the claim of the PISA studies to assess ‘knowledge and skills for life’ is invalidated by the failure to take this fact into account. Dohn, N. B. (2007). Knowledge and skills for PISA—Assessing the assessment. Journal of Philosophy of Education, 41(1), 1-16.

14 Jessica on chocolate http://www.oecd.org/edu/school/programmef orinternationalstudentassessmentpisa/336927 93.pdf#page=88