Assessing symbol sense in a digital tool CADGME, Hagenberg July 2009 Christian Bokhove FISME, St. Michaël College

Context Christian Bokhove 11 yr Teacher maths/ict secondary school St. Michaël College, Zaandam, the Netherlands, tradition math/ict projects Phd research. (www.dudocprogramma.nl) aimed at math curriculum. Freudenthal Institute of Science and Mathematics Education, Utrecht University, the Netherlands Supervisors: Paul Drijvers and Jan van Maanen Educational research

Problem statement Transition secondary  higher education Use of ICT Lack of Algebraic expertise Entry exams Use of ICT “Use to learn” vs. “Learn to use” Position statement NCTM (2008): ICT can be a valuable asset

In what way can the use of ICT support acquiring, practicing and assessing relevant mathematical skills Assessment - Formative (for) v Summative (of) - Feedback (Black & Wiliam, 1998) ICT tool use - Instrumentation - Task, technology, theory (Lagrange, 1999) Algebraic expertise - Basic skills - Symbol Sense (Arcavi, 1994) Acquiring, practicing and assessing relevant mathematical skills with ICT can be scrutinized in three relations: Tool use and assessment. ICT tools can be used efficiently for formative and summative assessment, providing functional feedback. Tool use and algebraic skills. Through instrumentation and instrumental genesis an ICT tool can be used for acquiring algebraic skills and vice versa. Because of the fact that technology, task and theory go hand in hand, ones understanding of algebra also shapes ICT tool use. Algebraic skills and assessment. Formative assessment enables us to study the qualitative aspect of basic skills and ‘real understanding’. This focus on formative aspects is closely related to symbol sense. Assessment frameworks for mathematics form the background for the relation between algebraic skills and assessment. We aim to integrate the assessment of algebraic skills into one prototypical design, used for giving us insight into these three aspects and their relations. Christian Bokhove

Criteria for tools Evaluation instrument Externally validated First formulate want we want, then see what there is A selection: Assesses both basic skills and symbol sense; Provides an open environment and feedback to facilitate formative assessment; Stores both answers and the solution process of the student; Steps; Freedom to choose own strategy; Authoring tool for own questions; Intuitive interface (‘use to learn’ vs. ‘learn to use’) Close to paper-and-pencil notation;

Digital prototype www.fi.uu.nl/dwo/voho (enter as guest, at the moment in dutch) Digital Mathematics Environment (DME) www.fi.uu.nl/dwo/en for English version 30 items basic skills & symbol sense Designer: Peter Boon, always in close collaboration with teacher’s field. Store results in environment SCORM, so every module can be used in VLE’s including Moodle

Case studies / 1-to-1s 6 multihour think-aloud 1-to-1 sessions with 17/18 year olds I want to know what’s going on in their minds Qual. analysis (video, camtasia, atlas TI) Quality of tool (no focus) Symbol Sense Feedback

Symbol Sense Four example exercises 1. Equations with common factors Solve 2. Wenger, 1987 Rewrite as v=

Four examples (continued) 3. Does the student recognize the quadratic terms? 4. Recognize common factors when rewriting

First example: student’s work More examples Video clips

Feedback Feedback is part of formative assessment. Types of Feedback (Hattie & Timperley, Uni. Waterloo) Corrective Procedural Syntactical Meer.. From the 1-to-1’s we distilled modifications for our protoype (Matrix items vs. Feedback) Content Tool itself Feedback to be added Logging feature (for research) Second cycle with large group

DEMO module http://www.fi.uu.nl/dwo/en/ Random vars. I forget one solution, and get the above Custom feedback. Just divide by quadratic term. Work towards form x3 Added random variables but fixed Random variables Note: this adds complications. The author has to think about the implications. Feedback rules Features: applets This feedback can be authored

Improving the tool Latest developments Mathematica connection. Enables: (Note: secondary school algebra in the Netherlands only needs a small amount of traditional CAS) Notation d/dx (Chain Rule) Limits (left, right, infinite) Substitutions (e.g. Chain Rule) Integrals (also +C) More sophistication in feedback Feedback rules (webservice connection with research Jeuring, Open University) Integration of tools like Geogebra, graphing tool, rotating cubes (All benefits of a close collaboration with the designer)

New developments: integrals

New developments: rule feedback Het gaat alsvolgt: ik stuur een vergelijking op met de vraag om een afleiding. Dan krijg ik een lijst voor de stappen terug met 1. De toepaste regels Dat kunnen er 6 verschillende zijn: - remove division - scale to one - variable to left - constant to right - distribute - merge terms 2. De vergelijkingen na toepassing van de betreffende regel. Wat met name lastig was voor hun, is de formulering van afleidingsregels met de juiste granulariteit, zodat die passen bij de intuitie van de leerling. Soms worden bij herschrijvingen 10 regels toegepast terwijl wij dat normaal in een stap doen. Daar zijn ze mijns inziens redelijk in geslaagd, al is verbetering mogelijk (zie mijn reactie) (Stelling van mij: Didactisch handelen is, in interactie met de leerling spelen met deze verschillende niveaus van granulariteit) Wat ik bij mijn feedback gedaan heb is deze regels vertaald en (bij tip2) de oude vergelijking (die ik opstuurde) en nieuwe vergelijking (de eerste stap van de afleiding) toegevoegd. Ik stuur dus steeds de vergelijking op van de laatste goede stap van de leerling en krijg dan een afleiding vanaf dit punt. Alleen bij solve doet ie alles zelf vanaf de eerste vergelijking in een keer. Echt interessant wordt het bij de 'submit-service', waarbij je de vergelijking + laatste stap van de leerling + nieuwe stap van de leerling opstuurt. Dan krijg je terug: SyntaxError. The submitted term is syntactically incorrect. Buggy. One or more buggy rules match, a list with their identifiers is returned. NotEquivalent. The student has made a mistake. Ok. The submitted expression is equivalent and one or more rules have been applied, following the strategy. A list of identi ers of applied rules and a new state are returned. Detour. The submitted expression is equivalent but one or more of the applied rules do not correspond to the strategy. A list of rule identi ers Unknown. The submitted expression is equivalent but none of the known rules match. Hoe 'intelligent' de feedback is die hieruit kan worden afgeleid (met name bij ingewikkelde vergelijkingen) moeten we natuurlijk nog afwachten. Maar wel een interessante mogelijkheid. Meer informatie over de service heb ik bijgevoegd. Ook een artikel over de bovengenoemde granulariteit.

New developments: GeoGebra integration

Closing statement Educational research: CAS serves education and not the other way round More info: http://www.fi.uu.nl/~christianb http://www.fi.uu.nl/dwo/en

Second example: student’s work

Third example: student’s work

Fourth example: student’s work

Student work: clips First example Martin tries to solve the first exercise Movie clip Second example Barbara tries the Wenger exercise. Movie clip