Wednesday, April 9, 2008

Conceptual Assessment Framework - or, in other words, dynamic interrelated rubrics on steroids!

Gibson, et. al., Chapter 15, p.308-322 has my head spinning and I really need to reread it but time does not permit at present; all the well as some soaking in and rumination need to occur before a reread would do any good.   I think what Gibson is saying is that simulations can provide educational opportunities on many different levels depending on who is the intended audience, who is the assessor and what is being assessed and the extent of formative vs. summative evaluation and how it is being used to determine overall assimilation of new knowledge and understanding.  What has me completely baffled is what the interrelationships would have to look like in the relational database that would be needed use to track and draw conclusions necessary to support feedback on what type and how much learning occurred; and how this relational database would relate to the table of rubric data necessary to quantify data received.  Pages 318 and 319 seem to provide some clues that changes in reasoning from Level I to Level II can be identified and accumulated to provide some of this evidence but setting it up would still seem quite a challenge.

Gibson et. al. chapter 16 and especially beginning with page 330 started to shed some light on my baffling in chapter 15.  The Navigational Path Model looks strangely like a self-similar geometric construction otherwise known as Fractals and partially explained by Chaos Theory.  

While chapter 16 doesn't seem to overtly imply that chaos theory or fractals could be useful in organizing and predicting student performance as a form of assessment, it does lay the groundwork by showing that avatar tracking generates a significant amount of data that varies over time, is semi-predictable based on varying inputs and generally knowable within a range of outputs.  "Using the Matrix as an example, a node can be a white rabbit tattoo, or the choice of a red or blue pill.  It is the point in time when the players are required to make a choice that could potentially lead them down a different path" Gibson et. al 2007, p. 330.  References are also provided to Information Foraging Theory by Pirolli and Card 1999 which does have fractal and chaos theory inferences as shown by a quick search of the web.

Off of my wild tangent and back to the gist of chapter 16, rules and procedures are recommended to facilitate organization and assessment of game play using more or less traditional data mining methods given definitive and quantifiable outcomes that were included during the original design of the game.  Game design for assessment includes the use of ADDIE crossed with data flow diagraming and pseudocoding.  While these traditional methods of assessment are proven and very workable using traditional database technology, I wonder about their ability to anticipate for conditions that could indicate learning or new knowledge development outside of what was anticipated by the designer's ability to predict or foresee and thus dismissed as irrelevant or incorrect.  Until new methods of assessment that are not specifically related to existing database analysis methods can be developed and proven, our best bet might be to conclude as Loh did on p.344 of Gibson et. al. 2007 when he surmised that the teacher's role in learning in the online gaming environment may best be that as Dungeon Master, host and referee and ultimately assessor of relevant learning.  And perhaps aided by the best implementation of avatar and other tracking systems that provide the most relevant data necessary to the teacher for the real work of facilitation of and assessment of learning.

Well it's 11:50pm and Chapter 17 and 18 are up but out of time to read this evening - I need more hours in the day and more days in the week...

A quick scan of the remainder of this week's reading seems to show that my conclusion of chapter 16's reading may be partially answered in chapter 17 and partially affirmed by chapter 18.  More to come after a good night's sleep - looking forward to tomorrow's discussion in SL on EdTech Island at 8:00pm est.








http://en.wikipedia.org/wiki/Chaos_theory
Despite initial insights in the first half of the century, chaos theory became formalized as such only after mid-century, when it first became evident for some scientists that linear theory, the prevailing system theory at that time, simply could not explain the observed behaviour of certain experiments like that of thelogistic map. What had been beforehand excluded as measure imprecision and simple "noise" was considered by chaos theories as a full component of the studied systems.