WoW

In preparation for the presentation that I'll be doing next week on James Gee, I played World of Warcraft for the first time last night. I installed a 10 day trial on one of the Lounge computers and played under guidance of James S. (who was playing with the character name PotteryBarn). The first thing that struck me was the amount of text and the level of text involved in the game. Unlike other games that I've played, particularly console games like Star Wars or Halo, WoW is a text-intensive environment. It takes some time to get used to where the text will appear in the interface. At one point, James told me that I had picked something up and I asked him how he knew. He pointed out the text on the lower left part of the screen that reported my action. Until he pointed that out, I wasn't attending to that part of the screen at all. As a new player, I didn't know where to place my attention to receive relevant information. As I played, exclamation points appeared at the bottom of the screen. When I clicked on each exclamation point, a text box appeared explaining some aspect of the game that I had just encountered for the first time. These messages were providing me with small pieces of information at the point of need. I was still in control of the display of this text, but it was offered when the context was appropriate, i.e. when I had encountered the concept, the concept was labeled and explained for me.

Being a low level player, my movements are limited to certain areas, while I learn. There are other players around and computer-operated characters that provide structure, guidance, instructions, directions (how to get from A to B), and other kinds of interactions. James explained to me that WoW is unique in that it is scalable in many directions. The game works well for someone who plays eight hours per day and also works well for someone who plays an hour per week. It works for those who wish to go deep (this is drillable) into the back story and for those who primarily wish to fight battles. The game environment supports many different approaches to game play. The largest proportion of the hour or so that I spent playing last night was spent running from one place to another. What objectives I chose to adopt, what goals I completed, all of that was up to me.

For next week, I need to figure out how to communicate all of the information that I want to get across in this environment without detracting from the experience. I know some of them will simply be overwhelmed by the interface. We'll see.

More Thoughts on Formative Design

In Formative Design, the researcher works hand in hand with the classroom teacher to design the study and values the practitioner's knowledge and expertise in every stage of the research. 

Someone asked the question last night about generalizability... something along the lines of, "how do you make what you learn through formative design generalizable to other populations? Do you follow up with experimental or quasi-experimental research?"

This got me thinking about what's really going on. The implication in that question is that the variability that is necessarily introduced when the researcher and the instructor are collaborating during formative design represents the introduction of confounding variables. Each teacher may be implementing things a little bit (or a lot) differently, or even the researcher's presence may create a condition that is impacting student outcomes and can't be recreated in typical classroom settings. But you can't do without, because one of the values of formative design is that interventions shouldn't just be dropped down on practitioners, that teachers should be involved.

Maybe the real intervention, the one that has the most impact on student learning, is treating teachers like professionals and giving them control over what is happening in their classrooms. Maybe simply having a university researcher sitting down and saying, "I value your expertise. Let's work together to design an intervention that will work with your students," maybe that's what really works. Maybe it doesn't matter nearly as much what actual procedure or technique you are using with the students. 

That certainly is different than saying that interventions fail because of issues with fidelity of implementation. Instead, it's saying that interventions fail when teachers are not engaged as professionals in solving problems in their classrooms. I'd love to see a large scale study in which the treatment is formative design research on an intervention and the control is a fidelity model of implementing an intervention. Maybe you should even vary the intervention being implemented, because since the intervention itself may not be the most critical factor.

Formative design

Dr. Dennis visited our class and talked to us about formative design. I like the focus on valuing practitioner knowledge. The pragmatic epistemology also appeals to me.

Formative design differs from action research in its focus on utilizing and developing theory. Differs from experimental and quasi-experimental in that you can modify what you are doing based on what you see happening. It's not quite the same as Mixed Methods.

Part of pragmatism says that even conflicting ideas can be informative. How do conflicting theories come together to inform classroom practice? Praxis is an important concept in formative design.

This approach emphasizes close alignment between theory, research, and (classroom) practice, and it's iterative.

Your approach should be skepticism, not romanticism or advocacy. Your goal isn't to prove your theory is right. You have to balance your beliefs with what is actually working in the classroom.

This is a methodology in its infancy, so researchers working in this area must be explicit in all stages of the research.

Timeline for Formative Experiments

Phase 1: Preliminary phase - recruit schools and teachers, meet with stakeholders, discuss goals, obligations, responsibilities; negotiate plans.

Phase 2: Gather demographic data. Use "ethnographic" methods to create a thick description of the classroom, school, and community.

Phase 3: Gathering baseline data. Establish where participants are in relation to the pedagogical goal prior to implementing an intervention.

Phase 4: Implement the intervention; gather data.

Phase 5: Post assessment to provide comparison to baseline.

Phase 6: Consolidate findings and write it up.

For information on formative design and related issues, see:

Bauman, J., Ware, D., & Edwards, E. (2007). "Bumping into spicy, tasty words that catch your tongue": A formative experiment on vocabulary instruction. The Reading Teacher, 61(2), 108-122.

Johnson, R. B., & Onwuegbazie, A. J. (2004). Mixed methods research: A research paradigm whose time has come. Educational Researcher, 33(7), 14-26.

Reinking, D., & Bradley, B. A. (2008). Formative and design experiments: Approaches to language and literacy research. New York: Teachers College Press.

Reinking, D., & Watkins, J. (2000). A formative experiment investigating the use of multimedia book reviews to increase elementary students' independent reading. Reading Research Quarterly, 35(3), 384-419.

Ivey, G., & Broaddus, K. (2007). A formative experiment investigating literacy engagement among adolescent Latina/o students just beginning to read, write, and speak English. Reading Research Quarterly, 42(4), 512-545.

Dennis, D. (n.d.) Co-constructing the implementation of multi-level texts in middle school science classrooms. Unpublished proposal for research.

This Week's Reading

RED 7745
Historical Research Duke 7
Content Analysis Kamil 3
History of Reading, NRC 2007
Monaghan 2007
Oral History, King and Stahl
Stahl & King, History
Stahl (by King)

EDF 7265
Tomasello, M. & Brooks, P.J. (1999). Early syntactic development: A Construction Grammar approach (pp. 161-190). In M. Barrett (ed.), The Development of Language. East Sussex, UK: Psychology Press Ltd.

EDF 7408 Stats 2
S: Ch 4; C&S: Ch 7

Class notes, Reading Research

Be open to the expertise of others, partners in research. Don't let ego get in the way. Your methodological decisions are dictated by your research questions and your orientation toward the data.

Kinch says, the higher the proposition density, the harder it is to understand a text.

Protocol analysis is inductive, a window into the cognitive process. You are making inferences to construct a model the cognitive process, a model of a reader's thoughts.

Subjects are trained in protocol analysis before data collection with similar task using different material. During training, reinforce process talk, not product talk. By doing this, you are skewing the verbal protocol toward the type of data you are looking for. If you don't train subjects, you can get a lot of unusable data. When you do, you run the risk of not finding things that you aren't looking for.

Subjects have to be engaging in goal-directed behavior so that you can collect data; they have to be trying to do something, not just read without a specific purpose.

The researcher's verbal prompts need to be on topic. Talking about making a hamburger while cleaning a carburetor will disrupt the cognitive process. In protocol analysis, you talk about the task that you are doing; it doesn't disrupt, it just slows down cognitive processing.

Just enough categories to account for the data. Start broad, with many categories, then check back with the data to narrow.

Next week, historical analysis.
Harvey Graff looks at what literacy does to cultures. He sees literacy's outside impact on culture, separate from culture. Doug Hartman and Jennifer Monahan research the history of reading. Also, content analysis and document analysis. Jim King and Norm Stahl work with oral histories. We will be looking at USF's collection of educator oral histories on iTunes U. Lastly, we will be looking at how one situates the history of reading for course delivery.