Things I Love Right Now

• Deviled crabs from Brocatto's in Ybor City. Fantastic.
  
• Weck's Deli in Land'O'Lakes. The Bailey Avenue sandwich is wonderful.
  
• Battlestar Gallactica. Holy cow, that's good stuff. Only a few episodes left and I have my theories...
  
• Lost. I've stuck with the show, even when I hated what they were doing with it, but I'm loving it right now. And I have my theories...
• Oscar Movies. We've seen Vicky, Cristina, Barcelona, Changeling, Frozen River, and Revolutionary Road in the past week. We've seen all of the noms for Best Pic, Direction, and all of the acting awards, except Robert Downey Jr. in Tropic Thunder and all of the screenplay nominees except for In Bruges. I love movies and I love sharing them Michael.

More Thoughts on Formative Design

In Formative Design, the researcher works hand in hand with the classroom teacher to design the study and values the practitioner's knowledge and expertise in every stage of the research. 

Someone asked the question last night about generalizability... something along the lines of, "how do you make what you learn through formative design generalizable to other populations? Do you follow up with experimental or quasi-experimental research?"

This got me thinking about what's really going on. The implication in that question is that the variability that is necessarily introduced when the researcher and the instructor are collaborating during formative design represents the introduction of confounding variables. Each teacher may be implementing things a little bit (or a lot) differently, or even the researcher's presence may create a condition that is impacting student outcomes and can't be recreated in typical classroom settings. But you can't do without, because one of the values of formative design is that interventions shouldn't just be dropped down on practitioners, that teachers should be involved.

Maybe the real intervention, the one that has the most impact on student learning, is treating teachers like professionals and giving them control over what is happening in their classrooms. Maybe simply having a university researcher sitting down and saying, "I value your expertise. Let's work together to design an intervention that will work with your students," maybe that's what really works. Maybe it doesn't matter nearly as much what actual procedure or technique you are using with the students. 

That certainly is different than saying that interventions fail because of issues with fidelity of implementation. Instead, it's saying that interventions fail when teachers are not engaged as professionals in solving problems in their classrooms. I'd love to see a large scale study in which the treatment is formative design research on an intervention and the control is a fidelity model of implementing an intervention. Maybe you should even vary the intervention being implemented, because since the intervention itself may not be the most critical factor.

Formative design

Dr. Dennis visited our class and talked to us about formative design. I like the focus on valuing practitioner knowledge. The pragmatic epistemology also appeals to me.

Formative design differs from action research in its focus on utilizing and developing theory. Differs from experimental and quasi-experimental in that you can modify what you are doing based on what you see happening. It's not quite the same as Mixed Methods.

Part of pragmatism says that even conflicting ideas can be informative. How do conflicting theories come together to inform classroom practice? Praxis is an important concept in formative design.

This approach emphasizes close alignment between theory, research, and (classroom) practice, and it's iterative.

Your approach should be skepticism, not romanticism or advocacy. Your goal isn't to prove your theory is right. You have to balance your beliefs with what is actually working in the classroom.

This is a methodology in its infancy, so researchers working in this area must be explicit in all stages of the research.

Timeline for Formative Experiments

Phase 1: Preliminary phase - recruit schools and teachers, meet with stakeholders, discuss goals, obligations, responsibilities; negotiate plans.

Phase 2: Gather demographic data. Use "ethnographic" methods to create a thick description of the classroom, school, and community.

Phase 3: Gathering baseline data. Establish where participants are in relation to the pedagogical goal prior to implementing an intervention.

Phase 4: Implement the intervention; gather data.

Phase 5: Post assessment to provide comparison to baseline.

Phase 6: Consolidate findings and write it up.

For information on formative design and related issues, see:

Bauman, J., Ware, D., & Edwards, E. (2007). "Bumping into spicy, tasty words that catch your tongue": A formative experiment on vocabulary instruction. The Reading Teacher, 61(2), 108-122.

Johnson, R. B., & Onwuegbazie, A. J. (2004). Mixed methods research: A research paradigm whose time has come. Educational Researcher, 33(7), 14-26.

Reinking, D., & Bradley, B. A. (2008). Formative and design experiments: Approaches to language and literacy research. New York: Teachers College Press.

Reinking, D., & Watkins, J. (2000). A formative experiment investigating the use of multimedia book reviews to increase elementary students' independent reading. Reading Research Quarterly, 35(3), 384-419.

Ivey, G., & Broaddus, K. (2007). A formative experiment investigating literacy engagement among adolescent Latina/o students just beginning to read, write, and speak English. Reading Research Quarterly, 42(4), 512-545.

Dennis, D. (n.d.) Co-constructing the implementation of multi-level texts in middle school science classrooms. Unpublished proposal for research.

Pragmatic Use of Reading Models

None of our metaphors for reading are complete. None are practical in every situation. None are true, whatever that means, in every context.

That means that, although texts are intertextual and meanings are conditional and provisional, simpler models of reading may be appropriate, depending on the context, the audience, and the purpose. It also means that as researchers we are free to use a cognitive model of reading when the situation calls for it, even if we believe a strict cognitive model to be inaccurate. All models of the reading process are inaccurate. They are all imperfect approximations. Still, all models of reading aren't equivalent or equally supported by systematic inquiry. It's just that, incomplete as it is, a cognitive metaphor, for instance, may be the most efficacious approach in a given situation. It's not ideological agnosticism, but it's something like methodological pragmatism.

Defining 'Text'

Reading a 1992 piece by Doug Hartman. I like this approach to defining text.

"The text has been rendered by intertextually-informed research in two ways. Both, influenced considerably by literary theory and semiotics, require an altered notion of what constitutes a text. Although we usually think of the text as the object one reads-a textbook, a section of a passage, or the alphanumeric code printed on a page-it need not be confined to the boundaries of printed language. A text includes both linguistic and nonlinguistic signs. A text can be an utterance, a gesture, a thought, a structure, a function, or a piece of art, music, or drama (Rowe, 1987; Short. 1986; Siegel, 1984); in this more inclusive sense, a text is any sign that communicates meaning."

with the end note:

"For the sake of clarity and consistency, I will generally use “the text” to refer to the physical, linguistic object, and “text,” “a text, ” “textual resource,” and “textual utterance” to refer to the more inclusive conception of a linguistic or nonlinguistic sign that communicates meaning. "

Here's the full ref:
Hartman, D. (1992). Intertextuality and Reading: The Text, the Reader, the Author, and the Context. Linguistics and Education, 4, 295-311.

And those other refs:
Rowe, D.W. (1987). Literacy learning as an intettextual process. In J.E. Readence & R.S. Baldwin (Eds.). Research in literacy: Merging perspectives, Thirty-sixth Yearbook of the National Reading Conference (pp. lOl- 1 12). Rochester, NY: National Reading Conference.

Short, K.G. (1986). Literacy as a collaborative experience. Unpublished doctoral dissertation,
Indiana University, Bloomington.

Siegel, M.G. (1984). Reading as signification. Unpublished doctoral dissertation, Indiana University, Bloomington.

Bates 2004, Language deficits across groups

Bates, E. (2004). Explaining and interpreting deficits in language development across clinical groups: Where do we go from here? Brain and Language, 88, 248-253.

This commentary by Elizabeth Bates accompanies a group of articles in a 2004 issue of the journal Brain and Language. The studies deal with clinical populations -- children with Williams syndrome, children with Downs syndrome, children with left-hemisphere damage (LHD), right-hemisphere damage (RHD), "late talkers", children with the diagnosis of specific language impairment (SLI), and others. The articles make comparisons of language development across these populations.

Bates points out three findings of the cross study work (first two from Holland):

1. Kids with a variety of impairments experience the same sequence and types of problems, although at different rates. The metaphor suggested is that the problem space, English, is like a highway. These kids are all stuck in the same slow lane.
2. Young kids who have either RHD or LHD usually move into the normal range in elementary school. They seem to "catch up" somewhere between the ages of 3 and 5.
3. Kids with Williams syndrome perform language tasks that are in keeping with their mental age. This contradicts other research that suggests that those with Williams syndrome are capable of complex grammatical constructions and sophisticated speech far beyond their levels of cognition. The claims of language savant status for Williams syndrome kids would have been used to challenge the notion of cognitive prerequisites for language.

Regarding the first finding above, Bates asks, if RHD/LHD kids catch up, why can't those with Williams or other impairments? Bates refers to Holland's supposition that, while the otherwise healthy brain of a young child who has an injury to a specific location is sufficiently plastic and adaptable to reroute language processing around the injury, other disorders are either located in "gate-keeper" systems or too

 broadly distributed to allow the brain to compensate. 

Some research implications of these findings noted by Bates:

• More research should be done using subjects with 'normal' language abilities placed under stressful processing situations in an attempt to simulate language processing disorders.
• More longitudinal studies across the language acquisition period.
• The need to recognize a new metaphor for language development. Rather than a Swiss Army knife, filled with specialized tools for specific situations, Bates compares the language-enabled brain to a giraffe's neck -- adapted to serve a new function (reaching high leaves) while retaining its older functions (turning the head, passing air and food). Bates refers to the "Functional infrastructure for language" chart to show older adaptations combining to provide the basis for language. 

To extend that implication further, it's possible that language impairment always results from the impairment of one of the subsystems. There is no language impairment; only an impairment of one or more of the systems that allow language acquisition to occur. "Non-linguistic deficits can have serious consequences for language, temporarily or on a more protracted basis." p. 252. As Bates points out, this may have major implications for diagnosis and treatment of language disorders.

Podcast Podcast

At IRA in Phoenix, I'll be presenting about the nuts and bolts of podcasting. I'm developing a short podcast series to accompany the presentation. 

The topics so far:

Episode 1: What is podcasting?

Episode 2: Overview of the basic steps

Episode 3: What do I need?

Episode 4: How do I do it?

Episode 5: Planning Your Podcast

The audience for the series is practitioners, so I want to keep everything grounded in classroom practice. I'd like the tone to be informal. I think it will be tricky to strike just the right balance with the tone. First step, develop the scripts! I think I'll use Celtx.

Vellutino & Schatschneider 2004

Vellutino, F., & Schatschneider, C. (2004). Experimental and quasi-experimental design in literacy research. In (chapter 7) N. Duke, & M. Mallette, editors. Literacy Research Methodologies. New York: Guilfords.

(aka Duke Ch. 7)

These authors did an outstanding job of summarizing and contextualizing the issues and parameters surrounding experimental and quasi-experimental design. I agree with Melissa that this chapter should be required reading in Stats. I think the difference here is in perspective. These writers place statistical concepts and procedures firmly within the context of literacy research. In Stats texts, I think the presentation of the material concentrates on the details without the context - and that's probably reasonable since we are learning concepts and procedures that are common to many disciplines and our classmates are not all interested in literacy research. However, that is a good argument for a quantitative statistical measurement class taught specifically for literacy researchers within the Reading program.

Vellutino & Schatschneider propose evaluation of experiments and quasi-experiments based on the adequacy of the hypothetical counterfactual and how well the design addresses concerns about internal validity, external validity, statistical conclusion validity, and construct validity. They give examples of how one might evaluate research using their own studies as exemplars. They make it clear that all studies have weaknesses and flaws and that understanding the flaws is important to understanding how to connect and apply research beyond the contexts of the original studies.

I think that this chapter (and its references) will be useful for evaluating existing research and designing new studies.

This Week's Reading

RED 7745
Historical Research Duke 7
Content Analysis Kamil 3
History of Reading, NRC 2007
Monaghan 2007
Oral History, King and Stahl
Stahl & King, History
Stahl (by King)

EDF 7265
Tomasello, M. & Brooks, P.J. (1999). Early syntactic development: A Construction Grammar approach (pp. 161-190). In M. Barrett (ed.), The Development of Language. East Sussex, UK: Psychology Press Ltd.

EDF 7408 Stats 2
S: Ch 4; C&S: Ch 7