Sunday, December 27, 2009

Notes on Tim Small (part 3)

This is part 3, last of a series. Back to part 2

In this post I look at the ways Small applies his work to self- and peer- learner assessments, and how that might be useful for my research.

The practical application of the theoretical framework and analytical methods described in the article is in techniques for assessing learners that respect the continuum from aesthetic and experiential to efferent, without necessarily privileging one end of the spectrum. Small states that his approach "enables the products of enquiry-based learning and creative activity to be assessed objectively, but by criteria sympathetic to the experience they represent, rather than simply by arbitrary, external standards of accuracy, correctness or comparison." (p. 269)

This feels very much in accordance with where I am coming out with my research: a methodology for self- and peer- assessment of participatory representational practice, based on a set of values that foreground aesthetic experience and engagement (especially since they are usually passed over or marginalized), but don't leave out the "practical", "shared" and "constructive" aspects of working with groups in some sort of applied setting.

The main drive in Small's paper is developing criteria for learning assessment. Essentially, he proposes 'criteria for the criteria', providing a basis for the set of criteria he proposes. He looks for assessment criteria that, first, recognize and uphold the personal drive and responsibility of the learner (p. 261). For "learner" you can certainly substitute "practitioner." Second, the criteria should reflect the dynamic nature of the "journey" a learner experiences, between "self and text", "inner and outer", etc. For a practitioner, each session is in effect such a "journey", moving between (in varying ways) the aesthetic and efferent poles.

Third, he wants to give "safe passage" to the personal knowledge being created. In the analyses I've done to date, this part is perhaps less applicable, since practitioners in sessions are not there to develop their own personal knowledge, though they do draw on and perhaps build up such knowledge. However, this could be very applicable to the kind of self/peer assessment exercise that I took a first step toward at my IFVP session.

Finally, the criteria should respect the (quoting Fish 1980) "authority of the interpretive community". That part also is probably not as applicable to my research since the kind of practice I'm looking at isn't embedded in one particular institution or context. That is, there is not yet an existing "interpretive community", though I sure wish there was one. Helping to create such a community might be a possible future outcome.

In my research, I've come up with dimensions and criteria and observed them in practice, saying how they are exemplified in practitioner moves and choices. Is the next step to have these be self- and peer-assessed? As Small argues, engagement and commitment can only really be "accessed" through "subjective" self-evaluation (p. 268). Does this also mean that, for example, CEU can't really be assessed (different from "observed") except by either participants or practitioners themselves?

This conundrum does seem to keep recurring in some discussions of what I've done to date. What good does it do for an external observer (as I've been doing) to make assessments? Doesn't the whole point then hinge on how good the external observer is at making such assessments, rather than the value of the assessments, dimensions, and criteria themselves?

I don't want that to be the whole point. Does this argue that I should stop doing my own analyses and just move right to having people do them for themselves? Does it kind of negate some or much of the value of doing (and finishing!) the video analyses? Or can I legitimately say that I did the foundational work of discovering the dimensions and applying/iterating them, and the next step/future work is having people self-assess using them?

To move beyond this conundrum, I need to have practitioners apply my constructs to their own work for themselves, as the IFVP session was a first step towards. I may try to do at least one more such session before the final submission of my thesis. I do think that there is value for practitioners themselves to engage in such assessment exercises, if well facilitated. As Small puts it, "In involving learners systematically in reflection upon what they have achieved, we can ask them to question their own purpose, consider their stance, trace the movement of their selective attention, address their limitations and review their strategies." (p. 268) This kind of work will surely lead to improved practice, especially in the area of better understanding how one's actions as a practitioner affect the people one works with and for.

Last of a series. Tim Small's original article was published in the Curriculum Journal, Volume 20, Issue 3 September 2009 , pages 253 - 270.

Part 1
Part 2

Notes on Tim Small (part 2)

This is part 2 of a series. Back to part 1

In this post I focus on Tim Small's analytical methods and compare them to the approaches I've developed.

In his article, Small describes a method for analyzing students' responses to poetry that he developed for his doctoral thesis. He created a model for "analysis of written responses to poetry on the continuum between text and self" (p. 256), shown in Figure 1 below. It shows the dimensions of possible responses, ranging from the "efferent", outcome-based responses such as "spotting technical devices" or commenting on a poem's form, to aesthetic and experiential responses such as visual associations or even a student's making observations on the way they responded to the poem, such as "I was amazed how much my appreciation of the poem grew, the more I explored it." (p. 259).

Figure 1: Small's model for analyzing responses to poetry (Small p. 256)

He then applies this model to instances of actual responses to poetry by graphically mapping specific responses (taken from students' own writing about the poems) onto the model's dimensions, as shown in Figure 2:


Figure 2: An example of one of the analyses (Small p. 258)

This approach has a family resemblance to some of the analysis methods I've developed, particularly the "framing analysis" and the "CEU" tool. CEU diagrammatically represents the flow of practitioner/participant/representation coherence, engagement, and usefulness in a particpatory representation session, as seen in Figures 3 and 4:

Figure 3: Portion of a CEU analysis

Figure 4: Summary ("heat map") form of CEU analyses

The 3 dimensions of the CEU analysis are more limited than the 6 principal segments (with 24 sub-segments) of Small's model, which is more similar in form and intent to what I've called my "framing analysis" (Figure 5). I use this to characterize practitioner actions during a session in aesthetic, ethical, and experiential terms. It looks at how the practice and context interweave, and in what ways the aesthetic and ethical dimensions of the practice intertwine. I use this as the basis for a normative or ideal model to hold situations of practice up against. The model used in framing analysis provides a set of components, elements, and exploratory questions to help determine how a context of service, the unique set of people, goals, constraints, situation, and subject matter, can inform the "shaping" the practitioner performs on the representational object(s), and vice versa.

Figure 5: Components B and C of framing analysis model

I've applied the framing analysis in a textual rather than diagrammatic way (see an example in Figure 6), though it might be interesting to try that. I use it as a set of prompts or lenses on the way a practitioner acts in a session, answering the questions as a way to make sure I'm looking hard enough at what happened in a particular instance of practice.

Figure 6: Example of component A.2 from a framing analysis


Small
justifies the analytical approach he took in his doctoral research by saying that the the findings themselves are tentative, but that it was concerned "as much with testing the model on a range of 'real' responses as with drawing firm conclusions from what it revealed." (p. 255) That resonates for me with how I am feeling about all the analysis of videos I've done. What is valuable about all that is not the hard and fast ideas of what constitutes 'good' practice or "lessons for practitioners", although there is certainly some of that there. Rather, doing all the cycles of analysis, reflection, and framework-building have been a testing and strengthening of the framework, finding in the data the constructs I've developed (or seeing ways to refine them).

What I've come up with is a way of looking at practice (as Small looked at poetry students) that highlight the aesthetic and ethical dimensions of the practitioner experience and a set of tools that provide representations of those dimensions, grounding them in repeated empirical analysis of actual practice. The question became "do I find what I think should be there" (and the answer I believe is "yes"). Now I'm beginning to see if practitioners themselves could do this kind of analysis of their own practice.

Some of these ideas make me think that I don't so much need to focus on "sensemaking moments" per se in my approach, but rather moments when the type and quality of attention and movement shift in some way from what came before (often, but not exclusively, in response to a sensemaking trigger).

Small provides a nice way to describe the potential value of analytical tools/visualizations of practice: "metaphorically creating a third dimension and offering a perspective from which to view the continuum" (p. 260), which offers "the means by with the reader gains access to a perspective from which to view the very response process he or she is engaged upon". This can result in "the mental space for viewing, understanding and communicating about the movement of his or her selective attention". Analytical artifacts, as "viewing instruments", create a "common vantage point ... for evidence to be included and shared in a peer or joint assessment dialogue." If and when I next try some of my tools out with practitioners evaluating their own practice (as the IFVP session was a first step towards), these considerations could provide a way to assess the success of the effort.

In the next and final part of this series, I'll look at the ways Small applies his work to self- and peer- learner assessments, and how that might be useful for my research.

Sunday, December 06, 2009

Notes on Tim Small (part 1)

This is part 1 of a series.

I've recently spent some quality time with a paper Simon sent me: Tim Small's "Assessing enquiry-based learning: developing objective criteria from personal knowledge". Although Small writes primarily about student learners, there are many resonances with my research on participatory representation practitioners. In many places you could substitute "practitioner" for "learner" and the ideas work just as well.

Three dimensions of the paper are of most immediate value to me: Small's characterization of the experiential aspects of learning, particularly commitment and aesthetic engagement; his analytical methods; and the practical applications in the areas of self and peer assessment.

In this post I focus mostly on Small's ideas about engagement and commitment on the part of the learner, and the connections with practitioner experience as I have been looking at it. I'll look at the other dimensions in future posts.

Small provides helpful distinctions between the aesthetic ("lived-through experience" in the actual encounter) and the "efferent" (outcomes, "what you can carry away" p. 254) as two poles of a continuum. This idea of a continuum, rather than an either/or or absolute view, is just as applicable to participatory representational practice. The artifacts, methods, tools, and outcomes involved in such practices, important as they are, fall on the efferent end of the spectrum, but do not sum up all that is important about the practices; much of value lies at the other pole or in between.

Small quotes Rosenblatt: "Any literary transaction will fall somewhere in the continuum between the aesthetic and the efferent poles . . . there is a to and fro movement of attention between the words and the experienced, felt meaning being elicited, organised and reorganised." (p. 255) This concept of a transaction is similar to what I refer to as a practitioner move or choice. It maps quite well onto (for example) the coherence, engagement, and usefulness dimensions in my CEU tool, with "usefulness" at the efferent end of the scale, "engagement" at the aesthetic end, and "coherence" in between.

The way Small invokes Polanyi's idea of commitment in the context of developing criteria for learning assessment (p. 261) reminds me of Dewey's concept of "impulsion" in the artist's experience. "The task is to develop criteria that, first, recognise and uphold the drive and responsibility of the learner" (p. 261). Commitment involves a search for truth (discovery) and requires "strength of engagement" as well as "sensitivity" to self and text (read "representation"); a "readiness to interrogate", to "enter into a sort of dialogue with the text, object, or place." (pp 264-265). This is driven from within, finding expression in a kind of grappling with the stream of experience in order to make meaning: "A process of comprehending: a grasping of disjointed parts into a comprehensive whole." (p. 261, quoting Polanyi ("The Study of Man, 1959, p.28))

Real engagement of this sort, a willingness to be affected in some different or new way, "depends on the (at least partial) adoption of an aesthetic or personal stance... rather than simply the efferent stance of the analyst, who reads/learns according to a ''pre-ordained system of selection' (Rosenblatt 1978 p 89)". (p. 265)

The same kind of commitment, to make a session "work", to bring about positive outcomes, is in the practitioner and is something you can see or not see. To be a participatory representation practitioner in the kinds of settings I've looked at is to make a commitment, to live it for the duration of the role and the session. The overriding concern is how to make the session work. Not all the practitioners I've looked at were successful, but all did try to make generative and/or repairing moves when and how necessary. There could be some sort of measure or dimension of how far or how radical such moves had to be, just as another way to compare. The kinds of analysis I've been doing could be a measure of what practitioner commitment looked like and meant for that time (the session), what choices and trade-offs it occasioned, providing a context and a stable framework to describe them against.

The degree of engagement itself lies on a scale. While there is always a "transaction" (p. 255) between any aesthetic artifact and its audience, too often, even in participatory representation sessions, it's invisible, passive, or inconsequential. Like most movie watching, or walking by a painting hanging in a museum, most aesthetic encounters are superficial. Sometimes they aren't, and while that is sometimes a result of happenstance -- just being struck by something -- more often, it the encounter goes beyond the superficial it's because of some commitment on the part of the person -- some decision or desire (or classroom assignment) to make it mean more: "the commitment of the learner (to the truth he or she is seeking to discover)" (p. 263).

It reminds me of my college seminar on the films of Ingmar Bergman. I hadn't much liked the few Bergman movies I'd seen before the seminar, but I liked the professor and wanted to take the class. We had to see a film (this is in the stone age where the only way to see films was in a cinema, once, taking notes frantically in the dark, no pause or rewind) and write a 2-page paper due the next morning on some aspect. I wrote my first paper on "Wild Strawberries", basically scoffing at it. The professor said "no way" -- you can't take that superficial an approach -- you have to engage. That was sometimes painful (hard to *enjoy* a film like "Cries and Whispers") but it made me really pay attention and try to get at what was going on, and ended up transforming the way I looked at film and the whole question of the relationship of aesthetics and ethics.

In the case of participatory representational practice, for the practitioner (if not necessarily for the participants), that kind of commitment to engage has to be made; it's part and parcel, at least for the duration of the session. Part of that commitment is to get at least a partial engagement from the participants. What can practitioners do to foster, inculcate, and sustain this? To repair it in the face of disruptions? To improve the quality of participant engagement?

Small establishes the movement between inner/outer, subjective/objective, learner/teacher and talks about the "joint responsibility" (p. 268) they (should) have in assessment. That can be related to the putative "joint responsibility" of participants and practitioner in participatory representation construction. In the ideal mode, there is a joint responsibility and a movement between, a co-construction; it can even be said that it is the task of the practitioner to enable and foster that co-construction more than it is to create the artifact, or that the expertise of the practitioner should (only) lie in how good they are at creating the artifacts. Instead, the focus should be on creating the climate in which co-construction occurs, distinct from the 'same old thing' of normal meeting talk on the part of participants (however productive it might be; if the representation itself doesn't take on that which it needs to take on, it will not realize/contribute its full value), as well as all the work done by the practitioner him or herself on the representation (however well realized it might be).

Reframing practitioner and participant engagement in this way, in the context of participatory representation construction, leads to questions that can better characterize the specific role a representation can play in a session, and the relationship of practitioners and participants to it than taking a purely efferent approach. These include: in what ways does the artifact/representation speak for itself? What role does the artifact actually play, both in the here and now of the session, and later? What is the value of the artifact, proportional to coherence, engagement, usefulness etc.? (artifacts that aren't engaged in are akin to the well-written report that gets put on the shelf, or the graphic recording mural that no one looks at afterwards). How does one create situationally appropriate interventions, when the representation itself matters? What things do people do to make the session work? Why did they do them? What impact/effect did they have?

In the paper, Small makes a connection between aesthetics and ethics, at least where they meet in communication, and what I would see as the responsibility of practitioners to examine their relationships to representations and participants: "We might accept that a clearer understanding of the creative process would help us to get better at it, particularly if we were involved in communicating about our art, in teaching or being taught, for example. We might agree that it is the responsibility of all art with a communicative purpose to be self-referencing to a degree – even if only in stating its relation to its frame or context – and that that is what makes a work an objective entity, gives it its ‘comprehensive unity’." (p. 262)

Practitioner commitment requires awareness of, and sensitivity to aesthetic engagement, which (again resonating with Dewey) stresses being open to the uniqueness of each encounter (what Dewey calls "an experience"). Small writes: "Aesthetic engagement is denoted by a sensitivity both to text/object and self, combining two interactive functions: receptivity to the pattern of verbal and material symbols observable in and around the object/place, and subjectivity, the capacity for ‘infusing intellectual and emotional meanings’ into those symbols, (Rosenblatt 1968, 25) according to the ‘never-to-be-duplicated combination of attributes and circumstances’ brought by the reader/learner to each learning experience." (p. 265)

Nicely complementing the discussion of engagement (for my CEU purposes) is Small's discussion of "coherence and clarity": "the appropriateness of language, form, and structure to their purpose" as well as "completeness" and "persuasiveness", (p. 266)

More on the other dimensions of the article in these posts:
Part 2
Part 3