The experience of studying representational artifacts (like a film)

When I was a freshman at university taking my first Introduction to Film class, the professor said "up until now you've just let movies wash over you. After this class you'll never experience a film that way again."

He was partially right. In that class, we drilled deeply into editing, color, lens length, mise-en-scene, and the hundred other techniques that make up a film, looking at how (for example) the use of sound techniques in one stairway scene in Citizen Kane contained clues that encapsulated the whole complexity of the film's characters and meaning.

Even today I can still pick out such details -- when I remember to focus on them and make a special effort. Otherwise, movies just wash over me like they did before being a film student.

As a film student in those days (late 70s/early 80s), you watched movies in a cinema or on a projector in a classroom. If you were lucky, you saw a film you had to write a paper on twice. Usually it was once, with no ability to rewind, pause, or anything like that. So studying a film as it unfolded was usually a matter of scribbling frantically in a notebook in the dark, and hoping you could make sense of your notes later to reconstruct (for example) the sequence of edits in one scene of a Bergman film.

Capturing "practice" in this way was a challenge, especially making sure you got enough appreciation of both the nuances of technique as well as the sense of the film as a whole, so you could relate the two.

I am hoping that the techniques I've been developing in my research will eventually help participatory representational practitioners to 'read' and reflect on their own practice, the way a film student can (with effort) read a film, and to be able to prise apart the individual techniques, moves, themes etc. and make sense of them in the context of the larger meaning of the situation they're engaged in (the context of their practice).

And another terrific one covering the same terrain

Coda—Creativity and Improvisation in Jazz and Organizations: Implications for Organizational Learning
Frank J. Barrett
ORGANIZATION SCIENCE
Vol. 9, No. 5, September-October 1998, pp. 605-622

This widely cited article (276 according to Google Scholar) is full of evocative quotes from and stories about jazz musicians (Coltrane, Miles, Sonny Rollins, many others), with parallel organizational learning examples. It's almost too rich in ingredients that match my main interests: improvisation, aesthetics, sensemaking, narrative (many discussions of how jazz musicians both link to the "stories" of the jazz canon and create and rewrite new ones on the fly), ethics (in the ways the musicians relate to one another and make choices that affect each other's performances), and many examples where instant, unplanned move-by-move choices and actions make a huge difference.

A couple of examples from the paper, the full text of which appears to be online as a pdf.

Provocative disruptions as a leadership technique (connections to sensemaking in the unexpected challenge given to the performers), practitioner ethics (the choice Davis made to present the material to the musicians this way, violating their expectations with an expectation they would rise to the occasion), narrative (breaches of canonicity), aesthetics, as well as improvisation:

Miles Davis not only practiced this provocative competence in live concerts, he also extended this to the recording studio. This is illustrated in a famous 1959 session. When the musicians arrived in the recording studio, they were presented with sketches of songs that were written in unconventional modal forms using scales that were very foreign to western jazz musicians at that time. One song, contained 10 bars instead of the more familiar 8 or 12 bar forms that characterize most standards. Never having seen this music before and largely unfamiliar with the forms, there was no rehearsal. The very first time they performed this music, the tape recorder was running. The result was the album Kind of Blue, widely regarded as a landmark jazz recording. When we listen to this album, we are witnessing the musicians approaching these pieces for the first time, themselves discovering new music at the same time that they were inventing it. (p. 609)

Move-by-move, sensemaking, aesthetics, improvisation, and narrative all in one:

Jazz players are often able to turn these unexpected problems into musical opportunities. Errors become accommodated as part of the musical landscape, seeds for activating and arousing the imagination. Drummer Max Roach sees the value in errors, "if two players make a mistake and end up in the wrong place at the wrong time, they may be able to break out of it and get into something else they might not have discovered otherwise." Herbie Hancock recalls playing an obviously wrong chord during a concert performance. Hearing the unexpected combination of notes, Miles Davis used them as a prompt, and rather than ignore the mistakes, played with the notes, embellishing them, using them as a creative departure for a different melody. Any event or sound, including an error, becomes a possible springboard to prime the musical imagination, an opportunity to re-define the context so that what might have appeared an error becomes integrated into a new pattern of activity. Looking backward, the "wrong" notes appear intentional.(p. 610)

There's a lot more, way too much to include here. It's an embarassment of riches. Check it out.

The best paper relating jazz improvisation to organization theory I've read

Exploring the Empty Spaces of Organizing: How Improvisational Jazz Helps Redescribe Organizational Structure
Mary Jo Hatch
Organization Studies, January 1999 vol. 20 no. 1 pp. 75-100

The paper uses improvisation as a "redescription" metaphor (Rorty) of organizational structure. It has strong ties to writing on experience, sensemaking and aesthetics. It's full of theory relating the art and performance of jazz improvisation to thinking about organizational structure, but even more has understanding for the jazz nuances, evidenced by her writing about groove, feel, etc. One of many examples: "Groove and feel in jazz terms involve making structural aspects of performance (e.g. tempo and rhythm) implicit, which jazz musicians accomplish by rendering them subjects of their emotions and physical bodies (i.e., by literally feeling tempo and rhythm in an emotional and physical sense)." (p. 89)

The paper way extends what I was trying to talk about in this post. The writing is exceptionally clear and evocative for an academic paper, without losing authority (funny how that can be).

A few excerpts below.

A very nice description of tacit communication and intuitive "moves" in improvisation, especially soloing:

Soloists encourage the exchange of ideas by leaving space in their playing for other musicians to make suggestions, for instance they may leave gaps between their melodic phrases, or play their chords ambiguously by leaving out certain notes that would distinguish one chord from one or two others. Of course, they do not explicitly think, 'Okay, now I will leave a space for someone else to fill.' Space-making and filling are more spontaneous than this. Jazz musicians listen to the playing of the other musicians and, in listening, spaces are created and filled by a logic that emerges as part of the interaction of the musicians. This simultaneous listening and playing produces the characteristic give and take of live jazz improvisation and also provides the conditions for conflict that can introduce the unexpected that inspires performance excellence, but also risks disaster. (p. 79)

Ethical choices in listening in improvisation:

Ideally, each musician listens to all the other players all the time they are performing a tune. Nevertheless, many musicians freely admit that they reach this ideal only once in a while, primarily when they achieve peak moments of jazz performance. At other times, the musicians will concentrate on listening to one or two of the other players intensely, often shifting their focus from one player to another as the tune develops. (p. 80)

The experiential component of communication:

The jazz metaphor suggests that whenever we interact, communication rests as heavily upon emotional and physical feeling as it does on the intellectual content of the messages involved. (p. 89)

The way we can (sometimes) spontaneously and instantly connect and "groove" with co-workers on projects, even if new to each other, if the situation and communication are right:

Just as jazz musicians assign tempo and rhythm to the emotional realm and communicate on this basis to one another as they improvise (even when they have never played together before). workers may equally depend upon their ability to emotionally communicate as they coordinate their efforts for organizational achievement in the context of temporary teams or fluid networks. ... communication does not necessarily depend upon self-disclosure, but rather is an intimacy based in shared action. That is, we are as capable of using our emotions to form working relationships as we are of using them to form friendship or familial relationships, and this capacity can extend to those with whom we have no relationship at all apart from the opportunity to act together at a particular moment in time.... Rhythm, harmony, groove and feel have emotional and aesthetic dimensions, and when these aspects of work processes are engaged we may likewise find the experience of flow that Csikszentmihalyi claims constitutes peak performance. (pp. 89-90)

I see Hatch has other intriguing articles -- e.g. Hatch, M.J. and Jones, M.O. (1997) Photocopylore at work: Aesthetics, collective creativity and the social construction of organizations. Studies in Cultures, Organizations and Society , 3(2):263-287. [ISSN 1024-5286]; Hatch, M.J. (1996) The role of the researcher: An analysis of narrative position in organization theory. Journal of Management Inquiry , 5(4):359-374. [ISSN 1056-4926] -- but these will have to come AT (after thesis).

There is other excellent writing relating jazz improvisation to organizational and similar issues -- Sawyer and Schön come to mind -- but this paper is my current fave.

And one of the most useful on mediation

Complementing this post, here is another excellent paper that looks at dispute mediation from a reflective practice point of view.

It's titled "Mediating Ethically: The Limits of Codes of Conduct and the Potential of a Reflective Practice Model" by Julie Macfarlane (Osgoode Hall Law Journal 49, 2002), and lives up to its title.

The article criticizes reliance on codes of conduct to guide professional practices such as mediation where ethical considerations are paramount. Codes of conduct are too abstract, and it's not possible to actually separate them outside of the "context and circumstance" (p. 60) of professional action:

Codes reduce ethical choices to a set of generic principles, fastening on relatively uncontentious virtues for the mediation process, which appear in a virtually identical form across numerous codes of conduct. Ethical issues are identified as discrete topics such as mediator impartiality, conflicts of interest, and self-determination; ethical dilemmas are those that threaten the integrity of these principles. Codes of conduct for mediators also assume that it is possible to describe and regulate the process of dialogue and the content of dialogue quite separately. Principles for process management dominate codes of conduct for mediators and often precede the commencement of dialogue (p. 60)

Even apparently "functional" choices can have ethical consequences. The article provides good examples of ethical choices on the move-by-move level:

Even the most mundane and mechanical decisions have a habit of turning into issues of principle in the volatile climate of conflict. For example, the plaintiff’s refusal to meet during the daytime is characterized by the defendants as “typical of their uncooperative stance.” Thus the question of scheduling, and how the mediator deals with it, is transformed into an ethical dilemma. The mediator must decide whose preference shall prevail and what values are implicated. Deciding whether or not and how to quiet Party A thirty minutes into his or her monologue raises fundamental questions about the role of parties and mediators in a facilitated dialogue. (p. 57-58)

Practitioner choice-making is constant:

The reality of mediation is that ethical judgment making occurs constantly, intuitively, and often unconsciously. (p. 59)

The article argues against thinking only in terms of outcomes, and promotes the need for understanding such micro-choices on the move-by-move level:

Mediation is as much a process, replete with ongoing negotiated understandings, as it is a result. The “snapshots” represented by beginnings and endings which dominate standard-setting in codes of conduct for mediators are but a fragment of this process. Instead, the expanded definition of what amounts to ethical choices suggested in this article shifts the focus from an evaluation of end result, or proper procedure in mediation set-up, to the choices made in the course of micro-managing the dialogue between the parties. (p. 67)

The article provides two lengthy case studies from the author's own experience as a mediator, analyzing them from a reflective practice viewpoint. Macfarlane stresses the importance of taking such an approach for 'young' fields like mediation:

The reflective-practice model seems especially appropriate to a field such as mediation that is at an early stage of professional and self-conscious development, and to a form of intervention that is so diversified, unregulated, and context-dependent. As an examination of “practitioner cogitation,” it focuses on teasing out the values and assumptions behind the choices often made intuitively by mediation practitioners when they face ethical dilemmas in the course of their practice and the values they imply. These values can then be debated, critiqued, and diversified across different frames of action. (p. 73)

and lays out what this will require from practitioners:

putting the principles of reflective practice into practice requires the conscious nurturing of a collaborative professional environment in which personal experiences and choices are shared in a continuous, self-critical, non-defensive, and open dialogue. It needs practitioners—new and old, experienced and less experienced—to talk and write analytically and self-critically about their approaches to ethical dilemmas. (pp. 85-86)

Following such an approach for practitioner education and development could result, among other things, in making practitioner choice-making, and its ethical and sensemaking characteristics, "first class objects" in both the research and professional practice communities.

One of the most useful papers on facilitation I've come across

For my research, anyway, in terms of research methods chosen, subject matter, and clarity:

Jean-Anne Stewart (2006). High-Performing (and Threshold) Competencies for Group Facilitators. In Journal of Change Management, Dec 2006, Vol. 6 Issue 4, p417-439.

Based on her doctoral thesis, it describes a qualitative study of UK facilitators with the aim of identifying the key competencies for effective workshop facilitators. It lays out a convincing justification for qualitative research in this area and provides clear diagrams of facilitation roles, processes, and competencies. The "motives and traits" identified are similar in kind and aim to my "framing model" attributes.

Her competency model is strong on communicative and ethical characteristics, though mentions almost nothing on aesthetic capabilities except references to skills with audio-visual aids. The simultaneity of being able to apply the various types of competencies was seen as characteristic of more expert facilitators:

...high-performing facilitators were frequently described as ‘being able to do it all at once’. The research review group referred to the acronym of LEAPS (listen, empathize, ask, paraphrase, summarize) as an example of how the facilitator uses a group of competencies almost simultaneously.
(p. 431-432)

Finally, the list of potential benefits of such research will be a useful reference for when I come to my conclusions/future work chapter (soon enough):

Facilitator training could be designed to ensure effective competency development

It would provide a framework for skills development for facilitators

Qualifications could be designed based on the competency model

Clients would be able to select facilitators with the appropriate competencies to meet their workshop requirements

The competencies would provide a common language for facilitators and clients

Workshops would use the most effective facilitator to ensure the desired outcome, thus avoiding wasting people’s time in unproductive sessions
(p. 438)

One-paragraph description of my dissertation research

How to make participatory visual representations (pictures, diagrams, knowledge maps) coherent, engaging and useful. I study the ways that fourteen practitioners using a visual hypermedia tool engage participants with the representations on the screen and the ways they make the representations matter. I'm especially interested in the sensemaking challenges that the practitioners encounter in the heat of their sessions, and in the ways that the form they give the representations (aesthetics) relates to the service they are trying to provide to their participants (practice ethics).The thesis places this in context of other kinds of facilitative and mediation practices, especially those involving some sort of crafted representation or artwork, as well as research on reflective practice, aesthetic experience, and participatory design.

Logically, aesthetics and ethics are identical (Leach 1954)

I saw this quote:

Leach (1954) claimed, "to understand the ethical rules of a society, it is aesthetics that we must study."

in Deb Orr's dissertation*. I Googled it to its original appearance in a 1954 essay titled "Ritual as an Expression of Social Status." Here's the surrounding text, all of which is worthwhile (boldface added):

From the observer's point of view, actions appear as means to ends, and it quite feasible to follow Malinowski's advice and classify social actions in terms of their ends -- i.e. the 'basic needs' which they appear to satisfy. But the facts which are thereby revealed are technical facts; the analysis provides no criterion for distinguishing the peculiarities of any one culture or any one society. In fact, of course, very few social actions have this elementary functionally defined form. For example, if it is desired to grow rice, it is certainly essential and functionally necessary to clear a piece of ground and sow seed in it. And it will no doubt improve the prospects of a good yield if the plot is fenced and the growing crop weeded from time to time. Kachins do all these things and, in so far as they do this, they are performing simple technical acts of a functional kind. These actions serve to satisfy 'basic needs'. But there is much more to it than that. In Kachin 'customary procedure', the routines of clearing the ground, planting the seed, fencing the plot, and weeding the growing crop are all patterned according to formal conventions and interspersed with all kinds of technicall superfluous frills and decorations. It is these frills and decorations which make the performance a Kachin performance and not just a simple functional act. And so it is with every kind of technical action; there is always the element which is functionally essential, and another element which is simply the local custom, an aesthetic frill. Such aesthetic frills were referred to by Malinowski as 'neutral custom', and in his scheme of functional analysis they are treated as minor irrelevancies. It seems to me, however, that it is precisely these customary frills which provide the social anthropologist with his primary data. Logically, aesthetics and ethics are identical. If we are to understand the ethical rules of a society, it is aesthetics that we must study. In origin the details of custom may be a historical accident; but for the living individuals in a society such details can never be irrelevant, they are part of the total system of interpersonal communication within the group. They are symbolic actions, representations. It is the anthropologist's task to try to discover and to translate into his own technical jargon what it is that is symbolised or represented.

Ritual as an Expression of Social Status (1954)
pp. 153-154 in The Essential Edmund Leach
Volume 1: Anthropology and Society
Edited by Stephen Hugh-Jones and James Laidlaw
2000. Guildford: Biddles Ltd.
http://books.google.com/books?id=KC4FiQsBszkC

If I can do the right things in my thesis, it will show the indissolubility of aesthetics and ethics in participatory representational practice, at least in one small corner of it. Nice to have some backup from 1954.


----------------
* Orr, D. E. (2003). Aesthetic practice: The power of artistic expression to transform organizations. Unpublished PhD dissertation, Benedictine University

Making public policy visually clear

A great example of demystifying and de-gobbledygooking a public policy document, on Jeannel King's site. It would be very interesting to hook it up with a web-based visual discussion system like DebateGraph or Cohere, and even more (from my perspective) to map a live discussion based on it with Compendium.

Engagement in face-to-face meetings

A helpful post from Susan Nurre on the IAF forum points to an article on meetingsnet.com that in turn points to a Cornell University School of Hotel Administration white paper (free registration required) on "The Future of Meetings: The Case for Face-to-Face," by Christine Duffy and Mary Beth McEuen of Maritz*. I'd never thought about looking at this area of literature (hospitality studies), but it might actually be fruitful for my research.

The article focuses mostly on "large-group meetings and events", which are not my area of focus. But the concept of "engagement" is threaded through the article in ways that are applicable to the (primarily small group) kinds of phenomena I'm looking at, tying them to specific organizational benefits:

Amidst the phenomenon of ever-increasing information overload and ever-decreasing speed cycles, attention is scarce and fleeting. In this environment, the lack of a focused and attentive human mind is one of the greatest limiting factors in effectively executing business strategy.

and (citing John Medina):

... when people are engaged with information in multi-sensory environments, they are more likely to remember the information (compared to single-sense experiences) and creatively come up with solutions.

Some interesting (and true in my personal experience) observations on the disadvantages of virtual meetings for particular purposes:

Participants stated that the significant temptation to multi-task during virtual meetings was distracting and had an impact on the meeting objectives being achieved. While this may not be a significant problem when the goal is to share information to which people can refer later, it is a real problem when the goal is to initiate something new, whether that is new learning or a new set of priorities that require a shift in attitude and action.

Since my day job is working primarily on the design of new user interfaces, I have seen over and over that getting designs off on the right foot really does require face-to-face interaction. It somehow does not gel right without that.



-----
* Way back in the beginning of my career, when I lived in St. Louis, I almost did a video and writing project with Maritz. It would've been my first freelance project and was quite exciting. It didn't pan out (for reasons I don't recall).

Paying attention to a representation -- or not

An interesting photo and comment from Eugene Eric Kim about participant attention (or the lack of it) in a graphic facilitation session. Of course the photo depicts just one moment out of a session that might have had periods of direct participant engagement, but it's a good illustration of the main questions I am trying to get at with the research:

• What (and when, and how, and why) does a practitioner do to promote engagement with the representation?
• How do they make it of value to the participants?
• What makes the difference between something that (however virtuosic) is just background to the main discourse, and something that is (or becomes) integral?

Great posts on improvisation & visual practice, and dialogue mapping from child's point of view

Two posts that both, in different ways, take a child's viewpoint that illuminates a) 'rote' practice, skill development, improvisation, and graphic facilitation, and b) dialogue mapping.

a) from Jeannel King in her blog Process Arts and Facilitation: The Practice Will Set You Free

b) from Kailash Awati in his blog Eight to Late: What should I do now? A bedtime story about dialogue mapping

Making Representations Matter -- a mini-workshop at the Knowledge Media Design Institute (5 Aug 2010)

I recently gave a talk and mini-workshop at the Knowledge Media Design Institute, University of Toronto, titled "Making Representations Matter: The Practice of Shaping Participatory Media Artifacts." The session started with a review of the research to date (see slides below), but the bulk of the evening was spent with the attendees doing a collaborative exercise followed by a discussion using some of the presented concepts.

For the exercise, the participants broke up into two groups and were assigned to pick facilitators and engage in a collaborative representational task of their choosing, using markers and whiteboards, for about 30 minutes. I roamed back and forth and took notes on facilitator moves and participant responses. We followed this by over an hour of reflection and discussion. The audience, composed of researchers, grad students, and practitioners (including a few people from the community with no direct tie to KMDI), seemed to really engage with the ideas.

Participants working on their collaborative representations

One helpful idea that came out of the discussion was to make an explicit distinction between the "interactional" aspects of facilitation -- e.g. those having to do with facilitator actions that create better communication and discussion among participants -- and the "representational" aspects, particularly those that have to do with ensuring direct participant engagement with the representation and how facilitators can help ensure that the representation itself ends up with suitably "persistent" qualities -- that is, that it can serve for future audiences as an evocative representation. If this distinction isn't made clear, the discussion tends to veer towards the interactional aspects rather than the representational.

For future work, it's encouraging that giving people an opportunity to experience and reflect on shaping participatory representations, in combination with presenting some of the concepts and findings from the research, seems to work well. This was my second direct attempt to do this, following last year's initial foray at the IFVP conference. I had learned from that to make sure there was enough time for both the experiential and reflective portions of the event (and not too much time given to the initial presentation), and this time that worked well. In fact, the discussion went way over the allotted time and on into the following dinner at a nearby Thai restaurant. It's good to see that the constructs from the research can apply to non-software-based representational forms, like whiteboards and markers.

Graphic facilitation from the practitioner's perspective

Check out this post, and its 31 responses (as of today), on Julie Stuart's blog. It's a response to a mostly favorable Harvard Business Review post on graphic facilitation. Several things are interesting to me from a research as well as practitioner perspective.

From the research viewpoint, what's of interest is the way Julie and her community talk about their practice and the ways the aesthetics of their practice intertwine with the way they see the service they provide to their clients and participants -- what I would call their practitioner ethics.

For example, here is Julie talking about the complex of activities -- the stance -- of a graphic facilitator in the heat of the moment of practice:

When I graphically facilitate, I’m listening as a journalist would for the key themes and highlights in the story, organizing the information spatially, instinctually finding the structure, giving visual emphasis and hierarchy to the story as it emerges on the paper. And also paying attention to where the group needs to go next.

And the way her personal experience is uniquely and inextricably tied to the many dimensions that make up her practice (how she is able to listen, shape, and facilitate simultaneously), which is evocative of the way McCarthy and Wright talk about how taking an experiential view moves thinking about practice away from abstracted types and generalizations:

There is no one else on the planet with my particular combination of skills: public policy background + non-profit management + journalism + conceptual art + stock trading + politics + teaching + facilitation.

Julie, and many of her respondents, take issue with the phrase "pricy artist's handiwork" that appeared in the HBR article, and more generally with being referred to as "artists" in the context of graphic facilitation practice:

I don’t think of myself as an artist when I do graphic facilitation work. Yes, there are drawings that depict recognizable icons but art is about a tenth of what’s involved with this work. . . . Art—what we tend to think of as fine art—has original content. Creating visual maps from content that emerges from a group’s collective process and not from me, doesn’t qualify as art, in the original sense.

In this we see the consequence of the reductionist meanings that the terms "art" and "artist" have taken on, that art is something that only fine artists do, and that real art is something you would generally only find in a museum, rather than seeing artistry as something inherent in all levels of human culture. Dewey (in one of my many favorite quotes) said this in Art as Experience:

The intelligent mechanic engaged in his job, interested in doing well and finding satisfaction in his handiwork, caring for his materials and tools with genuine affection, is artistically engaged. (p. 4)

While arguing that the term "artist" is used pejoratively in the HBR article, Julie's post also takes a similar position to Dewey, hoping that when seen without the reductionist implications, graphic facilitators

.... aren’t seen as “the artist,” the person in the room owning the creative process for the group. It is my hope that we will be the enablers of everyone else’s creativity. That we will teach people to make their own pictures so that we aren’t seen as “the artist” but the person able to bring the artistry out of the people we work with who are hungry to express themselves creatively. I believe there is artistry in everyone. As Ellen Dissanayake writes in Homo Aestheticus, this used to be an accepted fact. Our culture has taken the ownership of creativity out of the hands of many and put it into the hands of a few. I would like to return it to the many.

There is more to say about the piece, and the 31 respondents mention many different facets. The number and energy of the responses, most of which are from fellow graphic facilitation practitioners (and participants) brings me to the other point of interest. As a practitioner of another form of visual facilitation, it's amazing to see the level of community and mutual reinforcement I've seen in the graphic facilitation world.

Compendium has had over 80,000 downloads, and there are currently more than 1600 members of the yahoogroup, but we've never seen anything like that kind of energized practitioner community. I think this is largely because most Compendium users are not using it for the kind of facilitative practice with the tool and approach that we originally developed it for. Judging by the entries on our download log, most people are using it for individual work, mostly as a mind-mapping tool. There are certainly facilitative practitioners using Compendium out there, but the practitioner community hasn't gelled in anything like the same way demonstrated in Julie's post and responses. It's something to strive towards. If/when I finish the phd, maybe I can put more energy in that direction; it's been a long while now since I did.

Excellent sheep

“So are you saying that we’re all just, like, really excellent sheep?”
- a student quoted by William Deresiewicz in his The Disadvantages of an Elite Education

It's funny that I just read Deresiewicz's article (following a link from eekim) after reading the novel The Good Son by Michael Gruber, and seeing the movie Inception last week, all while visiting a bunch of colleges and thinking and talking about college with family, friends, and old professors.

The common thread in the article, book, and film is the idea of false consciousness -- we think we're living out an authentic and coherent experience, with autonomy and clear knowledge of what we're up to, but in fact (at least in part) we're inside a hall of mirrors and enforced system of ideas, that has a huge mechanism in place for keeping you thinking you know what you're doing, when you are playing a part written by others, keeping you from seeing the limitations and blinders.

Now that I think of it, this was also the theme of another recent read -- The Reluctant Fundamentalist by Mohsin Hamid. (Spoiler alert -- read no farther if you are going to read the book). I am sure I am far from the only reader who believed through much of the book that the title referred to Islamic fundamentalism and that the narrator -- telling his tale to a mysterious American in a Pakistani cafe -- was going to become a terrorist or the like. However, it actually refers to the narrator's realizations about his job as a management consultant in an elite New York valuation firm, which praised its ability to focus on the "fundamentals" in the businesses it examined (or helped destroy) for its clients. This book, too, talked about the blinders and self-enforced sense of privilege instilled by an elite education (in this case from Princeton, which is one of the colleges we just visited).

In The Good Son, a number of the characters, particularly an NSA language analyst, realize that their successes and ambitions were, similarly, self-reinforcing products of the system that they lived within without realizing (or, more accurately, questioning) it, which fall apart disastrously when push comes to shove.

We visited the University of Michigan, my alma mater, on this college trip. I found myself getting choked up during the information session, especially when they showed a (kind of corny) video, all on the theme of the wonderful and diverse opportunities available to the students. It occurred to me that this was because my years in Ann Arbor were all about becoming -- moving from the person I was before I got there to the (still limited, but far less so) person I was afterward.

Does/did the U of M foster the same kind of entitled blinderedness that the works above talk about? It's not a purely elite institution by any means, but some of that is undoubtedly there. I woke up (in some ways) while there, learned to see many things about our culture more clearly, but -- as with any education or acculturation -- I also have to wonder about what I'm not seeing.

An honest look at participatory design and user involvement

A refreshingly honest article about the realities of participatory design and involving users in systems design projects. It accords with my experiences as an IT/usability/PD practitioner for the last two decades.

"It is time to speak honestly about the gap between our intentions to build working systems and our ability to do so in practice. This gap is typically not caused by a lack of effort on behalf of developers or users, but rather is the result of misdirected efforts. The systems development and implementation process will continue to be overly challenging if we work against the tide by trying to make users fit our theories of how and when they should participate in development initiatives. Instead, we suggest catching waves with users at opportune moments, working to hear what they are saying, and then adjusting our, and their, expectations about when a system is completed." (p. 55)

As the authors say, it's not a matter of not including and involving users -- that has to be done. My own design practices revolve around it. What it's saying is that doing so is not a panacea and does not in and of itself guarantee you any special insight or improved chance at success. You still have to interpret, be flexible, do different things at different times, improvise and make sense of what you are hearing, how you are listening, what the users themselves can and do pay attention to, etc. Doing so is also an ethical stance for design practitioners -- no matter how good, they aren't going to know and perfectly understand in advance what user input means and what has to be done.

The article is Wagner, E. L. & Piccoli, G. (2007), 'Moving beyond user participation to achieve successful IS design', Communications of the ACM 50(12); pp.51-55.

Canny and uncanny people

Something quite compelling in this statement, when reflecting on the ups and downs of pursuing a large, lonely research project:

"Canny people tend to succeed in their own lifetimes; uncanny people tend to be recognized and appreciated only centuries later, because during their time their actions appeared to be either insane or pointless."

From "The new dynamics of strategy: Sense-making in a complex and complicated world" by C. F. Kurtz and D. J. Snowden, IBM Systems Journal, Vol 42, No 3, 2003

One can reassure oneself, during the downs -- the times when one indeed feels that one's efforts are both insane and pointless -- with the idea that one is, in fact, uncanny. Yeah, that's the ticket.

Human Technology journal article now live

In the Special Issue on Creativity and Rationale in Software Design
John M. Carroll, Guest Editor


Our article is: The Practice Level in Participatory Design Rationale: Studying Practitioner Moves and Choices

Albert M. Selvin, Simon J. Buckingham Shum and Mark Aakhus

Unity of purpose, communication, and representation

Further on the idea of integral representations and the role of practitioners in making representations matter...


As I was working through writing up the "how good/successful was the session" question, it struck me that one way to characterize this was by considering how closely three dimensions were unified in each session -- the purpose, or intended (as well as emergent) reasons or goals for each session; the communication, both verbal and non-verbal, the way the participants (as well as practitioners) interacted, and the representation, the visual artifact (Compendium maps in the cases I'm studying, but could be any kind of visual or written representation, or even just a verbal representation if there is some sense of a central focus for the session):



The best sessions see a fusion of the three dimensions. Purpose, communication, and representation become indistinguishable in practice; why we’re doing what we’re doing, how we interact with each other, and the visual artifact appear to be unified:



By contrast, in less optimal sessions, the dimensions are disconnected from each other. For example, people could be talking about something that is more or less removed from the ostensible purpose of the session, and the representation itself is ignored or irrelevant:

Integral representations

As I've been working through writing up the comparative analyses, one thing that's struck me is that a key differentiator of practice styles and expertise is how much (and in what ways) practitioners make the representations themselves matter to the participants and to the proceedings, as opposed to being (in varying ways) a sideshow, background, or decoration.

For example, one of the Shaping aspects in Category B (practitioner interaction with participants) is the "degree of intervention to get participants to look at the representation". It occurred to me that, while all of the studied practitioners used various physical and verbal means to do this, the style, purpose, and strength with which they did this varied greatly. Both of the 'expert' sessions (Hab Crew and Remote Science Team) did this frequently and in depth throughout the session, but also did it with a degree of naturalness. They did not have to use much special force or emphasis because, due in large part to their expertise, the representation was integral to the proceesings, embedded in how and why the group was working.

This isn't something that comes for free and takes a lot of factors to achieve, but the phrase integral representations (seemingly widely used in mathematics) seems to me to sum up a lot of what this research is aiming toward: how to make such representations integral to their participants and audience, how to make them matter, and in what ways.

And the final 'raw' cross-session analysis artifacts

Cross-session Shaping comparisons (table version) (Compendium version)

Cross-session Framing comparisons (table version) (Compendium version)

CEU comparisons

Some 'raw' comparisons of the "CEU" (Coherence, Engagement, and Usefulness) analyses across sessions are here.

Next and final step for the comparative analysis -- create a summary of the cross-session Framing analysis.

Comparative framing analysis

Compendium version here.

One more piece of cross-session analysis to go -- CEU comparisons. I hope to finish those tomorrow, then it's on to start writing up the first draft of the analysis chapter.

Comparative analysis of sensemaking moments

A fuller writeup later, but in case you would like to look:

In Compendium form

In summary tables

Preliminary results from cross-session 'shaping' analysis

I've finished the first round of comparative analysis (earlier steps described here). This morning I translated the mapping of each session along the 29 shaping dimensions into Excel. It's not a perfect or exhaustive study (that would take another set of years) but did yield some interesting comparisons.

In the mapping linked above, I'd ranked or grouped each of the 8 sessions for each of the dimensions, adding rationale for each ranking and sometimes example artifacts, such as screenshots, photos, or transcript quotes.

The Excel work was originally to see if any patterns emerged by comparing across the dimensions. I think some are. At least it gives some compelling visuals.

In the spreadsheet, for each dimension, I arrayed the sessions left to right, assigning a unique color to each (blue for Ames Group 1 (AG1), orange for the Hab Crew session (Hab), etc.). Where the dimensions were more simple groupings (such as dimension A1, which just shows the practitioner choice of method), I indicate that rather than a high-to-low array. There is a lot more info and description of each dimension and ranking method in the maps.

Group A are 5 dimensions concerning aspects of a session's pre-existing plan and other pre-session factors such as choice of method and approach. The results look like this:

The 9 Group B dimensions cover aspects of practitioner interaction with participants:

Group C comprises 5 dimensions showing various aspects of the sessions as meetings and discussions:



And Group D is 8 dimensions that focus more specifically on the shaping of the representation:
What was most interesting about moving the analysis into Excel was better being able to compare the dimensions themselves and think about what they might be showing as a whole. It's still early going but it does seem like there are some interesting patterns from a participatory shaping point of view.

For example, it occurred to me that some of the dimensions that came out of the data group into categories that could be used as a kind of index of 'goodness' at least from a shaping point of view. I looked over all the dimensions and identified these 13 that could contribute to such an index:

• A5: Degree of practitioner adherence to the intended method during the session
• A6: Participant adherence/faithfulness to the intended plan
• B8: Practitioner willingness to intervene – frequency and depth of intervention
• B10: Degree of practitioner-asked clarifying questions to participant input
• B11: Degree which practitioners requested validation of changes to representation
• B13: Degree of intervention to get participants to look at the representation
• B14: Degree of collaboration between multiple practitioners (if applicable)
• B15: Degree of collaboration/co-construction between practitioners and participants
• C17: How “good”/successful was the session?
• D22: How much attention to textual refinement of shaping
• D23: How much attention to visual/spatial refinement of shaping
• D24: How much attention to hypertextual refinement of shaping
• D25: Degree of ‘finishedness’ of the artifacts
  

I then derived "scores" for each session along these dimensions by assigning 8 points for the highest-ranked session in a dimension, 7 points for 2nd place, etc. Of course these ratings are largely subjective, but in aggregate and for comparison reasons I think they have some validity (and at least, I've captured the rationale for each rating I gave, so someone else could look at the data and understand (if not agree with) why I ranked them this way).

By doing this, an overall ranking of sessions along this "shaping index" emerges. As you can see in this table, not surprisingly the Hab Crew and Remote Science team sessions are at the top (since they had the most experienced practitioners and were in situ sessions drawn from actual projects). But the Rutgers and Ames sessions also array along this meta-dimension. I then derived a "software proficiency rank" and a "facilitation proficiency rank" from the questionnaire data to compare to the shaping index, and (even to my statistically untutored eye) there appear to be some good correlations, especially with facilitation proficiency.


Here are graphical views of the (self-reported) facilitation proficiency and software proficiency comparison scores:


This made me think there are more interesting comparisons to be derived from the other composite comparisons from the questionnaire data. I could probably spend months going through all the possibilities:


However, I don't want to drown in all this data or let it take me too far off course of really getting at the aesthetics, ethics, and experience aspects of these sessions. Next I will be looking at the sensemaking moment analyses from a comparative perspective, and see what that yields.

Comparing the questionnaire data across sessions

I put up some charts that show how the practitioners in each session group compare along skill/experience lines.

This is not the main cross-session analysis I'm doing, which is more qualitative in nature, but it will help in the overall comparisons.

For example, it's easy to see that Ames Group 2, which was in many ways the least successful of all 8 groups in terms of getting participants to engage with the representation, also had the lowest self-assigned scores in the facilitation measures (slides 2, 3, 4, 5, 6, 7, and 11), although they had a relatively high level of skill/experience with software and hypermedia (slides 1, 9, 10, 11, 12).

One question -- do the (manually added) vertical lines separating the sessions on slides 6-12 help? Or do they clutter up the display?

There are probably better Excel ways to show this, but I'll worry about that later.

Comparing across sessions (part 2)

Following up on this post.

I was able to spend some time this weekend working on this, and got about a fifth of the way through the first round (shaping analysis). What was enjoyable was working with the material in Compendium. I want to be able to do all sorts of comparisons between the sessions as well as look across and through the data in unforeseen ways, which doing this part of the analysis in Compendium will help with, as well as potentially to be an interesting, web-accessible way of giving others access to the materials.

First I went through all of the 'shaping forms' (one for each session, as you can see here), and looked for aspects that appeared to recur throughout each of the session (i.e., dimensions, according to grounded theory). I put one dimension each on index card, then sorted them into groups.


I then typed up all of the dimensions in a Word doc, refining the descriptions as I went, then put them in a rough order within each of the five groups (the doc is temporarily here).

Next, I imported all of that into Compendium. I then worked through the six dimensions in the first of the five groups (Group A: "Aspects having to do with initial plan and other pre-session aspects, such as choice of method and approach").

Doing this in Compendium helped me to further refine, order, and sub-group the different dimensions. I was refining the approach as I went, so not all the 6 dimensions I got through are fully consistent yet, but I came to an approach where I would do the following for each session within each dimension:

• characterize where each session lay along the high-to-low (or other values) for that dimension
• give a justification/rationale for why I gave a session that value, captured as a Pro
• array all 8 of the sessions along the 'axis' for that dimension, generally with the 'high' at the top of the map and 'low' at the bottom

You can see the interim results at these links: With left nav menu or Full page view. This map is the most interesting one, at least visually, since it has clickable thumbnails of the artifacts it refers to.

This is the kind of analysis I envisioned doing when I started this work six years ago. More to come.

Two new items

I've posted anonymized versions of all the individual session analyses done to date here. As I start completing the comparison analysis documents I'll post them there as well.

You can also access this pre-print version of a forthcoming journal article that I co-wrote with Simon Buckingham Shum and Mark Aakhus. It's the fullest description to date of the phd research.

Comparing across sessions

I thought it would be useful to note my thoughts about how the cross-session analysis will unfold before actually starting it.

First I'll look at each of the individual session analyses and compare them horizontally. For example, look at each of the eight shaping forms and look for interesting correlations and differences between the ways shaping happened, look at each of the sensemaking moments for the sessions and compare them, look at the way practitioner focus compares in the grid analyses, etc.

Generally I'm going to be looking for the way the following dimensions compare across the eight sessions:

- intended and 'lived-in' narratives, to give context to the breaches (sensemaking triggers) as well as 'canonicity' of the sessions
- sensemaking triggers (discontinuities, dilemmas, and anomalies that the practitioners respond to)
- shaping (the aesthetics/form of the representations, before and after sensemaking triggers)
- collaboration & practitioner/participant interaction, especially choices the practitioners make that affect the 'interests' of the participants (i.e. ethics)
- types of practitioner focus
- types of improvised practitioner actions
- what is done in the sessions to make the representations 'work' (to make them coherent, engaging, useful etc.)

I may construct some tables showing short summaries of each of the sessions as they fall into these categories.

Hopefully some interesting patterns will emerge. I should be able to come up with some axial coding-type dimensions (e.g. what are the 'more' and 'less' types of values that would emerge when I compare sensemaking triggers, what would 'more' and 'less' types of values be for aesthetic shaping and ethical choice-making, etc.

That may lead to having something interesting to say about this group of sessions as a whole.

Finished another piece

This post is only to gloat over completing a sub-milestone on my PhD work, the second of the three analysis phases.




As the picture shows, I first did a set of six different kinds of analysis on each of the eight sessions I'm looking at for the thesis. I finished that about a month ago (after more than two years of work). The piece I just finished was comparing the questionnaires that I gave each practitioner to the individual session analyses.

I had asked each person questions about their software and facilitation skill levels and experience, and their opinions about the sessions they had conducted. There was nothing breathtaking that emerged from that analysis, but it filled in another piece of the puzzle by being able to compare what had happened in a session to the level of practitioner experience and skill.

For the last analysis sub-phase, "Cross-Episode Analysis", I will (at last!) look across all eight sessions and compare them on a number of criteria. I've been looking forward to doing this for years now, but had to finish all the individual analyses first.

Hoping to do this part in the next two months or less (I'll probably have to take some vacation time to do so). Then the actual writing of the thesis will start. It's been a long time coming (since Oct 2003).

The ethics of shaping

As the light at the end of the PhD tunnel starts to turn from a pinprick to a dime-shaped glow, several people that have recently listened to me talk about my research have mentioned that they see similarities in the work I do at my day job.

I work in software usability and user interface design for systems used by call center reps inside a large company. Like the practitioners I've been looking at for my research, making decisions about the UI in enterprise software design has the same degree of connections between choices about the form to give a screen and the way this will affect the interests of the people who come into contact with it. And, as with participatory representations, it's not just a unitary set of considerations. There are multiple kinds of people involved -- clients, end users, other IT teams, auditors -- each with diverse imperatives that drive them. Even "users" are not a monolithic block with one set of interests. What works best for an experienced, expert user is not the same as what would be best for a new user encountering a task or screen for the first time (among many other sorts of differences between users).

We are constantly balancing considerations like speed of development, ease of maintenance, testability, business rules, time constraints, future changes and plans, and usability. Each of these dimensions have ethical implications and trade-offs. As a usability person, it would of course be easiest to give ease of use the principal value (and of course, it's our job to do so) and give it the highest ethical importance. But if we know that the best UI design will have costs and cause problems for others with equally legitimate interests, we have to weigh those factors against others in our choice-making.

For example, take a simple change to an existing dialog box in an ordering system. In our work we get requests for these all the time, perhaps half a dozen a day (along with much larger projects). From a usability point of view, we always want dialog boxes to have clear window titles that give an overview of the situation; concise and straightforward text in the box that lets the user know why the box appeared and what they can or should do; and clearly labeled buttons that spell out the choices they can make (e.g. "Proceed with Order" and "Return to Address Entry" instead of "OK" and "Cancel" or "Yes" and "No", so often misused or unclear). N0-brainer, right?

In the abstract, yes. But what if the issue comes up (as they so often do) two days before code freeze for a complex release in which this change is just one of hundreds affecting many interdependent systems? And (as is often the case) what if the dialog box is in an older part of a system where all such boxes are coded as simple alerts using constants in a programming language which don't allow for descriptive button labels or window titles?

Insisting on "proper" usability design in this case would require custom programming, which takes time (that the development team doesn't have) and additional testing, which adds costs that comes out of someone's budget (which then is less available for other things). It may mean a change to documented requirements, which then requires formal review by people who already have not enough time for all they have to do, and creates risk to delivery which may jeopardize delivery and testing.

A simple answer in this case might be, "just make simple text changes using the existing code for this release, and do a proper job for the next release when there's more time". And often we do take that approach -- when we can. Sometimes there won't be another opportunity because the dialog box is in a part of the system that won't now get touched for a long time, and it would be too expensive to open the code just to make that change (since that, again, would require development time, testing, requirements documentation, etc.). So in many cases the changes are "now or never".

These kinds of dilemmas are very familiar to anyone working in software development, and I'm not saying anything new about them. However, from a research point of view I am particularly interested in highlighting how the choices that have to be made about something's visual form relate to the ethical dimensions of such choices -- the conflicting imperatives that are all valid and which all reflect legitimate interests.

My focus is on the ethical choices involved in making decisions about form, when in the context of shaping interaction and experience for others with mediating tools and representations. In my research I've been applying it to the special case of people playing a group facilitative role with hypermedia representations, but really it's more broadly applicable. What I feel emerging is a set of ways to enable other people to think and talk about these choices as they relate to their own work. I'm not so much interested in being the expert assessor myself, though I have had to do dozen of assessments of practice in the course of the research (I just finished the last of the 47 individual analyses yesterday!) and I do similar kinds of assessments every day on the job.

Rather, I want to enable and enhance people's ability to think about these issues for themselves, and give them some tools to do so. I want to make the question of "how does the way I shape the things I make affect the people who interact with them?" something that's accessible for people to talk about and get insight on.