Book Review: Visualizing Argumentation:
Software Tools for Collaborative and Educational Sense-Making

Paul Arthur Kirschner, Simon J. Buckingham Shum, Chad S. Carr
Springer Verlag (April 2003)

ISBN 1852336641 (paper)

Buy it from Amazon.com
Buy it from Amazon.co.uk

Other Reviews

In an age increasingly dominated by the Internet, limitations on time and place in projects are seen as problems. Computer-Supported Cooperative Work (CSCW) aims to facilitate the difficult task of getting people with different understandings to work together. An interesting aspect of this is to get them to make sense of complex arguments (such as requirements, and safety cases) so they can share ideas and collaborate on improving these arguments.

The aim of the whole exercise .. was to promote rational consensus on the main issue. The next stage .. was to review the arguments as presented on the visualization and to see what this implied... By this time, however, something remarkable had already happened. As the negative case was being visualized, one argument emerged as conclusively establishing that the proposition was false…

The remarkable part was that when this objection was laid out clearly in the context of all other relevant considerations, its overriding force was fully appreciated in a way it had never been when the arguments were rehearsed in standard ways. [Tim van Gelder, p112]

The basic assertion of Computer-Supported Cooperative Work (CSCW) is that there is a synergy when the right technology combines with skilled human effort. The special claim of this book is that CSCW tools that illustrate arguments and their supporting evidence contribute significantly to that synergy. Tim van Gelder in a way summarizes the case: at least sometimes, it is virtually enough just to elicit and diagram an argument. Once everybody can see the battle-lines drawn out, the need for debate often melts away. Presumably the psychological reason for this is shared experience. Haven't you felt, on hearing on the radio that 70% of the public agree with a claim, that you yourself are 70% persuaded and 30% not persuaded? Everyone knows the arguments, but not everyone visualizes them clearly. If so, consensus is just a visualization away.

Part I, Foundations, consists of just two chapters - on the roots of Computer-Supported Argument Visualization (let's call it CSAV for short) by Buckingham Shum, and a cognitive framework for cooperative problem solving with CSAV, by Jan van Bruggen and others. Both discuss the IBIS method (which is supported by several tools).

One wonderful example: John Henry Wigmore's Chart Method for analysing legal evidence was devised in 1913. It was certainly an AV method though the computer was just pen and paper. Hypertext, too, was from the start about argumentation, as Vannevar Bush made clear in his famous article As We May Think in 1945. Another major root is the philosophy of language of Stephen Toulmin, whose Uses of Argument (1958) counters 2000 years of excessive Aristotelian logic. Toulmin devised an argument structure involving Datum, Claim, Warrant, Backing, and Rebuttal, all to be arranged graphically - again, clearly an AV format. Buckingham Shum gently pokes fun at 'tame' problems, contrasted with Horst Rittel's 'wicked' problems that CSAV tries to address. He observes that

'Tame problems may even be amenable to automated analysis, such as computer configuration design or medical diagnosis by expert system.' (p11)

This is a pointed reference to the most famous examples of Artificial Intelligence such as Mycin. CSAV's answer is that machines are far too stupid to solve wicked problems, but that computers and people together may do so.

van Bruggen makes the point that CSAV "is primarily used for the solution of ill-structured problems" (p27) - the term goes back to Herbert Simon, and is certainly in the same area as wicked problems. Such problems require complex reasoning, the coordination of multiple agents, and CSAV environments that meet stringent cognitive and communicative requirements.

Part II, Applications, consists of seven chapters illustrating different CSAV approaches. Gellof Kanselaar and others write on designing tools for collaborative learning; Carr writes on teaching legal argumentation; van Gelder on deliberation (i.e. critical thinking); Jeff Conklin on dialog mapping using IBIS; Albert Selvin on helping groups use CSAV; Robert Horn on large, long-lived interdisciplinary arguments; and Buckingham Shum and others on internetworked (i.e. hypertext) argumentation.

This part of the book inevitably repeats the basic message in various ways, arguing the merits of different approaches toolkits. Kanselaar illustrates the TC3 tool with several diagrams full of Dutch text, structured much like Toulmin; the article argues that education is a process of argumentation, but their analysis showed that people actually used their tool for visual representation, not to trigger discussion or ideas.

Carr is on stronger ground talking about legal arguments; these are admittedly complex and adversarial, and concepts like claims and evidence are well accepted. He uses Toulmin's scheme, and shows in a careful study that - contrary to expectation - students did not get better exam scores after using the QuestMap CSAV tool, though they may have ended up with better ability to construct arguments (rather than the ability to write them, as tested by the exam).

Van Gelder starts slowly, recapitulating the basic comparison of argument in text versus diagrams, but arrives at

'Deliberation is usually done quite poorly' (p104)

In contrast, Conklin says that IBIS needs a 2-day course and then some practice "with the emphasis on practice". Either IBIS is much harder than Reason!Able, or one of the two authors is wrong. Dialog Mapping basically consists of IBIS use with a skilled facilitator, who is supposed to deal with both the people-facing and the system-facing issues at once (it isn't clear why a 'scribe' to help with the actual writing and drawing isn't part of the method - this is a well-known role in systems work at least - but 'or two people' is mentioned in a footnote). The case study is interesting but startling: the QuestMap CSAV tool (implementing IBIS) was applied not just to support face-to-face meetings with Dialog Mapping, but also for the wholly different job of virtual meetings, which could be asynchronous. Not surprisingly, some people "reacted strongly" to this "as just another passing fad of management". Indeed there are powerful arguments against reliance on technological mediation in place of actual meetings. The case study company, SCE, used the tool for a decade because of the 'commitment of Dr Hertel, the Director'. This is quite a low-tech reason for the success of a hi-tech tool, but probably typical in organisational change - just arguing about gadgets is missing the point.

Selvin talks about face-to-face VA: the method (Compendium) and tools (Mifflin, QuestMap) support facilitation of meetings, helping to impose an appropriate structure. Obviously the risk is that the structure is wrong, or resisted by participants. Selvin claims that 'I .. believe that I could add value to almost any group in a problem situation' through years of experience, but that few (or not enough) similarly skilled facilitators exist, and that it is an open question whether the required skills can be taught.

Horn also revisits the basic story of CSAV, offering guidelines for good practice when presenting big philosophical arguments. The example is the perennial Can Machines Think?, and arguments are given in their earliest or strongest forms, such as those of Heidegger, Dreyfus (What Computers Can't Do), and Searle. The documented argument ends up in seven large maps - each a 3x4 foot poster! - presenting about 800 steps or 'moves' and 60 photographs linked by 'is supported by' and 'is disputed by' arrows. He claims that this work 'has opened up the development of a new field for the understanding of intellectual history'. The future holds a hypertext version of the approach:

"we envision the ability over time to witness the creation of vast webs of argumentation maps on the web that cover many fields and that show us how humanity working together has asked, debated, and sometimes even answered the Great Questions." (p183)

This bold claim seems to be partly contradicted by the statement

"we are not interested in developing a chatroom. The content review needs to be done by specialists."

Well, if so, and only the few can create the maps, how is a vast web ever going to be created?

Finally, Buckingham Shum takes up the challenge of hypertext, more or less as thrown down by Horn. Will scholarly knowledge still be published solely in prose, or as some kind of semantic web? Will that enable more effective dissemination and analysis of ideas? Sven Birkerts would I think say that it might assist dissemination but that it threatens the really deep integration and human 'sensibility' that goes beyond mere 'analysis'. This question seems to strike at the heart of CSCW (and CSAV for that matter). If McLuhan is right and the medium shapes the message, then mediating between people with tools changes the nature of the communication, and of what people become able to understand. The brave pioneering authors of this book are obliged (in a quiet sort of way) to believe that the tools, rightly and intelligently applied, will help to bring on the bright new dawn of technologically-mediated splendour. The dour sceptics wonder whether the new media will not instead reduce people's attention span and ability to reason. At the least, the case for CSAV is today not proven.

Doug Engelbart - who has been writing on CSAV since before it existed, back in 1962, concludes with an Afterword. He ranges grandly from his Bootstrap Institute on increasing humankind's collective problem-solving capabilities. It is almost impossible not to believe that computers could do this today; it was very different five decades ago. Engelbart hopes that CSAV tools and methods will themselves improve the pursuit of new tools and processes - a bootstrapping approach. Perhaps today's CSAV tools, and their rather limited diffusion in industry, don't quite seem to be this powerful; but perhaps they aren't far from reaching the critical point where they really start to help.

One omission that seems glaring to a systems engineer's eyes is any mention of safety case argumentation. The situation in which developers are most likely to see or document arguments - if they see any - is in safety engineering. Well-known notations include Goal-Structuring Notation, Adelard's Claims Arguments Evidence, and Weighted Factor Analysis. These are definitely CSAV tools and methods, but they seem to fall foul of the 'not invented here' syndrome: the CSCW school doesn't read books or papers about safety, and vice versa. It seems too much to hope that different schools of thought - CSCW, HCI, Software, RE, Systems - should read each others's literature, when the evidence shows that they hardly read their own. The irony, if that is what it is, is that as Sven Birkerts so passionately argues in The Gutenberg Elegies, the rising tide of e-data that washes through our networks actually shortens our attention span so much that we can hardly concentrate on the in-depth argument and reflection presented by a book any more. CSAV may, perhaps, help with argument understanding; but equally, it looks like part of the larger problem.

This is a multi-author books that fits together rather well, and has a clear and important message for everyone involved in complex business and system problems. The question of whether computers can assist human thought is a deep and challenging one, and deserves everyone's attention.

You may like to visit the book's website.

© Ian Alexander 2003

You may also like: