A fascinating story in today’s FT by Richard Waters asking whether Watson can save IBM (it is pay-walled but well worth a read). IBM’s Watson is the much known and increasingly heavily marketed IBM artificial intelligence phenomena. It beat real humans in a complex general knowledge game (Jeopardy) and its apparent ability to help diagnose cancer has been much promoted, with this example being particularly famous:
A demonstration showed how quickly Watson is able to diagnose illnesses, and provided a real life case that took doctors and nurses six days to diagnose, and only ended with the correct diagnosis because a nurse had seen the disease before. Based on symptoms input, Watson was able to correctly diagnose in minutes. The demonstration took place at IBM Watson’s New York City, New York office on May 27, 2015.
The story reports on IBM’s struggles to apply Watson. The ‘we will go to the moon’ style statements of Watson’s potential have been tempered by a great deal more pragmatism. They have broken down Watson into 40+ different products. These products are quite specific or even described as basic, although I’d describe them as more prosaic. Nevertheless IBM report these are selling quickly. They include things like sentiment and character analysis from things like tweets (God help us all, well me anyway) and intelligent data analysis.
Watson’s famous victory in Jeopardy inflated expectations that great leaps could be made in terms of computers understanding complex, unpredictable information and answering complex questions. The promise has not yet been fulfilled. The FT story suggests that the well trailed example of a Texas hospital trying to use Watson to help diagnose cancer has proved difficult, with the report suggesting this has not (yet) been successful – despite the demonstration above. The hospital’s head of innovation says, “Turning a word game-playing computer into an expert on oncology overnight is as unlikely as it sounds” but retains her hope that it can get to something like the hoped for system.
The claim seems to be that Watson is being used to brand more pedestrian applications than the original conception of Watson as an adaptive artificial intelligence. Another way of putting this is it is an early stage technology rather than an mature one: a (still rather impressive) ZX Spectrum to the MacBook of 2020. Pragmatism sometimes yields results though. An Australian energy group is reported as using Watson’s natural language processing facilities to investigate its database of documents and mine intelligence from it: the example given is working out from 30 years of projects the best calculation is for pipeline pressure. According to IBM, the application still involve “high-end ” AI and quick processing of unstructured data .
A further problem that I was pleased to see recognised is that of trust. Where an artificial intelligence system makes human like predictions, it does not ordinarily offer human like explanations as to how it reaches its decisions. It is a problem likely to be important in law, where reasoning can be characterised as part of, not just preceding, the result. IBM are reported to be trying to overcome this, so far without – according to Waters – significant success but at least they are trying.
For lawyers, the position may be tantalising. Some of Watson’s products – and the Australian example – promise real applications. Yet law’s data may be peculiarly complex and nuanced. Success in current AI seems to be most likely in areas of work with already high levels of predictability and structure. That is the claim made in a very interesting paper by Remus and Levy which seeks to draw some boundaries around the near term possibilities of robot lawyers. The paper is ably summarised here by Caroline Hill. Remus and Levy sometimes under-estimate or miss current potential amongst existing providers of AI-like services, but they also make a valuable attempt to remind us that there is a big gap between what can currently be done and what is claimed as possible. There needs to be a much stronger focus on the currently possible, what works and what does not work, why it does not work and why, and the normative implications of greater artificiality, automation and probabilisation of law. That debate has been masked by a phoney war between futurists and Luddites and is greatly hindered by an absence of real evidence.
A final lesson from the FT story is this: a key driver of the advance in artificial intelligence has been the availability of large amounts of data. Watson’s apparent ability to cope with unstructured data may encourage Law firms and legal departments with the ability and willingness to access repositories of such data to look for their equivalents of pipe pressures through the knowledge mining above. What could be learned by intelligent search through case files, opinion letters, advice letters, deal bibles, contracts and the like? This would require investment of time and money that the large accountancy firms may have more of a culture and appetite for than law firms and may lead to incremental rather than transformational change: more of Team GB/David Brailsford’s 1% increases in effectiveness than a Steve Job’s like seizing of the market, perhaps. Equally, the holy grail of better identifying what works requires tying lawyer input (advice given, documents drafted, litigation and trial strategies) to outcomes in terms of actual behaviour, value to the client and so on. One sometimes sees and hears of small advances in the making of such linkages but lawyers generally have relied on a folksy, experiential approach to linking the quality of what they do to the impact of their work on clients and the broader world. Better making that linkage is a key challenge for legal practitioners with pretensions for seizing high volume and high value impact.
So whilst the FT and the Remus and Levy pieces can be seen as pricking the bubble of law tech evangelism somewhat, I prefer to see in it a healthy maturity. More real work is being done to apply technology. More failures are coming faster. After the unknown comes the hype and after the hype comes the disappointments and progress of reality. We can probably expect advances, but advances that are modest and worthwhile (though not as worthwhile as they are hyped to be), rather than transformational. And we will tend not to know much about them yet. At least in the near term.