Online Dispute Resolution has long been an example of the computational law future. In this post, Madeline Oldfield provides a first-hand look into how her team in Victoria, Australia worked to design and pilot an ODR system, the lessons learned, and recommendations for others.
This is a story about how a group of committed stakeholders sought to reimagine civil justice by exploring whether online dispute resolution could improve the access to justice for citizens in the Australian State of Victoria. Hopefully, this account motivates others to think outside the legal box, enjoy the realm of ambiguity, and inspires them to embark on their own journey to facilitate computational justice.
In March 2016, 35 stakeholders across legal, community, government, academic, and technology sectors attended an expert roundtable to discuss the potential for an online dispute resolution (ODR) pilot for a Civil and Administrative Appeals Tribunal (the Tribunal) in the Australian State of Victoria. Their interest had been sparked in part by the inability of citizens to gain access to justice in both state1 and federal2 jurisdictions. And so began a not-so-subtle revolution.3
The Tribunal, its stakeholders, and the community were frustrated by the slow, complex, and cumbersome manual processes inherent in the ‘traditional’ paper-based legal dispute resolution system, a system that causes disadvantage to regional and vulnerable citizens. They saw ODR as an opportunity to improve access to justice for small civil claims disputes.4
There was general consensus that embracing technology, through ODR, could help citizens better access justice services. Even though legal expert systems have been around for decades, ODR was still very much an emerging tool.5 Fundamentally, there was doubt that the law could be practised ‘computationally.’ To dispel this doubt, it required all participants (be they parties, practitioners, decision-makers, or administrative staff) to learn new skills and apply a different way of thinking when approaching the task of dispute resolution.
ODR was unproven in Victoria and, while there was some guidance from the international community in terms of their journey towards using ODR,6 the State had not the luxury of time nor budget. There was no road-map and no evidence that ODR could or would improve access to justice. There were but a bunch of political, practical, and potential challenges that to date had resigned the idea of practicing law computationally into the “too hard” basket.
Nevertheless, the Tribunal and its stakeholders could not continue to ignore the ever-increasing gap between met and unmet legal need: research had indicate that there were roughly 630,000 legitimate consumer legal issues annually, of which 150,000 were not dealt with by the legal system.7 For those who did, and stubbornly stayed the course, only around 7100 small claims were heard and determined by the Tribunal in the same 12-month window.8
It was apparent that the traditional approach to dispute resolution was unable to address these systemic issues. A new approach was required — one that involved a new and digital reality in which end-to-end dispute resolution could be achieved efficiently without undermining the integrity of the justice system.
So, how did the project team manage to build an entire, virtual, end-to-end justice service so citizens could ask the Tribunal to hear and determine small civil claims disputes online in only three months for less than AUD $800,000?9 By focusing not on the technology, but on the people.
Experts within the stakeholder group advocated for a test-and-learn approach to explore the potential for ODR. So the ODR pilot program was born. Recognising that success required a people change management endeavour, the pilot program embraced modern project and design methods. Instead of using Agile’s tried and tested ‘fail fast’ mantra (because who in government likes admitting to failure?), it ‘learned quickly and often’. Work products were tangible and interactive. An ODR Experience Lab was built, wherein ideas could be formulated, tested, and refined. Ideas were turned into reality using ‘minimal viable products’ (MVPs) to demonstrate ODR’s value to the community by improving access to justice for Victorians.
By committing to adopting a human-centered design approach, people remained at the heart of design and implementation efforts. Suddenly, it seemed possible to solve seemingly intractable problems.10
A series of unique experiments were developed to prove or disprove the following hypothesis:
If [the Tribunal] was to introduce online dispute resolution, the Victorian community would experience improved access to justice.
The result of the pilot program enabled the Tribunal and its stakeholders to understand how ODR and digital dispute resolution services added value in the current strategic and transformational climate, without having to commit to significant investment. And while this wasn’t a new concept, it was the application that made this approach unique: the pilot was able to establish a definition and baseline (qualitative and quantitative) by which ‘access to justice’ could be measured.
There are a few ways to measure access to justice. The pilot program focused on only five11 as its starting point (after all, if the pilot could not account for five metrics, how could it scale to 10, 20, or beyond?):
1. Accessibility: Justice initiatives should reduce the complexity of the justice system. Accessibility includes initiatives that create or alter rights, or give rise to decisions affecting rights, and should include mechanisms to allow people to understand their rights.
2. Appropriateness: The justice system should be structured to create incentives to encourage people to resolve disputes at the most appropriate level.
3. Equity: The justice system should be fair and accessible for all, including those facing financial and other disadvantages.
4. Efficiency: The justice system should deliver fair outcomes in the most efficient way possible.
5. Effectiveness: The interaction of the various elements of the justice system should be designed to deliver the best outcome for users. Justice initiatives should be considered from a system-wide perspective rather than on an institutional basis.
Using these five measures, the pilot created a balanced scorecard to understand whether ODR would be desirable, viable, and feasible. Mostly, the platform needed to be desirable to the people using the it (be they the party to the dispute, the practitioner, the decision-maker, or the administrative staff). Moreover, the platform needed to be financially viable; there’s no point designing something that would be cost prohibitive. Finally, the ODR platform needed to be technically feasible — vapourware and wireframes would not work.
Rigorous customer and stakeholder testing activities were conducted so as to identify pain points that could be mapped to available data points. This enabled deeper understanding into the customer experience. Lean Six Sigma methods were used to identify waste in the dispute resolution value chain, and identify opportunities to remove blockages preventing access to justice.12 This information was captured in a Service Design Blueprint that was displayed in the ODR Experience Lab. There was no escaping the power of this work product. It was brought to life across a 5 metre x 3 metres space:
Mapping out the project provided the stakeholders with a common language to understand the depth and breadth of the challenge ahead. The right methods became intertwined with the right mindsets.
Community expectations have changed dramatically over the past 25 years. We bank online. We shop online. It is fast. It is easy. We expect more and more access to services online. We expect immediate, seamless experiences that anticipate our needs and resolve irritating pain points. The justice sector is not exempt from such expectations.13 So any ODR experience had to be more than just working tech; the entire experience had to be better.
To be successful, the Tribunal needed to win the ‘hearts and minds’ of its key stakeholders – whose professional realm was steeped in evidence based traditions. Many stakeholders were steadfast in their views. This manifested in both extreme support and fierce opposition to using ODR to facilitate law computationally.
But to take the solution to the next level, there was a need to cultivate empathy in the ODR design and eventually trial real-life small civil claims disputes. The parties to the dispute were not faceless: data indicated that small business owners were the most common users of the Tribunal’s small civil claims dispute service. This brought its own challenges in terms of supporting the liquidity and potential survival of Victorian small businesses, reducing lost productivity to attend a physical hearing (which could take the entire day), and overcoming geographical barriers to attend in person, particularly given the tyranny of distance and importance of small business to regional and rural communities.
So with the median time for finalising a dispute at the Tribunal around 11 weeks,14 and Victorian Small Business Commission estimating that small businesses get paid, on average, 30 percent of their original claim with the Tribunal in business-to-business disputes,15 there were significant opportunities to improve access to justice outcomes.
An extensive international search, culminating in an open-market competitive tender, revealed that there was no single ODR platform or managed service to solve all stakeholder pain points. As such, it was necessary to consider multiple solutions, capabilities, and other creative means by which end-to-end digital dispute resolution could be delivered. There was barely time (or money) to buy a Commercial-Off-The-Shelf (COTS) ODR tool, let alone build one.
To achieve the impossible, the pilot started with an impossible ask. It asked for ODR tools with the capability to create a working prototype or proof-of-concept in a 4-week timeframe, within a maximum budget envelope of AUD $100,000. Research revealed a list of domestic and international suppliers of ODR COTS products, case management, complaints management and conflict resolution tools. All were invited to respond. Each respondent was required to meet a set of mandatory criteria, including the requirement to demonstrate a commercially available, working technology.
An Australian start-up16was ultimately successful after demonstrating their solution’s approach to human-centric experience design. They demonstrated their ability to be flexible and agile, and to pivot when new or important information came to hand that might help achieve improved access to justice. This rapid prototyping enabled the pilot to continually ‘learn quickly, learn often’ by incorporating feedback from stakeholders in real-time, ultimately mitigating risks and improving the experience before the live trials commenced.
The success of the live pilot required far more than just a sound platform. It had to consider the change management elements to wrap around the platform. This gave the tech meaning, and provided the situational and contextual application to bring the software to life through:
Customer deployment and readiness plans — which included identifying, briefing, and supporting participants (applicants and respondents), recording feedback and tracking key performance indicators to assess the impact on access to justice.
Business deployment and readiness plans — focused on the internal stakeholders administering online dispute resolution.
Additionally, the ODR platform had a dedicated support team to help with the issues identified during live testing, which were triaged and resolved in real-time. This function not only helped stakeholders feel supported when using new technology, but it also gave the team an opportunity to formally capture areas for continuous improvement. This was critical to ensure the platform was fit-for-purpose and capable of evolving to meet user needs.
Reflecting on preliminary research and pilot outcomes, the following key insights were uncovered:
1. No government, internationally or domestically, had successfully implemented an automated end-to-end ODR customer experience. Humans still play a vital role in the process.
2. There is no one-single ODR solution. ODR takes many forms and spans a spectrum from self-resolution to guided resolution. And that’s okay.
3. To be relevant today and in the future, the government needs to provide modern, efficient, easy-to-use digital services to its citizens.
4. ODR for small civil claims at the Tribunal is desirable, viable and feasible. There are positive indicators that ODR can improve access to justice, particularly in the areas of accessibility and efficiency. In fact, so positive that the pilot was overwhelmed with community interest: although only 30 cases were planned to be in the scope for the live pilot, ultimately 65 cases were heard with 21 settling prior to hearing.
5. ODR is an ‘and’ proposition – there will be people who will not, or cannot, use a digital channel to access justice. ODR needs to be provided alongside traditional and face-to-face models of service delivery in the justice system. Being effective and staying relevant is about providing different service offerings, not mandating service channels.
The entire process produced many insights and learning opportunities. With the benefit of hindsight and sharing knowledge from this experience, it is possible to learn and do better. The Pilot in Victoria underestimated community interest. So, while it was prepared with plans to scale, it was unprepared for just how quickly this would need to be done. In hindsight, there may be the need to stop sourcing bespoke ODR platforms. Instead, there is a need to start looking at broader, industry-leading platforms and digital offerings with similar capabilities alongside bespoke ODR platforms so there are better options to improve large scale, digital dispute resolution services. Unsurprisingly, future ODR endeavors should continue to assess and understand broad ecosystem challenges, while still ensuring human needs are at the centre of design. There is the need to to remain resolute that the process of ODR is a change management endeavour and not a system implementation.
Future ODR, and computational law endeavours more broadly, should continue to put community needs at the heart of their efforts — and do what they can to ignore the reasons why ODR shouldn’t work, instead focusing on the reasons it can be better than what currently exists.
So, what happens now? Where to from here? And what became of the little ODR initiative that could? Alas, in the middle of an election cycle, with political parties shifting to other, critical state-based needs, Victoria’s ODR pilot has seen funding shifted elsewhere.
While this hasn’t changed the findings, nor the passion from committed stakeholders, it does raise interesting questions about the priority of civil justice funding and whether it can compete with the growing demand for other critical services, such as healthcare, education, infrastructure, and even criminal justice. But what is known is that online dispute resolution services, and the methods, mindsets, and platforms needed to improve access to justice, are ready and waiting as soon as the priority call is made to facilitate law through more computational means.
As COVID-19 continues to force society to grow and adapt, there remains a glimmer of hope if this tale can inspire additional calls to action — something more than an article. Hopefully, this seeds a bias for action in such a way that inspires readers to embrace new ways of operating. After all, now is as-good-a time as any to start a civil justice revolution. You have nothing to lose in adopting new ways of working to facilitate law computationally. You have everything to gain by learning.
Over the last 2 decades, Madeline has worked closely with a diverse range of industry leaders, across multiple sectors, to shape business and service transformation programs during periods of rapid change and disruption. Taking a human-centred approach, every program she works with embeds this experience firsthand, including achieving international recognition as the Program Director of the ODR Pilot Program, which won the 2019 Project Management Institute’s Global Project Excellence Award. Madeline’s professional experience is complemented by a Master of Laws (Juris Doctor) and a double undergraduate degree in Commerce and Information Systems. In addition to several agile certifications, Madeline is also an Adjunct Fellow with Sir Zelman Cowen Centre.