Kia Motors Tests Outside Counsel Tech Skills
Editor’s note: This is part one of a two-part follow-up to “Tech Drive,” which reported on Kia Motors America audits to assess the technology skills of its outside counsel.
I audit outside counsel’s competence with technology. The audit is driven by fear — fear of wasting company money on low value-added work. The audit tests my hypothesis that lawyers, as a group, are deficient in their use of technology and that a direct consequence of this incompetence is that clients, like my company, Kia Motors America, outlay outrageous sums for unnecessary busywork. Thus far, my audit has confirmed my hypothesis.
The audit currently consists of four mock assignments that a law firm associate must complete using standard software (e.g., Word, Excel, Acrobat). Done correctly — i.e., utilizing some basic, built-in functions of those programs — the first of these assignments should take less than 20 minutes. Done incorrectly — i.e., not relying on the functions — the assignment takes more than five hours to complete.
Not a single associate at any of the nine firms I have audited has come anywhere close to the 20-minute mark on the first assignment. That is, all of the associates approached the assignment in ways that would have required five to 15 times longer than necessary. At $200 to $400 per associate hour, such inefficiency suggests to me that, indeed, waste is a righteous concern.
Failing my audit has repercussions. Of the nine major firms I have audited, all nine have failed — some more miserably than others. A few of these firms were auditioning for work and were not retained. Other firms, including longtime incumbents, agreed to rate reductions. One firm, for example, took an across-the-board 5 percent reduction that will be restored if they are able to pass a subsequent audit. Another firm agreed to a significant investment in associate training and has worked closely with me to upgrade a substantial number of their internal practices and processes. Finally, audit results influence my review of counsels’ billing entries.
My audit is a prototype and, like its creator, not without patent shortcomings. That the audit delivers material benefits despite its flaws is a testament to both the robustness of the audit concept and the depth of the legal professions’ technology problem.
THE AUDIT FRAMEWORK
Technology: The audit focuses more on competence than technology, but technology still matters. Particular points of emphasis include peripherals (e.g., a second screen) and mobility (e.g., mobile Wi-Fi). But the audit is primarily concerned with the proper use of core features of ubiquitous programs (Word, Excel, etc.).
People: People, not machines, are the common source of technology-related inefficiencies. As a group, lawyers suffer from a proficiency deficit, i.e., an inability to utilize the technology tools at their disposal. More pointedly, few lawyers are trained to use their principal software programs. Training is fundamental. Business software programs are not particularly intuitive. The requisite skills have to be learned. At a certain level of expertise, self-teaching becomes easier. But, self-taught or not, actual learning is essential. Lawyers, however, are neither trained nor tested; they are left to their own devices.
Process: As I see it, the failure to train lawyers to properly use what have long been basic tools of their profession is symptomatic of a much broader aversion to system building within large law firms. Weak ties and big egos stifle genuine organizational change. These challenges are positively correlated with size. An unfortunate consequence is that larger firms tend to promote their economy of scope without developing their economies of scale. They cross-sell their expertise; they create cross-functional teams. But large firms do too little to propagate best practices, institutionalize deep knowledge, or systematize their basic functions. The sole advantage to scale becomes the capacity to throw expensive bodies at a problem. It did not surprise me that of the nine firms I have audited thus far, the best performer was the only firm outside the Am Law 200. That small but prominent firm was markedly better than the two audited firms in the Am Law Top 10 (revenue).
THE AUDIT: NUTS AND BOLTS
The mechanics of the audit are simple. There are 4 mock assignments. The sacrificial lamb — an associate selected by the firm — is charged with preparing:
- Exhibit binders for arbitration.
- An opposition to a motion for summary judgment.
- Written discovery responses.
- A settlement agreement.
Each of the assignments is broken down into a series of tasks. For example, in preparing the exhibit binders for arbitration, the associate is given an Excel spreadsheet of deposition exhibits. See Figure 1.
The associate is directed to turn the spreadsheet into, among other things, a simple index of arbitration exhibits. Based on the “Include in Arbitration Binder” column in the original spreadsheet, the index might look like Figure 2.
Similarly, the associate is tasked with using the provided spreadsheet to generate discrete lists of exhibits associated with specific witnesses on specific topics (again, based on the information provided in the columns “Witnesses” and “Issues”). For example, see Figure 3.
On a superficial level, such tasks are commonly assigned to junior associates, and it is therefore appropriate to observe how they handle them. More importantly, completing these tasks efficiently requires a certain level of proficiency in Microsoft Excel. That is, instead of going line by line — as most participants do — the economical approach is to utilize some of Excel’s rudimentary data functions, like sort and filter.
The difference between a brute-force approach, i.e., going line-by-line and an automated approach, e.g., using filters, is significant. For the seemingly simple tasks just described, the brute-force approach requires more than an hour. The automated approach takes only a few minutes. A trained associate can complete Assignment #1 in less than 20 minutes. By contrast, Assignment #1 takes the untrained associate more than 5 hours.
While the particulars of Assignment #1 may not be a daily occurrence, the skills tested are generally applicable. Staying with the examples above, much of the data that Kia provides to outside counsel in the course of representation is transmitted as Excel spreadsheets. The data is essential but will often prove useless unless the recipients can organize and analyze it using the same basic Excel functions tested in the audit.
Assignment #1 also tests PDF skills using mock exhibits. Assignment #2 — preparing an opposition to a motion for summary judgment — further examines the associate’s ability to manipulate PDFs in the context of assembling a federal e-filing. Likewise, in preparing the memorandum for Assignment #2, the associate is tested on a number of basic skills in Word.
As mentioned above, beyond the associates’ training, the assignments are designed to reveal whether the firm has implemented procedures and processes that increase cost-effectiveness. In Assignment #3, for example, the associate prepares written discovery responses. The associate is not actually expected to draft the responses. Rather, the object of the inquiry is whether the firm has setup a logical, deliberate process for the associate to do so.
At the most basic level, the associate should turn the discovery demands over to a word processing department to create the response document. Most associates do just that and get back a Word document that provides a blank after each request. Typically, an associate will then open up the last discovery responses they drafted to copy and paste the standard boilerplate — i.e., general objections and common specific objections (e.g., burdensome, vague).
The premise of copy-and-paste has merit. Truth be told, there are a finite number of general and specific objections. There is little reason to reinvent the wheel. Execution, however, is generally haphazard. Few lawyers can identify the source of the boilerplate they are grafting onto their responses. Even if the original source document was stellar, integrity degrades as iterations multiply, mistakes are compounded, and artifacts of unique situations are carried forward to incongruent circumstances.
Ideally, the associate would start with a true template — a document that provides a comprehensive selection of general and specific objections that reflect the firm’s best practices. The associate should be able to use such a template to guide the word processing department to provide more than blank responses. Selecting from the template or a related document, the associate or partner, who might actually have the time to oversee the process once it is made less labor intensive, would, for example, direct the word processor to insert general objections 1, 2, 4, 6, 9, and 11. Similarly, the associate might instruct the word processor to insert specific objections 1, 2, 3, and 5 in response to Request for Production No. 1 and specific objections 1, 3, 5, and 8 in response to Request for Production No. 2. And so on.
Of course, drafting discovery responses should not end there. As with her expert selection of objections, the lawyer will add value in tailoring the responses to exigencies of the litigation. That is, the reliance on the process is not meant to remove the lawyer from the equation. Rather, the objective is to relieve the lawyer of costly drudgery and focus her efforts on the small but vital differences that make the case unique.
Similarly, in Audit Assignment #4, the associate modifies a settlement agreement. Again, the associate’s drafting skills are not at issue. Instead, the audit is investigating whether the firm provides effective mechanisms for tapping into institutional knowledge. Are there checklists for what should be included in settlement agreements (e.g., confidentiality, choice of law, attorneys’ fees)? If the agreement is lacking a necessary provision, does the firm have a curated repository from which the associate can select approved provisions?
Despite the snowflake-like uniqueness of every agreement, standard provisions are, as the name implies, standard. No one expects (and, indeed, no one wants) the associate to draft standard provisions from scratch. If the associate is not provided a readily available resource, he will go hunting for a prior agreement from which to copy and paste. He may very well settle on the easiest to locate, rather than the most suitable to the situation. Though there may not be any cost difference between the two methods, the deliberate guidance provided by the process-driven approach is more likely to deliver a quality outcome. Effectiveness is an even larger consideration than cost in assessing cost-effectiveness.
D. Casey Flaherty is corporate counsel for Kia Motors America. Flaherty’s opinions are his own, not those of Kia Motors. Email: CFlaherty@kiausa.com.
Reprinted with permission from the January 24 issue of Law Technology News © 2013 ALM Media Properties, LLC. Further duplication without permission is prohibited. All rights reserved