This article originally appeared in The Bar Examiner print edition, September 2015 (Vol. 84, No. 3), pp 16–22.

By Anne F. Zinkin and John Burwell GarveyThe calls for change in legal education are not new. In the 2007 publication Best Practices for Legal Education: A Vision and a Road Map, authors Roy Stuckey and others stated that “since the 1970s, numerous groups of leaders of the legal profession and groups of distinguished lawyers, judges, and academics have studied legal education and have universally concluded that most law school graduates lack the minimum competencies required to provide effective and responsible legal services.”1 The Daniel Webster Scholar (DWS) Honors Program at the University of New Hampshire (UNH) School of Law, a collaborative effort of the New Hampshire Supreme Court, the New Hampshire Board of Bar Examiners, the New Hampshire Bar Association, and UNH School of Law, was created in 2005 as an effort in New Hampshire to close the gap between legal education and legal practice by providing a practice-based, client-oriented education.2 ­Students who participate in the program do not take the traditional bar examination but rather are evaluated for bar admission based on their performance during the two-year program—the program itself serving as a variant of the bar examination.3

The program is now in its tenth year, and its effectiveness has been independently verified by a recent study conducted by the Educating Tomorrow’s Lawyers Initiative of the Institute for the Advancement of the American Legal System (IAALS) at the University of Denver, which sought to evaluate the program and its role in developing lawyers.4 The findings of the study are published in Ahead of the Curve: Turning Law Students into Lawyers: A Study of the Daniel Webster Scholar Honors Program at the University of New Hampshire School of Law.5 This article summarizes the study and its findings.

Overview of the DWS Program

When the DWS program opened to students in January 2006 as part of a three-year pilot phase, it was limited to 15 students in each graduating class. Now, with the program having become a permanent fixture at the law school, that number has increased to 24 students in each graduating class.6 Students are selected for the program by a committee composed of professors and DWS graduates, who conduct a holistic assessment of each applicant based on the applicant’s professional, interpersonal, and aca­demic skills, with the goal of creating a balanced group from the pool of qualified applicants. Students apply in March of their first year and are selected in June, participating in the DWS program during their last two years of law school.

As stated above, the DWS program is truly a collaborative effort. The program has a Supreme Court Oversight Committee, which includes two justices of the New Hampshire Supreme Court, the dean and associate dean of UNH School of Law (New Hampshire’s only law school), and the chair and eight members of the Board of Bar Examiners. (Article co-author Anne F. Zinkin is also a member of the Committee.) The Committee meets regularly to evaluate the program and make any necessary improvements and adjustments. The eight members of the Board of Bar Examiners are involved in evaluating the students’ work and progress on an ongoing basis throughout the students’ participation in the program.

The DWS program immerses participating students in experiential learning complemented by ongoing assessment and feedback.7 Through the program, students essentially complete a two-year, instead of a two-day, bar exam in which they take special courses, complete at least six credit hours of a clinic or externship, conduct three standardized client interviews, and meet annually with an assigned bar examiner who reviews their cumulative portfolio of work each semester. (The portfolio includes papers, exams, and legal documents drafted by the student; videos of student performances in simulated settings; student self-reflective analysis; and evaluations from professors and peers.)

The six special DWS courses generally involve substantial simulation as well as formative, reflective, and summative assessments. (See the sidebar #1 for an explanation of the three different types of assessments.) Those courses are Pretrial Advocacy; Trial Advocacy; Negotiations; Business Transactions; a miniseries covering six areas, including Client Counseling, Family Law, and Conflict of Laws; and a capstone course that integrates and builds upon the skills the students have already learned through the program. The capstone course, titled Advanced Problem Solving and Client Counseling, is taken during the student’s third year of law school and integrates lessons learned throughout the DWS program, with particular emphasis on the skills and values pertaining to a lawyer’s relationship with the client, including interview skills. The additional four required courses for DWS students (in addition to the four upper-level courses required for all UNH law students) are Business Associations; Evidence; Wills, Trusts, and Estates; and Personal Income Tax.

Assessment Types

Formative Assessment

Through formative assessment, students receive frequent and constructive feedback on their performance as they advance through the courses and the program. The feedback is provided by professors, lawyers, judges, other scholars, and bar examiners. It is delivered before the student has completed the course or program; this allows the student to reflect on the feedback and apply it to future exercises.

Reflective Assessment

Reflective assessment complements formative assessment. Through frequent reflection exercises, students consider the formative feedback they have received, evaluate their own performance, contemplate what they are learning about their performance, and develop a plan to address any weaknesses. Reflection allows them to better understand the lessons they are learning, how those lessons are intended to help them improve, and how those lessons are related to the practice of law and their roles as lawyers.

Summative Assessment

At the conclusion of each course, the professor evaluates the student’s performance and progress throughout the course. These evaluations are shared with the bar examiners, who also review the student’s performance for the semester. The DWS program is unique in its use of summative assessment to evaluate student performance in the full, two-year program, in addition to student performance in each DWS required course.

Source: Institute for the Advancement of the American Legal System (IAALS), Ahead of the Curve: Turning Law Students into Lawyers: A Study of the Daniel Webster Scholar Honors Program at the University of New Hampshire School of Law 23–24 (January 2015), available at http://educatingtomorrowslawyers.du.edu/images/wygwam/pdf_resources/Ahead_of_the_Curve_Turning_Law_Students_into_Lawyers.pdf (last visited June 8, 2015).

The standardized client interview became part of the DWS curriculum in 2008. It is based upon a model from the Glasgow Graduate School of Law, which itself based the standardized client on the standardized patient model used in medical education.8 Standardized clients are actors who are trained to act as new clients and who evaluate DWS students on their interviewing skills using two criteria: 1) the student’s interpersonal and professional interaction with the client, and 2) the extent to which the student obtains relevant information during the interview. Students are evaluated on a scale of 1 to 5 according to standardized criteria designed to evaluate eight effectiveness categories. Students must receive a total of 24 to pass each interview. All interviews are videotaped for later review and reflection.

To remain eligible for the program, DWS students must achieve at least a B– in all of their DWS courses and at least a B cumulative overall grade point average. Students who successfully complete the DWS program and who pass the Multistate Professional Responsibility Examination and character and fitness check are then certified by the New Hampshire Board of Bar Examiners as having passed the New Hampshire bar exam and are admitted to the New Hampshire Bar upon graduation.9 Thus, the DWS program “replaces a two-day bar examination with a two-year course path and assessment of each student.”10

Since its inception, the DWS program has captured the attention of legal education experts.11 In 2008, Lloyd Bond, Ph.D., one of the authors of the Carnegie Foundation for the Advancement of Teaching’s report Educating Lawyers: Preparing for the Profession of Law (hereinafter the Carnegie Report),12 said that the program “fuses instruction and assessment in the most intimate and integrated way that [he has] ever seen.”13

Findings of the IAALS Study

The stated mission of the DWS program is “Making Law Students Client-Ready.” The IAALS study examined whether the program lived up to its mission. To determine whether DWS students are actually better prepared to practice law than their non-DWS counterparts, IAALS worked with an evaluation consulting firm to “conduct quantitative and qualitative analysis of existing research to evaluate outcomes of the DWS program.”14 IAALS learned that (1) in focus groups, members of the legal profession and DWS alumni said that they believe that DWS graduates “are a step ahead of new law school graduates”; (2) based upon standardized client interviews, DWS students “outperformed lawyers who had been admitted to practice within the last two years”; and (3) “[t]he only significant predictor of standardized client interview performance was whether or not the interviewer participated in the [DWS] Program.”15 Based upon its study, IAALS believes that the DWS program “is ahead of the curve in graduating new lawyers ready to venture into the profession—and others can learn from its success.”16

The specific qualitative data that IAALS examined was a non-verbatim transcript of eight focus groups that were conducted at UNH School of Law in April 2013 by Lloyd Bond, Ph.D., and William M. Sullivan, Ph.D., the co-authors of the Carnegie Report.17 Focus group participants were placed in groups based upon their roles: New Hampshire ­judges, lawyer supervisors of DWS alumni, peers of DWS alumni, DWS alumni, current DWS students, UNH School of Law administrators, UNH School of Law faculty, and members of the New Hampshire Board of Bar Examiners.18 All eight focus groups perceived DWS graduates to be “a step ahead of new law school graduates” because they “are able to hit the ground running, working with clients and taking a lead role on cases immediately.”19

Focus group participants identified four interrelated factors that drive “the accelerated competence of DWS scholars”: formative assessment, practice context (the opportunity to participate in simulations with live clients), collaboration, and personal reflection.20 Focus group participants noted that the “constant nature of the feedback” from a bar examiner, peers, professors, and the DWS program director “encouraged reflection and improvement” and was particularly valuable in enabling students to become good legal writers.21 Focus group participants opined that the “combination of formative assessment and a practice context provides a particularly strong foundation for DWS graduates.”22 Certain focus groups identified “[t]he collaborative interactions between DWS scholars . . . as another important aspect.”23 Additionally, “[a]ll eight focus groups discussed the importance of DWS scholars reflecting on their own performance.”24

The specific quantitative data that IAALS examined were standardized client interview assessments administered to 69 DWS students and to 123 non-DWS new lawyers (lawyers with two or fewer years of practice) in 2009–2012.25 The non-DWS lawyers volunteered for the study at the request of the Chief Justice of the New Hampshire Supreme Court and participated during the New Hampshire Bar Association’s practical skills course (a course that newly admitted lawyers must complete during the first two years of practice).26 Performance on the standardized client interview was measured by two factors: (1) an overall assessment by the standardized client of the participant’s performance and (2) the percentage of relevant information points that the participant learned from the standardized client during the course of the interview.27

IAALS found that “DWS scholars significantly outperform[ed] non-DWS lawyers on both the overall assessment and the percentage of relevant information learned.”28 The overall performance of the DWS students was rated an average of 3.76 out of 5 compared to the overall performance of the non-DWS lawyers, which was rated an average of 3.11.29 IAALS concluded that the difference between 3.11 and 3.76 is “large and statistically significant.”30 Similarly, DWS students, on average, learned 89% of the relevant information points compared to non-DWS lawyers who, on average, learned 69% of the relevant information points, a difference also described as “large and statistically significant.”31

Although focus group participants postulated that DWS students were more successful than their non-DWS counterparts because they are higher-­performing students, IAALS found that, at least with regard to the standardized client interviews, there was no evidence to support that hypothesis.32 IAALS discovered that “[n]either LSAT score nor class rank” was “significantly predictive” of how a participant would perform on the standardized client interview.33 IAALS found that participation in the DWS program accounted for the superior performance of the DWS students as compared to the non-DWS lawyers in the standardized client interviews.34

Suggestions for Replicating the DWS Program

In an article that we wrote in 2009, we outlined 12 suggestions for replicating the DWS program in its entirety.35 (See the sidebar #2 for these 12 ­suggestions.) We recognize, however, that replicating the DWS program is not necessarily feasible in many jurisdictions, as each state has its own unique needs and challenges and must tailor initiatives that are realistic and cost-effective for that state. In New Hampshire—because the program is a collaborative effort between the New Hampshire Supreme Court, the New Hampshire Board of Bar Examiners, the New Hampshire Bar Association, and UNH School of Law—the program has received strong volunteer ­support in several capacities from the New Hampshire Bar, active judges, clerks of court, court reporters, and lawyers—and from DWS graduates themselves. This has helped keep the costs of the program relatively modest, as has the use of adjunct faculty to teach certain courses, and the use of software to run and assess simulations and maintain online portfolios. Such an experience may not be within reach in other jurisdictions, particularly larger ones.

Twelve Suggestions for Replicating the DWS Program

  1. Draft a mission statement.
  2. Review your school’s current curriculum to assess the extent to which it addresses the goals identified in the mission statement.
  3. Consider what you would like to teach in the new program, and how you would like to integrate it into the overall curriculum.
  4. Identify all available resources in your law school, legal community, and community at large.
  5. Design your courses, intentionally weaving them together so that they create a seamless fabric.
  6. Select your faculty.
  7. Communicate clearly and constantly with the faculty as you develop the program.
  8. Communicate with the students.
  9. Create an application.
  10. Have a selection committee—not just an individual.
  11. At least initially, limit enrollment.
  12. As for the bar licensing part of this ­program, the school will need to approach the licensing entity in its state to see if there is any interest.

Note: For elaboration on each of these 12 suggestions, see John Burwell Garvey & Anne F. Zinkin, Making Law Students Client-Ready: A New Model in Legal Education, 1 Duke Forum for Law and Social Change 127–129 (2009), available at http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1477391 (last visited June 8, 2015).

IAALS observed, however, that “[f]ull-scale replication is not . . . the only way to learn from the success of the DWS program.”36 IAALS believes that “the program can be unbundled into the key elements that foster [its] success . . . —and that can foster success in courses, programs, and schools across the country.”37 The key elements that IAALS identified were (1) providing a learning environment with formative and reflective assessment in a practice-based context and (2) building collaboration between the academy and the profession.38 (See the sidebar #3 for the IAALS recommendations regarding these key elements.)

IAALS Recommendations for Implementing the Key Elements of the DWS Program

To provide a learning environment with the re­quisite assessments in a practice-based ­context, IAALS recommends that programs

  • identify learning outcomes and benchmarks;
  • identify multiple sources of feedback (professors, lawyers, judges, other students, bar examiners);
  • create simulated practice environments and involve the student in real-life practice settings;
  • build in ongoing feedback checkpoints;
  • require students to gather feedback and ­capture personal reflections in portfolios;
  • review personal reflections and provide feedback on students’ development; and
  • use the full student portfolio for summative assessment.

To build collaboration between law school academics and practicing lawyers, IAALS ­recommends that the parties

  • proactively seek out opportunities to collaborate;
  • recognize and explicitly state common goals ­related to the development of new lawyers;
  • work together to identify ways to meet these goals;
  • commit to an ongoing relationship that lasts beyond recommendations and into implementation; and
  • be willing to look beyond what is currently ­possible to what might be possible.

Source: Institute for the Advancement of the American Legal System (IAALS), Ahead of the Curve: Turning Law Students into Lawyers: A Study of the Daniel Webster Scholar Honors Program at the University of New Hampshire School of Law 23–24 (January 2015), available at http://educatingtomorrowslawyers.du.edu/images/wygwam/pdf_resources/Ahead_of_the_Curve_Turning_Law_Students_into_Lawyers.pdf (last visited June 8, 2015).

IAALS concluded its study by stating its belief that the DWS program “gives us a glimpse into what is possible tomorrow if we are willing to look beyond the limitations of today.”39 In IAALS’s view, the DWS program provides “a guide to creating robust and effective law school courses, programs, and curricula that will better prepare lawyers for the realities of today’s profession.”40 To IAALS, the DWS program has truly and verifiably positioned its scholars to be “ahead of the curve.”41

Notes

  1. Roy Stuckey and Others, Best Practices for Legal Education: A Vision and a Road Map (Clinical Legal Education Association, 2007), at 1. (Go back)
  2. Editor’s Note: The August 2010 issue of the Bar Examiner included an article written after the DWS program had completed its three-year pilot phase. That article delves into greater detail on such topics as how the program was ­created, the program requirements, and the assessment methods used. See John Burwell Garvey, New Hampshire’s Performance-Based Variant of the Bar Examination: The Daniel Webster Scholar Honors Program Moves Beyond the Pilot Phase, 79(3) The Bar Examiner 13–23 (August 2010). (Go back)
  3. In 2014, New Hampshire admissions to the bar were as follows: 168 by examination, 74 on motion, 6 by transferred UBE score, and 22 by DWS program completion. (Go back)
  4. IAALS, located at the University of Denver, “is a na­­tional, independent research center dedicated to facilitating continuous improvement and advancing excellence in the American legal system.” Institute for the Advancement of the American Legal System, Our Mission, http://iaals.du.edu/ (last visited June 8, 2015). It leverages “a unique blend of empirical and legal research, innovative solutions, broad-based collaboration, communications, and ongoing measurement in strategically selected, high-impact areas” to em­­power others “with the knowledge, models, and will to advance a more accessible, efficient, and accountable legal system.” Id. Educating Tomorrow’s Lawyers is one of IAALS’s four initiatives. It identifies “innovative models of legal education that ensure knowledgeable, ethical and practice-ready professionals.” (Go back)
  5. Institute for the Advancement of the American Legal System (IAALS), Ahead of the Curve: Turning Law Students into Lawyers: A Study of the Daniel Webster Scholar Honors Program at the University of New Hampshire School of Law (January 2015), available at http://educatingtomorrowslawyers.du.edu/images/wygwam/pdf_resources/Ahead_of_the_Curve_Turning_Law_Students_into_Lawyers.pdf (last ­visited June 8, 2015). (Go back)
  6. To date, there are 147 graduates from the program from the classes of 2008 to 2015. Forty-three students are currently enrolled for the classes of 2016 and 2017. (There are no current plans to expand enrollment beyond the maximum of 24 students.) (Go back)
  7. See University of New Hampshire School of Law, “Evaluation Proves Webster Scholars Are ‘Ahead of the Curve’” (Feb. 6, 2015). (Go back)
  8. See Karen Barton et al., Valuing What Clients Think: Standardized Clients and the Assessment of Communicative Competence, 13 Clinical L. Rev. 1, 3–5 (Fall 2006). (Go back)
  9. N.H. Sup. Ct. R. 42. (Go back)
  10. IAALS, supra note 5, at 11. (Go back)
  11. IAALS, supra note 5, Foreword by Lloyd Bond, Ph.D., and William M. Sullivan, Ph.D.; John Burwell Garvey & Anne F. Zinkin, Making Law Students Client-Ready: A New Model in Legal Education, 1 Duke Forum for Law and Social Change 126 (2009), available at http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1477391 (last visited June 8, 2015). (Go back)
  12. William M. Sullivan et al., Educating Lawyers: Preparation for the Profession of Law (The Carnegie Foundation for the Advancement of Teaching 2007). (Go back)
  13. Garvey & Zinkin, supra note 11, at 126. Most recently, the program was selected for national recognition as one of three recipients of the 2015 E. Smythe Gambrell Professionalism Award (June 5, 2015, letter to John Burwell Garvey from the American Bar Association Standing Committee on Professionalism), an award bestowed by the Standing Committee on Professionalism of the American Bar Association (ABA) to “honor excellence and innovation in professionalism programs by law schools, bar associations, professionalism commissions and other law-related organizations.” American Bar Association, Gambrell Professionalism Award.  The Gambrell Award judges and the ABA Committee found that the DWS program “represents an exemplary and extraordinarily innovative approach to preparing qualified students for a life in the law by immersing them in experiential learning and exposing them to exceptional professional development resources and role models” and “offers a proven template for attacking fundamental challenges facing legal education today by focusing intently on experiential learning.” (June 5, 2015, letter to John Burwell Garvey, supra. (Go back)
  14. IAALS, supra note 5, Executive Summary at 1. (Go back)
  15. Id. (Go back)
  16. Id. (Go back)
  17. IAALS, supra note 5, at 12, 13. (Go back)
  18. Id. at 12. (Go back)
  19. Id. at 13. (Go back)
  20. Id. at 14–16. (Go back)
  21. Id. at 15. (Go back)
  22. Id. (Go back)
  23. Id. (Go back)
  24. Id. at 16. (Go back)
  25. Id. at 12. (Go back)
  26. Id. at 17. The practical skills course focuses on New Hampshire-specific practice and procedures and basic lawyering skills and would not be expected to offer non-DWS lawyers who had taken the course an advantage over those who did not in terms of performance in the standardized client interviews. (Go back)
  27. Id. (Go back)
  28. Id. at 18. (Go back)
  29. Id. (Go back)
  30. Id. The IAALS conclusion is based upon statistical norms in the social sciences pursuant to which a difference greater than .60 is considered to be large and statistically significant. (Go back)
  31. Id. (Go back)
  32. Id. at 20, 21. (Go back)
  33. Id. at 21. (Go back)
  34. Id. at 21, 22. (Go back)
  35. Garvey & Zinkin, supra note 11, at 127–29. (Go back)
  36. IAALS, supra note 5, at 22. (Go back)
  37. Id. (Go back)
  38. Id. at 23, 24. (Go back)
  39. Id. at 25. (Go back)
  40. Id. (Go back)
  41. Id. (Go back)

Portrait photo of Anne F. ZinkinAnne F. Zinkin is senior law clerk to Chief Justice Linda S. Dalianis of the New Hampshire Supreme Court. She holds a B.A. with Honors in General Scholarship from Trinity College and a J.D. from New York University, where she was an Arthur Garfield Hays Civil Liberties Fellow. Before becoming a career law clerk, she was an employment litigator at Morrison & Foerster and Paul, Hastings, Janofsky & Walker, LLP, in San Francisco, California, as well as a legal writing and research instructor at UC Berkeley School of Law and UC Hastings College of the Law.

Portrait photo of John Burwell GarveyJohn Burwell Garvey is professor of law and director of the Daniel Webster Scholar Honors Program at the University of New Hampshire School of Law in Concord, New Hampshire. He holds an A.B. cum laude from Harvard College and a J.D. cum laude from Suffolk University. Before becoming director of the Daniel Webster Scholar Honors Program, he was a trial lawyer for more than 25 years. In addition to teaching and heading the Daniel Webster Scholar Honors Program, Garvey serves as a mediator and arbitrator. In recognition of his work, he is the 2011 recipient of the New Hampshire Bar Association’s Award for Outstanding Professionalism and the 2015 Award for Distinguished Service to the Legal Profession.

Contact us to request a pdf file of the original article as it appeared in the print edition.

  • Bar
    Bar Exam Fundamentals

    Addressing questions from conversations NCBE has had with legal educators about the bar exam.

  • 2021
    2021 Bar Admission Guide

    Comprehensive information on bar admission requirements in all US jurisdictions.

  • NextGen
    NextGen Bar Exam of the Future

    Visit the NextGen Bar Exam website for the latest news about the bar exam of the future.

  • BarNow
    BarNow Study Aids

    NCBE offers high-quality, affordable study aids in a mobile-friendly eLearning platform.

  • 2020
    2020 Year in Review

    NCBE’s annual publication highlights the work of volunteers and staff in fulfilling its mission.

  • 2020
    2020 Statistics

    Bar examination and admission statistics by jurisdiction, and national data for the MBE and MPRE.