This article originally appeared in The Bar Examiner print edition, Summer 2019 (Vol. 88, No. 2), pp 8–20.
The downward trend in bar passage rates that began in 2014 has been a concern to all those in the law-student-to-lawyer pipeline. While the increase in the February 2019 MBE mean score shows promise (see this issue’s Testing Column), concerted efforts at the law school level to help students succeed on the bar exam are as important as ever. The recent revision to ABA Standard 316 to tighten bar passage accreditation standards for law schools adds motivation for schools to strengthen such efforts.
In this section we share with you the experiences of five law schools that have seen their bar passage rates rise as a result of curricular changes and/or academic support measures aimed at improving bar passage outcomes for their students. The approaches of these five schools vary, but all stem from a common commitment to help students succeed on the bar exam so that they can embark on their careers.
- Florida International University College of Law’s Academic Excellence Program is built on teaching students to optimize their learning, challenging some of the mainstream methods of bar exam study.
- University of Massachusetts School of Law’s mandatory first-year Academic Skills Lab and multifaceted Bar Success Program are rooted in fostering a community for learners at the school.
- University of New Hampshire School of Law’s mandatory first-year Preliminary Bar Exam is designed to give students feedback early in their academic experience about their substantive and skills-based knowledge sets.
- University of North Carolina School of Law’s curricular changes to its 1L legal research and writing program and restructured and expanded 3L bar preparation courses create opportunities for its students to practice and fine-tune their legal writing and analytical skills, as well as their test-taking skills.
- University of Utah College of Law’s 100-100 initiative strives for 100% bar passage and professional employment, for which the college implemented a number of measures, including expanded academic support through the second and third years of law school.
We hope that these law schools’ efforts are of interest and inspiration to legal educators, as well as to those in the bar admissions realm.
The following chart indicates each of the five schools’ LSAT scores at the 25th, 50th, and 75th percentile for years 2011–2016.
Reported LSAT Score at the 75th, 50th, and 25th Percentiles, 2011–2016
|Florida International University College of Law||75th||157||157||158||158||158||157|
|Florida International University College of Law||50th||155||156||156||156||156||156|
|Florida International University College of Law||25th||152||151||151||151||151||151|
|University of Massachusetts School of Law||75th||148||148||151||152||152||152|
|University of Massachusetts School of Law||50th||144||145||148||147||148||148|
|University of Massachusetts School of Law||25th||142||143||145||145||146||146|
|University of New Hampshire School of Law||75th||158||159||159||158||159||159|
|University of New Hampshire School of Law||50th||154||156||157||155||157||157|
|University of New Hampshire School of Law||25th||151||151||153||152||153||154|
|University of North Carolina School of Law||75th||165||164||163||163||164||163|
|University of North Carolina School of Law||50th||163||162||161||161||161||161|
|University of North Carolina School of Law||25th||161||160||159||157||160||158|
|University of Utah College of Law||75th||163||162||161||161||160||161|
|University of Utah College of Law||50th||161||160||159||158||158||158|
|University of Utah College of Law||25th||157||156||155||155||154||156|
Source: Data are from the American Bar Association Section of Legal Education and Admissions to the Bar, which obtains enrollment statistics from its annual questionnaire to law schools and makes the statistics available on its website. Section of Legal Education—ABA Required Disclosures, Standard 509 Information Reports, available at http://www.abarequireddisclosures.org/ (last visited July 16, 2019).
The Science of Learning Law: Academic Support Measures at Florida International University College of Law
By Louis N. Schulze, Jr.
In 2014, Florida International University College of Law (FIU Law) began to implement new academic support measures to continue and extend the school’s commitment to our students’ law school, bar exam, and law practice success. In 2015, when the Academic Excellence Program (AEP) was fully implemented and my colleague Professor Raul Ruiz took the helm of the school’s bar support courses, our students placed first among Florida law schools on the Florida bar examination. Their 89% pass rate was 20.1% higher than the statewide average, where previously our students usually had placed about 5% above that mark. Since then, FIU Law’s students have placed first in six of the last eight exams, finishing a close second in the other two.
We have since received inquiries about the methods the College of Law implemented to achieve these results. “Did you double the number of credits devoted to bar preparation courses?” No, we actually stepped back certain measures. “Did you start tutoring students as they approached the bar exam?” No, because tutoring undermines students’ metacognitive skills. “Did you find the silver bullet of predicting bar failure?” No, because such a singular determinant likely does not exist. “Did your faculty start ‘teaching to the test’?” No, because teaching to the test is exactly the opposite of what is necessary.
This article critiques some of the mainstream methods of bar exam study and explains the counterintuitive measures we adopted to support our students’ success.
It’s Not About Teaching Them What to Know; It’s About Teaching Them How to Know
First, rather than reteaching law, giving students more credits for bar preparation, or teaching to the test, we focus on teaching students how to optimize their learning. Studies in cognitive science tell us what makes the brain work well and what actually hinders the brain in acquiring and storing knowledge. Most of this scholarship indicates that the traditional methods students use for studying law are wrong. Therefore, from orientation until the last day of classes (and after), each facet of the AEP focuses on overwriting the popular but suboptimal learning methods students were fed in secondary and postsecondary education.
For example, scores of researchers agree that rereading is one of the least effective ways to encode knowledge.1 Rereading falls directly into the trap of “the illusion of mastery”2 and renders students’ methods less efficient.3 Yet law schools and bar prep companies encourage students to reread lengthy outlines endlessly, instilling guilt in students if they are not perpetually hovering 9 to 10 inches over the open face of an outline.
Learning science informs us that there is a better way. The most effective method for promoting comprehension and encoding is retrieval practice, otherwise known as “the testing effect”4 When the learner forces her brain to recall information without cueing, the brain works harder and thus learns better. We counsel our students from day one to focus most heavily upon this retrieval practice and to spend far less time on rereading outlines. Meanwhile, students following the traditional methods spend most of their time rereading and often forgo retrieval practice until far too late. This inefficiency wastes the opportunity to employ metacognition to assess their own learning, clarify doctrine, and solidify retention.
It’s Not About Us; It’s About Them
As bar exam pass rates began to decline about five years ago, schools began to consider how faculty could solve the problem. Some schools began taking rather authoritarian control over students’ learning: reteaching doctrine, mandating certain study methods, requiring the use of one particular bar prep program, and using tutoring instead of formative assessment5 all became more mainstream. Although well-intended, these particular methods undermine students’ metacognition and self-regulated learning skills.
Our approach aims to build better learners by keeping the responsibility for learning with the learner. Rather than tutoring—which tacitly outsources the responsibility for learning to the tutor—our program works to build students’ metacognition to enable them to assess their own learning. In our AEP courses, for instance, writing assignments are important, but the self-critiques students write afterward are even more important. We often assign more points to the self-critiques than to the papers themselves, and feedback focuses more on the quality of the student’s metacognition. This way, during bar study, students need not rely on their bar preparation program’s limited or mediocre essay feedback and can instead figure out their own strengths and weaknesses.
Our program also builds students’ metacognition by facilitating the use of self-actualized formative assessment—that is, students providing assessment to themselves through self-testing. Law students are so used to conceiving of multiple-choice questions as summative assessment—that is, judging and grading one’s ability—that they flee from them in exam preparation. When they achieve an abysmal 45% on their first set of such questions, they thereafter ignore these tools until just before the exam. In so doing, they miss out not only on retrieval practice but also on a crucial metacognitive process of self-assessment that would induce gradual improvement by identifying and eliminating doctrinal weaknesses.
Instead, we train our students to understand that an early 45% score is normal, and that getting questions wrong is an opportunity to see blind spots. With those blind spots identified, students can then mend their weaknesses and gradually see those scores rise.
Building better learners also means creating self-regulated learners. Self-regulated learning (SRL) is similar to metacognition, but on the macro level. While metacognition focuses on assessing one’s own learning, SRL focuses on assessing the efficacy of the methods one used to learn. Making too many decisions for students leads them to assume that someone else is monitoring their learning. That assumption leads to externalizing learning regulation. Thus, when the bar prep company requires students to reread the same outline endlessly, students blindly obey the almighty “Learning Overlord” despite sensing the ineffectiveness of the process.
It’s Not About the Silver Bullets; It’s About Multiple Independent Variables
The Holy Grail of institutional efforts to promote bar passage seems to be the idea of finding the one or two factors that most impact bar passage. But searching for silver bullets is not a productive strategy. One school finding statistically significant evidence of the impact of one variable at that school does not necessarily mean that another school can assume the generalizability of that correlation. Focusing on just one or even a handful of independent variables is a flawed approach both for predicting bar passage and for informing curricular changes.
Legal educators should develop a better understanding of predictive analytics. A truly robust statistical analysis of students’ likelihood to pass the bar exam, such as the one my FIU Law colleague Professor Raul Ruiz conducted, must include dozens of variables. The first step is to determine which variables are positively correlated with bar passage at a particular school, realizing that not all variables have as much impact as others. (Some correlations are strong, some are moderate, and some are weak.) Then, one must develop a predictive model that accounts for the relative correlative strengths of the independent variables. In addition, law schools should make greater use of available real-time data, which has more predictive power than the LSAT, a single test that students took three years in the past. By assessing data on students’ current performance, schools can more accurately choose the appropriate supportive interventions.
Improving students’ success on the bar exam is a challenging business. Relying on lore, intuition, and old assumptions—such as tutoring, hand-holding, and teaching to the test—will not get the job done. Moving the needle requires expertise in data analysis and knowledge of the science of learning. To support students’ success in a genuine way, law schools must understand that academic and bar support is itself a discrete discipline that requires the full-time dedication of educators whose academic careers are focused on gaining the expert knowledge necessary to make a real difference in bar passage.
- See, e.g., Peter C. Brown, Henry L. Roediger III, and Mark A. McDaniel, et al., Make It Stick: The Science of Successful Learning (Harvard University Press 2014). (Go back)
- Jennifer M. Cooper & Regan Gurung, “Smarter Law Study Habits: An Empirical Analysis of Law Learning Strategies and Relationship with Law GPA,” 62 St. Louis Univ. L. J. 361, 364 (2018). (Go back)
- That is not to say that there is no place for outlines. Any good bar prep company should provide outlines to start the process of learning and to serve as a reference source. And students building their own outlines is a crucial part of any law school study plan. (Go back)
- Mark A. McDaniel, Janis L. Anderson, Mary H. Derbish, and Nova Morrisette, “Testing the Testing Effect in the Classroom,” 19 European J. of Cognitive Psych. 494 (2007). (Go back)
- Formative assessment is the evaluation of students throughout a course to monitor their learning and provide ongoing feedback. (Go back)
Louis N. Schulze, Jr., is Assistant Dean and Professor of Academic Support at Florida International University College of Law.
By Laurel A. Albin and Rebecca C. Flanagan
Over a period of three years, the University of Massachusetts School of Law (UMass Law, established in 2010 as the only public law school in Massachusetts) increased its bar passage percentage for first-time takers in Massachusetts to 92.6% in July 2018—up from 75% in July 2017, 69% in July 2016, and 60% in July 2015. Such an accomplishment was possible because UMass Law faculty, staff, and—most critically—students came together to build a community in which everyone recognizes the importance of bar passage and works together to maximize the likelihood of success for the key members of the community: the students themselves.
To create a community for learners, students need to feel supported throughout their education. At UMass Law, this support is provided by staff members who are notable for their specialized knowledge of every student and willingness to go above and beyond to help, doctrinal faculty members who pay attention to what is on the bar exam and how it is tested, and a dean who fosters a culture of achievement and success through significant investment of resources in academic support and bar preparation. This support is evidenced by the two key elements of UMass Law’s community for learners and its commitment to bar passage success: its Academic Skills Lab and a multifaceted Bar Success program initiated in 2015.
Academic Skills Lab
At UMass Law, bar preparation begins at orientation. Matriculants are introduced to the importance of rule mastery, outlining, case briefing, and analytical methodology from the very beginning of their academic careers. The bar exam is introduced at orientation as a hurdle they can and will pass. In addition, UMass Law embeds an academic skills program required of all entering students in the first-year doctrinal curriculum. In contrast to having a traditional academic support program designed only for those students at risk of academic failure after they have already demonstrated difficulty, UMass Law works with all students to maximize their academic ability. This support is more intensive for students who begin to struggle after the first semester, but support is provided for all students, even those who are doing satisfactorily in their courses.
This mandatory first-year course, Academic Skills Lab (ASL), is taught by both tenured and tenure-track professors and instructors in Academic Success, which signals to the students that skills are an essential part of their learning. ASL requires students to practice the fundamental first-year skills, such as critical reading, time management, and outlining. Throughout the course, students are taught the explicit connection between first-year academic skills, success in law school, success on the bar exam, and success in law practice.
The ASL course is part of a first-year Academic Skills curriculum designed to teach and reinforce essential skills and begin to relate those skills to classroom learning, a process that helps prepare students for bar preparation. The curriculum includes optional workshops for second-semester students, ASL II for students struggling academically, Academic Fellows who act as mentors to students in their first and second years, and programming for students as they move from law school to the bar exam.
The Bar Success Program
UMass Law’s Bar Success Program is multifaceted, incorporating three key features to support UMass Law students.
- Bar Prep Course: All students are required to successfully complete a three-credit Bar Prep Course in their last semester. The course is designed to provide students with a head start on their bar exam studying and introduce, reinforce, and build the skills needed to be successful on the bar exam. The course is designed to focus on skills that students may not have regularly practiced in their doctrinal courses, such as reading and responding to a case file on the Multistate Performance Test (MPT), as well as skills students have practiced throughout their law school careers, such as essay writing and answering multiple-choice questions.
- Student Opportunity Fund: In recognition of economic pressures new graduates face and because UMass is fortunate to attract nontraditional students, the dean has increased the endowment of the Judge William H. Carey Student Opportunity Fund, which provides bar study stipends to graduates who would otherwise have to work during the bar study period. The bar study stipends are awarded based on need, academic achievement, and availability of funds.
- Bar Enrichment Program: UMass Law has a robust and dynamic Bar Enrichment Program that provides graduates with continued support throughout their bar study period. During the summer and winter bar study periods, graduates preparing for the bar exam are provided with enrichment opportunities targeted at fostering confidence and building bar-specific thinking and writing skills. Doctrinal professors volunteer their time to hold workshops on heavily tested bar exam topics. Skill workshops for the Multistate Bar Examination (MBE) are held at least once a week, and MPT and Multistate Essay Examination (MEE) writing workshops are held at least twice a week. Bar takers receive feedback on at least one additional MEE or MPT per week to further hone their writing skills. During the 12-month bar prep process, the Director of Bar Success is available for weekly standing appointments with less confident bar takers and for unscheduled meetings as needed.
The Bar Enrichment Program has generated an unforeseen benefit: UMass Law bar takers stay near the school in order to take advantage of the additional resources. This proximity leads to more group study and allows the Director of Bar Success to reach out to almost every UMass bar taker daily, if needed. Remote bar takers benefit from weekly standing calls to ensure regular contact.
UMass Law’s Bar Enrichment Program runs in conjunction with commercial postgraduate bar preparation courses. (The workshops are offered in the afternoons, evenings, and Saturday mornings so as not to conflict with commercial bar preparation company lectures that are held in the mornings.) To that end, the school negotiated contracts with the three major commercial bar preparation companies. The contracts require bar prep companies to provide specific services at a set price, including but not limited to ample tutoring hours, a minimum number of MBE-style questions in the question bank, seven graded essays and/or MPTs, multiple assessments, and a performance rebate for bar takers who complete a certain percentage of the commercial course. Bar prep company representatives hold weekly calls with UMass Law to assess bar taker progress and performance, and UMass Law’s bar team intervenes as needed with positive and constructive feedback. After the bar exam, the commercial companies provide data on each bar taker’s commercial bar prep performance for UMass Law’s future use. The school also contracted with an MBE-only commercial provider for on-campus live lectures at both the beginning and end of the bar study period at no cost to bar takers. Bar takers report that the Bar Enrichment Program workshops are a great addition to their bar study, as they provide one-on-one real-time interaction, allowing bar takers to have their concerns addressed immediately by a trusted member of the UMass community.
Just as the practice of law is evolving, bar success is an ongoing process at UMass Law. Critical to our efforts is student performance data. While some state boards of bar examiners provide student performance data only for unsuccessful candidates, and still others provide little to no information beyond a list of passers, we are grateful to the few state boards that provide full feedback for all bar takers, which, in addition to the overall scaled bar exam score and total scaled scores on the MBE and written components, includes MBE performance information by subject and detailed information regarding essay and MPT scores. Such information assists us in our continued efforts to support the key members of our UMass Law community. Access to data is crucial to the future success of UMass Law’s and other law schools’ bar success efforts. We would like to see a movement toward greater access to student performance data to empower law schools with the information they need to maximize bar success for their students.
Laurel A. Albin is the Director of Bar Success at the University of Massachusetts School of Law.
Rebecca C. Flanagan is Assistant Professor of Law and Director of Teaching and Learning Methods at the University of Massachusetts School of Law.
The Preliminary Bar Exam: Promoting Bar Success from the Start at University of New Hampshire School of Law
By Leah A. Plunkett
In spring 2014, as a continued commitment to implementing our strategic planning priority to lay the strongest possible foundation for our students’ bar readiness, the University of New Hampshire Franklin Pierce School of Law (UNH Law) adopted a new requirement for students to receive the J.D. degree: the Preliminary Bar.1 All first-year J.D. students must take a mini-version of the Uniform Bar Examination (UBE) in the spring of their 1L year. Our 2017 graduates, the first J.D. cohort to graduate having completed the Preliminary Bar, gave UNH Law the 15th-highest first-time bar passage rate in the country: 93.3% across 14 jurisdictions.
What Is the Preliminary Bar?
The Preliminary Bar (known on campus as the “Prelim” and consisting of a diagnostic product developed and graded by an outside bar preparation provider) totals roughly four hours of testing, encompassing roughly 80–100 multiple-choice questions, two essay questions, and one performance test. On the multiple-choice and essay questions, four doctrinal topics are fair game: Civil Procedure, Contracts, Property, and Torts. On the performance test (which clocks in at 60 minutes instead of the 90 minutes used for each of the two Multistate Performance Test items on the real UBE), any topic is within bounds and all relevant substantive law is provided to students.
The Prelim is designed to integrate a bar exam diagnostic early enough into students’ academic experience that they are able to receive actionable information about their substantive and skill-based knowledge sets. At the time they take the Prelim, students are not even one-third of the way through their J.D. experience; thus, they are not expected to be ready to pass an actual bar exam. But they are expected, with the feedback they receive from the Prelim, to be on track to take and pass an actual bar exam—on their first try—when they take it two years later.
How Does the Preliminary Bar Help Students?
Each student gets a detailed individual report of his or her performance on the Prelim, which offers data about mastery of both doctrine and skills. Faculty members receive copies of individual reports for all their advisees, and the Director of Academic Success receives copies of individual reports for all students. To the extent that a student’s individual report indicates that he or she might not be on track to pass the actual bar exam on the first try, the student receives that feedback early, with the 2L and 3L years still ahead, allowing ample time to focus on strategies to increase bar readiness. These strategies include
- doing coursework in topics and skills tested on the bar exam (for example, a student whose Prelim results show deficits in Torts might choose to take a course in Remedies);
- engaging in individual work sessions with faculty from the Academic Success Program to address subjects ranging from time management to the basics of IRAC analysis for written assessments (IRAC refers to the Issue-Rule-Application-Conclusion essay-writing framework used for essay writing and, depending on the type of work product requested by the call of the question, on the performance test; and
- trying new modes of processing material, such as different outlining techniques and doing more or different types of practice problems, as the student prepares for law school classes and exams.
How Does the Preliminary Bar Fit into Students’ Law School Experience?
In order to make the Prelim experience beneficial rather than burdensome, the faculty carefully calibrated it so that students would take the test seriously but not become overwhelmed by it or distracted from their coursework. This calibration includes running a weekly Academic Success workshop for Prelim prep that all students can attend; having the Prelim take place as classes wind up for the spring semester but before final exams begin; and letting all students taking the Prelim pick the date collectively (there is an anonymous vote between two potential Prelim dates selected by the administration, and the date with more votes is the test date). The Prelim is not an academic course; it’s a freestanding graduation requirement that appears on student transcripts as an “academic event.” Prelim results are not given a letter grade; they do not factor into GPA or graduation honors.
UNH Law does set a target score for Prelim performance; students who do not hit that target score as 1Ls are required to retake the Prelim as 2Ls to complete the overall Preliminary Bar graduation requirement. They must also complete some additional requirements to support bar readiness, including taking and passing one “essential skills class” (classes from the UNH Law curriculum that are designed to be especially supportive of bar readiness) with a C or above. However, the vast majority of 1Ls (typically over 80%) do hit that target score the first time around. And some upper-level students voluntarily retake the Prelim. After all, our students know that the more they practice, the stronger their position for the real bar exam will be. Preparation for the real bar exam is a marathon, not a sprint, as all of us who work on bar readiness know. With the Preliminary Bar Exam, our students start their training as early as possible, putting themselves in the strongest possible position to succeed on race day itself.
- For the full text of this requirement, see University of New Hampshire, Rule I: Requirements for the Juris Doctor Degree, I(B)(12), available at https://catalog.unh.edu/law/juris-doctor-academic-rules-regulations/requirements/. (Go back)
Leah A. Plunkett is the Associate Dean for Administration, Associate Professor of Legal Skills, and Director of Academic Success at UNH Franklin Pierce School of Law. She is also a faculty associate with the Berkman Klein Center for Internet & Society at Harvard University.
By OJ Salinas
The University of North Carolina School of Law (UNC) has seen an upward movement in bar passage for its first-time test takers over the last few years. UNC ranked first among all six North Carolina law schools for first-time test takers for the July 2018 and February 2019 North Carolina bar exams. UNC’s passage rate for first-time test takers exceeded the overall North Carolina state passage rate for both exams by over 14%. The performance by 2018 graduates on out-of-state bar exams, including the Uniform Bar Examination (UBE), has likewise exceeded state averages.
Laying a Strong Foundation with a Rigorous and Supportive 1L Legal Writing Program
In 2011, the law school faculty approved two substantial changes in the law school curriculum:
- increasing the number of credit hours for the 1L legal research and writing (LRW) program
- having full-time faculty members who specialize in legal writing (rather than adjuncts) teach the 1L LRW classes.
Since making these curricular and staffing changes, we have seen a steady improvement in how our students perform on the essay portion of the bar exam, as well as on the Multistate Performance Test (MPT).1
UNC’s 1L LRW program trains students to organize and write legal analysis in ways that law-trained readers expect and value. The program follows the growth mindset theory developed by Stanford University psychologist Carol Dweck,2 which argues that a person’s abilities are fluid, giving everyone the potential to “change and grow through application and experience.”3 The 1L program fosters a growth mindset by challenging students to learn from instruction, practice, feedback, and self-evaluation—all strategies that students should continue to use as they prepare for the bar exam.
The curricular and staffing changes in the 1L LRW program have allowed UNC students more opportunities to practice drafting a variety of documents based on client case files similar to those that students encounter on the MPT. The changes also give students opportunities to receive extensive individual feedback on their legal writing and analysis from full-time faculty members. The increased emphasis on individual feedback and required individual student conferences with LRW faculty members create opportunities to help at-risk students early and often. The one-on-one attention that our LRW students receive during their 1L year helps with student engagement—students feel that they are cared for and seen. The individual attention also allows for early intervention, which can influence a student’s motivation to succeed in law school and on the bar exam.
Restructured and Expanded 3L Bar Preparation Courses
UNC has offered two 3L bar preparation courses for several years. Applied Legal Concepts (ALC) I and II focus on topics tested on the Multistate Bar Examination (MBE). ALC I focuses on Constitutional Law, Contracts, and Torts. ALC II focuses on Real Property, Evidence, and Criminal Law and Procedure.4 The courses are each two credit hours and are graded pass/fail.
Flipped Classroom Format that Focuses More on Practice Questions
In August 2017, the law school decided to partner with a bar prep vendor to help restructure the ALC courses. Students and the administration have received the changes quite positively.
The restructured ALC courses still provide a summary of the most commonly tested substantive law on the MBE. However, students now receive much of their substantive law information at home before class. They now have access to an online early bar preparation portal that contains bar prep vendor materials similar to those they will use when studying for the bar after graduation, including access to online video lectures, handouts to complete while reviewing the video lectures, comprehensive substantive law outlines, and online MBE and essay questions.5
The restructured ALC courses allow students to become familiar with the type of bar prep vendor resources that they will be using during the summer bar preparation season. The restructured format also allows students to do more in-class practice on MBE and essay questions. This additional practice gives students more opportunities to work and engage with the substantive law, which often help students better understand the law and appreciate how that law is tested on the bar exam.
Finally, the restructuring of the courses also allows for more training in self-regulated learning. While many law schools provide bar support for their students over the summer, this bar support should be secondary to the work that each student puts in each day of the bar preparation season. Students who want to be successful on the bar exam must be able to manage their time. They must be able to self-assess whether they are studying effectively. They must be able to use their bar prep vendor resources to determine whether they need to modify their studying or seek additional resources. The new flipped nature of the ALC classroom encourages students to appreciate self-regulated learning strategies and apply them when they study for the bar exam.
Expanded Enrollment in the 3L Bar Prep Courses
In addition to restructuring how the ALC courses are taught, the law school decided to expand enrollment in the courses to accommodate the strong interest that our students have in early bar preparation. Enrollment in ALC increased by 38% from the 2016 academic year to the 2017 academic year.6 Enrollment increased by 119% from the 2017 academic year to the 2018 academic year.7 Enrollment for the 2019 academic year has already matched the 2018 level.
The law school decided to expand enrollment in ALC after recognizing that there were many members of the 3L class that were genuinely concerned about the bar exam. We learned that this concern remained constant despite law school GPA and stemmed from a variety of causes: tension relating to multiple-choice exams, lack of confidence in the ability to digest and learn the overwhelming amount of material needed to pass the closed-book bar exam,8 and the uncertainty associated with North Carolina’s transition to the UBE.
The expanded and restructured ALC courses now provide the law school an opportunity to assist a large percentage of the 3L class using relatively few financial resources. The increased enrollment in ALC has also allowed the law school to reach more students concerned about failing the bar exam without the stigma associated with taking an “academic support” class. More students can get a taste of what bar preparation looks like, review some of the most commonly tested areas of law, and begin practicing MBE and essay questions during their 3L year.
Finally, we have found that it helps empower students, particularly those who may have struggled academically in law school, to see a variety of students interested in early preparation for the bar exam. Students are less likely to feel isolated and discouraged when they see that so many students have similar (and legitimate) fears about the bar exam.
UNC continues to focus on the personal and professional success of its students. We appreciate that our curricular changes created opportunities for our students to practice and enhance their legal writing, analytical, and test-taking skills. We hope that the changes that the law school has implemented continue to produce positive bar exam results for our law school and a positive experience for our students.
- North Carolina began administering the MPT in February 2019 upon its first administration of the UBE. Recent UNC graduates who have taken the UBE in other states have generally performed well on the MPT. (Go back)
- Carol S. Dweck, Mindset: The New Psychology of Success (2006). (Go back)
- Id. at 7. (Go back)
- When the faculty approved the ALC courses, Civil Procedure was not tested on the MBE. The ALC courses are therefore not designed to cover Civil Procedure. Nonetheless, we have tried to provide a quick overview of some basic Civil Procedure law in ALC. (Go back)
- Although the ALC courses are primarily focused on the MBE, we also have the students answer essay questions that focus on MBE law. So, for example, if we are reviewing negligence law for the MBE, we might also have the students answer a negligence essay question. The essay questions help us to review some of the MBE substantive law, while allowing us to enhance writing skills like legal analysis and organization. (Go back)
- Twenty-one students were enrolled in ALC I during the Fall 2016 semester, and 32 students were enrolled in ALC II during the Spring 2017 semester. Twenty-nine students were enrolled in ALC I during the Fall 2017 semester, and 44 students were enrolled in ALC II during the Spring 2018 semester. (Go back)
- Eighty students were enrolled in each semester of ALC during the 2018–2019 academic year. (Go back)
- Because many of our law school exams are open-book/open-notes, a majority of our students have not had the chance to develop or test their memorization skills on final exams, and they appreciate the mnemonic and other test-taking strategies and tips that are covered in the ALC courses. (Go back)
OJ Salinas is Clinical Associate Professor of Law and Director of Academic Excellence at the University of North Carolina School of Law.
By Robert W. Adler
In 2015, the University of Utah’s S.J. Quinney College of Law moved into a new building. The state-of-the-art building was designed for student success, featuring advanced educational technology and room for collaborative as well as classroom learning. At the building’s grand opening ceremony, however, I noted that as impressive as it was, the new facility was only a tool, and only as useful as we made it. Therefore, to inspire an improved program in honor of our new building and to renew our commitment to excellent legal education and career opportunities for our students, I announced our 100-100 initiative. The initiative aspires to 100% first-time bar passage and 100% professional employment (within 10 months of graduation). Despite skepticism from some colleagues around the country about what they viewed as lofty aspirations, we rolled up our sleeves and developed concrete plans to work toward our new goals.
How Did We Implement the Initiative?
To work toward our goal of 100% first-time bar passage, we
- increased faculty time for academic support, including a new 2L writing class and a new early bar preparation class for at-risk students during spring of the 3L year;
- improved predictive assessment to identify at-risk students, based on factors that go beyond incoming indicator scores (LSAT and UGPA) to include class rank during law school, specific course performance, recommendations by faculty members and the Dean of Students, and student self-identification;
- expanded assistance to underperforming students throughout law school, encouraging faculty to work more closely and individually with students who struggle in their courses;
- encouraged all faculty members to expand formative assessments (whereby students are evaluated throughout a course with the goal of monitoring their learning and providing ongoing feedback); and
- ensured that our curriculum covered key bar exam topics without “teaching to the test,” in part by encouraging faculty who teach basic 1L and other bar courses (Evidence, Business Organizations, etc.) to cover all of the topics most likely to be tested on the bar exam.
To work toward our goal of 100% professional employment, we
- expanded alumni outreach—for example, by establishing alumni networks throughout the country to help our students and recent graduates meet attorneys and other professionals in their areas of interest;
- improved individual student employment counseling by involving academic support faculty with our Career Development Office to ensure that struggling students get more individualized counseling;
- used our new technology for remote interviews;
- broadened outreach to a wider range of employers in traditional and nontraditional law job sectors; and
- expanded training in areas such as business literacy and entrepreneurship.
What Are the Results?
We have made steady and significant progress toward the 100-100 goals. Our bar passage rates have improved every year since we announced the initiative, with first-time passage rates on the Utah bar examination (counting both the February and July exams) rising from about 81% in 2014 to more than 87% in 2017. In July 2018, despite the national decline in average Multistate Bar Examination (MBE) scores (to 139.5), our MBE average was 147.1 and our Utah bar exam pass rate rose to 90%.1
Our professional employment rate has also improved, from 90% for the class of 2014 to 92% for the class of 2015, 95% for the class of 2017, and 97% for the class of 2018. Indeed, the class of 2017 tied for 24th in the country for J.D.-required plus J.D.-advantage jobs.2 Thus far, the class of 2019 is outperforming the class of 2018 in at-graduation full-time employment, suggesting that they may come even closer to the 100% target. (Notably, our students of color in the class of 2017 were the first subgroup of our students to achieve 100% professional employment.)
To What Do We Attribute the Success of the Initiative?
First, we built on existing strengths. We already had one of the best student-to-faculty ratios in the country. We had small sections with formative assessments in substantive courses for every 1L student, a robust and personalized academic support program for 1Ls, an intensive and superbly designed and taught six-credit legal methods course (legal writing, analysis, and introduction to other legal skills), and voluntary bar tutoring for students determined to be at risk based on academic performance.
Our employment rates have consistently outperformed national averages, which we attribute to our strong legal writing and skills programs, extensive externship and other live-client training opportunities, a dedicated career development office, and a mentoring program that pairs students with alumni based on desired practice areas.
Second, we have two secret weapons: our people and our size. Two of our faculty members pioneered a highly successful summer bar tutoring program many years ago. Participating students pass the bar on their first attempt at a much higher rate than predicted by their academic performance. We improved this effort as part of our 100-100 initiative through careful statistical analysis to identify at-risk students.
Another faculty member joined this team to expand our academic support through the second and third years. We had been providing support to 1L students, and then to graduating 3Ls whose performance suggested a high risk of bar failure. Yet we provided no formal academic support in between, which is important not only to bar passage but to overall success in law school and beyond. This new support includes formal and informal information sharing about what it takes to succeed in law school and on the bar exam and new, upper-level academic support courses for students who continue to struggle academically.
Others on our faculty have also stepped up to support these efforts. Many are adding formative assessments to their courses and using summative assessments that both test traditional law school analytical ability and are relevant to the bar exam. (Summative assessments are those whereby students are evaluated after completion of a course with the goal of judging student achievement.)
The small size of our school (currently at a total enrollment of approximately 300 students) allows our faculty and staff to have an even greater impact. Especially for students who are struggling—either academically or in their search for employment—individual counseling and attention helps tremendously. Each student might struggle with slightly different issues. Some might need help with exam anxiety. Others need more one-on-one feedback on the structure and analysis of their practice essay answers. Many need help on strategies for taking the Multistate Performance Test (including timing and organization as well as analysis) or in analyzing multiple-choice questions. The small size of our school allows us to meet these students’ needs on an individual level.
The 100-100 Initiative’s True Measure of Success
Despite this success, we have not yet attained 100% in either bar passage or professional employment, although our class of 2018 came very close with a 97% professional employment rate. Some might argue that this proves our quest to be unrealistic, but that misses the point. The sheer ambition of the 100-100 initiative has inspired innovation and improvement among faculty and students alike. Our measure of success is that every additional student who passes the bar on his or her first attempt, and every student who finds a rewarding job, is another graduate who can pursue his or her professional dreams.
The author thanks Professors Louisa Heiny, Dave Hill, and Bill Richards, and Associate Director of Career Development Jaclyn Howell, for their input on this article as well as their service to S.J. Quinney College of Law students.
- A significant majority of our students take the Utah bar exam. Because the ABA only recently required tracking of nationwide bar passage, we do not have the same longitudinal data for other states. However, our pass rates have also improved over the past two years, for which we have collected nationwide data. (Go back)
- J.D.-advantage jobs are those for which a J.D. provides an advantage in obtaining the job or may be required, but for which bar passage is not required. (Go back)
Robert W. Adler was the Jefferson B. and Rita E. Fordham Dean at the S.J. Quinney College of Law from 2013 to 2019. He recently stepped down and is returning to the full-time teaching faculty.
Contact us to request a pdf file of the original article as it appeared in the print edition.