TRQ Research

Organic Chemistry

The American Chemical Society (ACS) Organic Exam has been given as the Spring Final for Organic Chemistry II at Lee University (Cleveland, TN) since 1991. Starting in 2005, TRQs were used in both semesters of Organic Chemistry. In the first fifteen years (1991 - 2005), the class average on the ACS Organic Exam was  41 percentile  . . . . in the next twelve years (when TRQs were used), the class average was  59.4 percentile . In addition, only thirteen students in the first fifteen years scored above the 90th percentile on this exam. From Spring 2006 - Spring 2017, forty-six students have scored above the 90th percentile.

Equally remarkable is the fact that in the past twelve years,  only three students that started Organic II in January failed to take the ACS Exam in May  - in contrast, 15% of the students in the first fifteen years withdrew from the class and didn't take the ACS Final. This longitudinal study shows that the TRQ learning method significantly helps the under-prepared students (based on a very low attrition rates and higher end-of-course grades). Additionally, TRQs provide a significant benefit to the "prepared" students as evidenced by the increase in students scoring above the 90th percentile.

Graph of American Chemical Society Organic Final scores from Lee University students from 1990 - 2005.Graph of American Chemical Society Organic Final scores from Lee University students from 2006 - 2017.

How is this possible?

It is entirely based on the learning component described by Skinner where students are asked to complete "small steps" - steps in which they will be successful - on a daily basis. After a large series of small steps, the student has traveled a vast academic distance. The Organic students at Lee University were given a few "small steps" to complete on the first day of the first semester and on most other days throughout the academic year. Students that failed the TRQ part of the class were simply not willing to "work" . . . . they refused to take a step that can be completed (and awarded full credit) in one of three ways . . . . everyone is successful.

Graph of math students in Tennessee and their corresponding math proficiency level.

Mathematics - Ocoee Middle School

In 2011 - 2012, TRQs were used by 78 students in a 6th grade math class for 20% of the instructional time - the teacher condensed her previous classroom activities into the remaining 80%. On average, the 78 students showed a  12.8 percentile increase  in the state's end-of-class exam (TCAP). The fourth row in the image (right) shows that 43.7% of the students arriving from the 5th grade were classified as  Proficient / Advanced , the remaining 56.3% were  Basic  or  Below Basic . After using the Math TRQs for their 6th grade year, 63.4% of these test students performed at the  Proficient / Advanced  level on this exam - an increase of 19.7%. As expected, the improvement in TCAP was most noticeable for the  Below Basic  (+26.5 %-ile) and  Basic  (+16.5 %-ile) math students.

Also in 2011 - 2012 and again in 2012 - 2013 at OMS, the TRQs were used in a "Response To Intervention (RTI)" class comprised of only  Basic  Math Students as determined by their previous year's TCAP score in Math. In addition to their daily 90-minute Math class, students attended the RTI class for 45 minutes every day for 1 semester where they exclusively used TRQs. There were ~ 140 students in RTI classes each semester . . . . of the 557 RTI students in 2011 - 2013, 197 students (35.4 %) moved up to the  Proficient / Advanced  level on the state's TCAP exam.

 How much fluency is too much fluency . . . . or how many TRQs are too many TRQs? 
In the first year of the study (2011 - 2012), 24 students used TRQs 90 minutes per week (20% of instructional time) as members of the 78 student 6th grade math class and 45 minutes each day for 1 semester as members of the "RTI" math class. While ~ 35% of students in the RTI cohort moved to the  Proficient / Advanced , 12/24 (50%) of the students that experienced this "double exposure" to the TRQ method moved from the  Basic  level to the  Proficient / Advanced  levels.

Mongolian Language Study

Graph of Mongolian words retained vs. Days from last studied.

In April 2014, 19 students from either an Organic Chemistry II class or an upper-level Psychology class (Learning and Cognition) were selected to participate in a research project where, on subsequent days, students were

  1. given 50 Mongolian words to study for 2 hours using their best learning method. Their ability to spell the Mongolian words when presented with the English equivalent was assessed at 2 hours, 24 hours and 1 week.
  2. given 50 different Mongolian to study for 2 hours (24 hours after the 1st 50 words) using the TRQ method. Their ability to spell the Mongolian words when presented with the English equivalent was assessed at 2 hours, 24 hours and 1 week.

The Mongolian language was chosen because none of the participants had any prior knowledge of the language. Immediately after the two hours of study, the initial assessment showed that students using the TRQ were able to correctly answer 14.5% more words than the control group. On the 24-hour and 1-week assessments, the TRQ students correctly answered 16.0% and 44.0% more words than the control group.

Note, that using the TRQ method for 2 straight hours is not the prescribed route of implementation, but we were interested in a direct comparison of the students' best learning method to the TRQ method.  The 1-week loss of 53.5% of words learned using the student's best method and 41.5% of words using the TRQ Method, shows the benefits of retrieval practice (the learning technique exclusively used in TRQs) for moving more information into long-term memory. 

Spanish Language - Medical Missions Team

Three graphs showing novice-, intermediate- and expert-level student Spanish TRQ results.

In May 2014, twenty-three university students participated in a Medical Missions trip in which International Health was a required class. One-third of the class grade was based on learning commonly used Spanish words / phrases along with Spanish medical words / phrases. Students had 26 different TRQs that contained ~ 40 questions each (more than 1000 TRQ questions in all) where they had to complete 6 sets on each of the 26 "decks" of information (156 assignments). The following results are based on a TRQ called Nouns 1 which contained 37 questions. Twenty-five questions appeared on each quiz - the student was shown the English word and they were required to spell correctly (with accent marks) the corresponding Spanish word.

From their performance on Nouns 1 SET 1, the students were divided into three groups to differentiate their learning - the novice group learned enough to play with the children, the intermediate group learned the to play with children and interact with patients at the clinic, and the expert group learned to interact with patients at the clinic and to assist in translation for non-Spanish speaking doctors on the trip.

  • Novice Group (11 students): their average score (each point in the graph represents 11 students) on the first four quizzes was less than 10 correct answers.
  • Intermediate Group (8 students): their average score (each point in the graph represents 8 students) on the first four quizzes was between 10 and 20 correct answers.
  • Expert Group (4 students): their average score (each point in the graph represents 4 students) on the first four quizzes was between 20 and 25 correct answers.

It is interesting to note that although the three groups of students start at very different levels of knowledge, after SET 2 the three groups are virtually indistinguishable (see the shaded rectangular area for each graph). The Novice Group averaged 18 quizzes to complete two TRQ Sets, the Intermediate Group averaged 13 quizzes, and the Expert Group averaged 10 quizzes.  On the next four TRQ Sets, all three groups averaged 14 quizzes.  Assuming the Quiz Points are 10, the Novice Group took 18 quizzes to earn 20 points, the Intermediate Group took 14 quizzes to earn 20 points, and the Expert Group took 10 quizzes to earn 20 points. The remaining 40 points on the Nouns 1 assignment were earned by all groups by only taking 14 additional quizzes spaced-out over four TRQ Sets and over several weeks.

In today's classroom, the teacher starts the learning process by introducing basic facts via lecture, worksheets, or collaborative groups . . . . however, these results seem to indicate that more than half of the students (the Novices) do not have the requisite knowledge to be successful participants in any of these activities. In the Novice Group, students "knew" 72% of the 37 Spanish nouns after completing the 1st TRQ Set and 92% of these nouns after the 2nd TRQ Set. Based on this study, one can reasonably expect higher levels of learning for a class when the students are required to complete at least two TRQ Sets prior to beginning the teacher-initiated learning activities.

In this study, students had to complete 6 TRQ sets for each of the 26 assignments (Nouns 1 is just 1 of the 26 assignments) while waiting a minimum of 12 hours between sets. On average, students spent ~ 30 hours (spread out over 3 weeks) taking TRQs and studying their incorrect answers. This "spaced retrieval" created and maintained memory constructs that persisted during (and after) the 3-week mission trip. To put this in perspective, the team traveling to Honduras spent 15 hours traveling from Cleveland, TN to Rio Viejo, Honduras and another 15 hours for the return trip. In the time it takes to travel to Honduras and back, these students were able to place in their short-term memory over 1000 Spanish words / phrases that they used in restaurants, churches, medical clinics, and when playing with children.  As the students used this information "in country", it's lifespan in their long-term memory increased. 


If you are interested in using TRQs in your Introductory Chemistry, General Chemistry, GOB Chemistry, Nursing Chemistry, or Organic Chemistry course, please contact support@chem21labs.com.