Homework

In today's classrooms, critical thinking is evaluated by homework, in-class quizzes and exams. Obviously, in-class activities have the added benefit of being monitored to ensure that the submitted work reflects the true ability of the learner. But, it is technologically possible to create an online homework system that reports a reasonably accurate assessment of the learner's ability. The key is to randomly assign similar questions such that no two students have the same problem set and that any help obtained on a problem will require the "helper" to demonstrate their thinking process from start to finish. Even in algorithmically generated questions, this is not the case as "short-cuts" are possible to just "get the answer." Chem21Labs' homework problems are generated from "problem groups" (a single problem group contains 5 - 25 "similar" questions) that covers the major topics of each chapter. Assume there are 5 problem groups containing 10 problems (a-j) in each group (50 problems in the question pool).  Student 1 has problems 1b,2g,3j,4a,5c  and their  study partner has 1c,2e,3f,4j,5h . This approach to generating a homework assignment guarantees that each student has a different problem set while covering the same material. While most instructors use this approach to homework, the founder of Chem21Labs implemented a novel approach to homework that was a true "game-changer" with phenomenal results. His approach is described after the last problem type.

A multiple select problem type.
  1. Multiple Choice - contains 2 to 25 choices (text or image). Incorrect choices are greyed-out so students can't re-select them on subsequent tries.




  2. Multiple Select - contains 2 to 20 choices (text or image). Students select 2 or more correct answers . . . . incorrect choices are greyed-out so students can't re-select them on subsequent tries. A sample question is shown to the right.




    3. A short answer problem type.



  3. Short Answer (or Fill-in-the-Blank) - if a student enters an incorrect answer, the program will display to the student all the characters at the beginning and end of the answer that they entered correctly. This information gives the student immediate feedback on where they made a mistake. This problem type is used extensively for naming compounds in both General and Organic Chemistry. In the example shown, SO3, Sulfur trioxide, was incorrectly named Sulfur dioxide . . . . the student receives immediate feedback that the green parts of the answer are correct and the red part is incorrect . . . .  Sulfur         ioxide .


  4. Chem21Draw - students draw a structure as the answer. The drawing canvas is created with HTML5 and Javascript and sits on the webpage next to the question. The incredible structure-specific feedback, drawing (and grading) of mechanism arrows, and the ability to organize a synthesis sequence can be found here.



  5. Unordered Text Answers - this problem type uses Chem21Draw to submit isomers and resonance structures where the answers can be entered in any order.
         Draw the three additional resonance structures for the cation to the right.

    6. Balanced equation display after Try 1.







  6. Balancing Chemical Equations - an equation template has been developed that contains 4 pieces of information for each chemical species . . . two required: coefficient and chemical formula . . . . two optional: charge and physical state. Instructors can easily customize this template such that the first assignment may require the student to enter just the coefficient; a second assignment may require the student to enter the chemical formula and coefficient, a third assigment may require the student to enter an ionic equation (students must supply the charges and physical states), and a fourth assigment may require the student to enter a net ionic equation.


  7. Balancing Nuclear Equations - a slightly different version of the Balanced Equation template has been developed for submitting Nuclear Equations. This problem type contains 4 pieces of information for each species: coefficient, atomic number, mass number and chemical symbol. Students receive immediate feedback on their answers and have multiple attempts to enter the correct answer.

  8. Multiple Short Answers - this problem type is used for multi-part questions like . . . .

    Provide the following information for 56Fe0.
         56 is the mass of this particular isotope.
         0 is the charge on this particular isotope.


    9. Tolerance options for numeric problems.


  9. Numeric Answers - this problem type requires a numeric answer that the computer evaluates to see if it falls within the tolerance established by the instructor.
    Tolerance types:
    • Absolute (±x)
    • Percent (±x%)
    • Significant Figure (±0-9 in the x SF)
    • Decimal Place (±0 in the x DP)
    In addition, the instructor can require the answer to be entered in Scientific notation and they can specify either Chemical Rounding or Excel Rounding for the last two tolerance types (see image).

  10. Ordering - this problem type requires the user to click / drag the displayed content to a certain order before submitting it for grading. This ordering type is primarily used to order lab procedure steps as a pre-lab assignment (see a working model here. The instructor distills the lab procedure down to a series of steps and the student uses the lab procedure to arrange the steps in the order they will be performed.

    This "ordering" problem type has also been used for problems involving Periodic Table trends (electronegativity, Ionization energies, atomic radii, metallic character, etc.) and for problems where various types of electromagnetic radiation are arranged by energy, wavelength, frequency, etc. The image below shows the feedback received after the first submission for the question . . .

    11. Arrange the following kinds of electromagnetic radiation in order of increasing wavelength.


A Novel Implementation Of Homework

Welcome to my pedagogical journey . . . . my journey ends with a " game-changing " pedagogical strategy that you will not find in any text book . . . . but here's how it began. I am Eddie Brown (Ph.D. Organic Chemistry) founder of Chem21Labs (2005) and Chemistry Professor at Lee University (1990-2021) in Cleveland Tennessee USA. I graduated from Lee University, earned my Ph.D. from the University of Tennessee and returned to Lee University (1990) where I taught General Chemistry, Organic Chemistry, Biochemistry and their associated labs. The first 2 - 3 years I was "honing my craft." The next 10 years were spent trying to find ways to get more of my students to engage in the material and to perform at higher levels academically. The workshops at the university and other professional meetings provided advise and some terrific-sounding slogans . . . .

  • Be the "sage on the stage" to motivationally "wow" them.
  • Be the "guide by the side" to motivate them to "take this journey" with me.
  • Students "don't care how much you know until they know how much you care".
  • "Great teachers inspire."
  • "Attend campus events / intramural games of your students and they will invest more in 'your class'."

To the best of my ability, I tried all the suggestions . . . . but nothing produced the results I wanted. Now, to be fair,  I'm not an "influencer"  . . . . I'm a chemistry teacher. It made me feel better about the situation when other teachers suggested that 25% of students will "get it" no matter what you do and 25% of students will "not get it" no matter what you do . . . . so teach to the middle 50%. Still others said that you can either go "a mile wide and an inch deep" or "an inch wide and a mile deep." My "acceptance" of this academic reality was short-lived . . . . I stubbornly wanted my students' learning to be " a mile wide and a mile deep ." I decided "there had to be a better way." The problem needed investigation and I decided to use a proven strategy . . . .  The Scientific Method . I am periodically stunned to hear scientists say something like "a student told me they really enjoyed "x" so I'm going to incorporate it in all of my classes because obviously it really works." The conclusion that something "works" can only be determined via the Scientific Method. Here's my approach to teaching "a mile wide and a mile deep" . . . .

  1. Question: what steps can the professor implement to improve the class' general knowledge and critical thinking skills in Organic Chemistry.
  2. Background Research: In 2004, I interviewed my top students to see what they did to make an "A" on the exam. Back then, students had an answer book for all their Organic homework problems and certain problems were "assigned" as important. My top students told me that if the Chapter 1 - 4 exam was coming up on Monday, they spent 2-3 hours reading/working problems on Chapter 1 on Wednesday, 2-3 hours on Chapter 2 on Thursday, 2-3 hours on Chapter 3 on Friday, 2-3 hours on Chapter 4 on Saturday and reviewed 2-3 hours on Sunday. I listened in stunned silence as my "best" students relayed very similar strategies . . . . what I was hearing for the first time in 15 years of teaching was that students had shown up to class for 3.5 weeks (10 hours of class time) and simply "took notes." I had incorrectly assumed that my best students didn't answer questions because they didn't want to appear smarter than the class . . . . the real truth is "they could not answer basic questions and they were not prepared to critically think" while they were in class with me . . . .  I wasn't "teaching" them .
  3. Hypothesis: devise and implement an assignable knowledge-acquring system where novices construct a database of course information similar to the expert's (my) database. If I assign this tool appropriately, students would arrive to class capable of participating in activities where this recently learned knowledge is applied in new ways (new to the student) and new connections are created between their developing memory constructs. In addition, this system would incorporate spaced-repetition assignments that maintain the database knowledge in long-term memory so that it can easily flow into working memory allowing students full access to previously learned knowledge. After class, students would use their growing schema to engage in higher level thinking contained in homework assignments . . . . but I needed to change the student perspective that homework is more than a grade, it is exam preparation - more about that in the next section.
  4. Experiment: used spaced-retrieval of course-critical facts to build memory constructs in long-term memory. Assign critical-thinking questions as homework and add importance to these questions by asking a subset (~ 10%) of these questions (exactly as they appear in the homework) on the next exam.  Wait . . . . did I read that correctly?  Why would you give students the test? I wish I didn't "have to," but here's the academic reality:
    • Students do not perceive "outside help" on "out of class" work as cheating.
      • Some students "work together" in study groups.
      • Some students "help" other students by showing them how to get the correct answer.
      • Some students "copy" other student's work.
      • Some students subscribe to websites that show them how to get the correct answer.
    • Homework grades are much higher than Exam grades . . . . not surprising since students percieve the "goal of homework assignments" is to get the best grade possible . . . . instructors see it as practice for the Exam.
    • Students blame their inate abilities (I can't change), their "exam prep" (it's too late to change) and their instructor for their poor performance on Exams . . . . all teachers hear the following after an exam:
      • "I'm not good at Chemistry."
      • "I studied the wrong things."
      • "If I studied 20 more hours, I would not have scored higher on the exam."
      • "Those questions came out of left field."
      • "The exam was too hard."
      • "The grading was too picky."

        • The "reality" of providing students with a "pool" of homework problems (typically 45-50 per chapter for General Chemistry and 60-75 per chapter for Organic Chemistry) eliminates the issues described above.  Harmful self-talk is banished. 

      • "I'm not good at Chemistry" becomes "I'm sure I can understand HOW to work these problems that will be on the exam."
      • "I studied the wrong things" becomes "Here's a list of problems I need to know HOW to work."
      • "If I studied 20 more hours, I would not have scored higher on the exam" becomes "Every extra hour I put in gives me a higher score on the exam."
      • "Those questions came out of left field" becomes "Those questions came from the exam pool."
      • "The exam was too hard" becomes "The homework was hard, but I crushed it and will do the same to the exam."
      • "The grading was too picky" becomes "I must enter the correct answer and show my work these specific questions . . . I can do that."

        • This approach has stripped the student of ALL the excuses they have used to rationalize academic deficiency. Now the student has a choice - a very simply choice -  to work or not to work . If they decide to work, the next question they must answer is "what grade do I want in this class?" I have to say that as a teacher it was such an enjoyable experience to have a student drop by my office needing help on Problems 13, 36 and 48. First, I knew that they had already successfully worked a lot of problems. Second, I would help them by asking them TRQ questions (described in #6 below) - questions that I knew they knew. The answer to these questions were recall facts that were relevant to the homework problem they were struggling with. Invariably, the student would say " Oh, Oh . . . . I can get it now! . . . . I see it! "

    • Instructors now encourage students to get "outside help" . . . . classmates, tutors, TAs and instructors . . . . on their homework problems by making learning a team effort and encouraging the whole team to be successful.
    • Instructors change the focus of the homework assignment to  THESE QUESTIONS ARE IMPORTANT AND YOU MUST KNOW HOW TO ARRIVE AT THE CORRECT ANSWER FOR THE UPCOMING EXAM.  I do suggest assigning 5% of the course grade to homework . . . . basically, it becomes a "completion grade" where you determine the date of completion. For two academic years (I didn't believe the results - remember I'm stubborn), I did not give course credit for homework. Students waited until just before the exam to work on the pool of questions, ran out of time and scored worse on the exam.

      Still not convinced about giving students a pool of exam questions? Give your expert opinion on the following situation:

      A student in your class reads Chapters 1 - 4, makes flash cards as she reads and daily uses these flash cards to move critical course material into her long-term memory. She studies the lecture notes and can work all the problems contained there and she completes and understands the homework problems. In preparation for the exam, she finds three of your old exams and works through them getting help to understand any problems that she misses. She walks into the exam confident that she can work any problem she has studied. What grade would you expect her to receive?

      The scenario just described can be "assigned" to your class. The pool of homework questions can include last year's exam questions . . . . and previous years. You can create a class of students like the one described above.
  5. Test: The American Chemical Society (ACS) Organic Chemistry Exam was used as the Spring semester Final Exam. This test was given to 637 students at Lee University (Cleveland, TN) from 1990 - 2017 by the same Organic professor.
  6. Analyze Data and Draw Conclusions: a more detailed account of the Organic ACS Exam Scores from 1990-2017 can be found here. To summarize . . . .
    • 1990-2005 ACS class results: 41 percentile . . . . . 2005-2017 ACS class results: 59.4 percentile . . . . ACS class results increased by 18.4 percentile in the test classes.
    • The number of A/A- increased from 19.4% (1990-2005) to 44.0% (2005-2017) in the test classes.
    • Attrition in the 2nd semester Organic Chemistry class went from 15% (1990-2005) to 0.8% (2005-2017) in the test classes.

    Timed / Repetitive Quizzes (TRQs) - a thorough description and explanation is found here - were used as the initial assignment to build memory constructs. Essentially TRQs are assignments where students earn course credit for studying. TRQs are an assignable flash-card system where repeated memorization assignments results in students "knowing" lots of course facts. The instructor is now instrumental in determining what facts are important, what facts are included in the TRQ assignment and what facts become part of their student's long-term memories. Once the memory constructs are formed, the student's innate mental processing architecture is engaged in problem-solving activities where this stored information is accessed, pulled into working memory, and connected to create a constantly evolving chemistry schema. When the student's intrinsic abilities fail, their schema can still be altered - by a classmate, textbook, internet, tutor, TA, or Instructor - an alteration that the learner hopes will persist at least until the next exam. With this approach, every student can be successful because their schema can be as effective as any other student's schema when used to answer in-class exam questions. How does this schema perform in novel situations? The ACS Exam Results suggest that the students' schema constructed in 2005 - 2017 is superior to those constructed in 1990 - 2005. Additionally, it seems reasonable that a superior schema would persist longer in the student's memory than an inferior schema.

    The TRQ / Exam Question Pool approach works for the following reasons:
    • Students can earn 100% of the TRQ points if they are willing to work . . . . in one of the TRQ "options" points are awarded after 75 correct answers.
    • Students can earn 100% of the exam points if they are willing to work until they can answer (and show their work for) every question in the exam pool.
    • Students have no excuse for not getting the grade in the class that they want. In the years (2005 - 2017) where this approach was used I had zero students stop by my office and ask "how can I get a better grade in this class?" It was obvious to them that my answer would be "earn all your TRQ points and then work on your exam questions until you can correctly answer any of them when you come into the exam."
    • A quick look at the student's online work allows me to catagorize students as follows:
      • Students that don't do their TRQs and then fail the exam, simply do not want to work in your class and do not care if you know that about them. There is nothing you can do to motivate someone that doesn't want to do anything.
      • Students that feel this approach is too much work . . . they do > 95% of the TRQs and about 50% - 75% of the questions in the exam pool. These students typically make B's and C's because about half of the questions on the exam are new to them.
      • Students that are "workers" - they are willing to work. They take advantage of recitations and office hours when they have questions. They appreciate the fact that the work is 100% prescribed and straightforward . . . nothing "out of left field". They do the work and perform very well on exams and in the course. It is this group of students that is"on the bubble" concerning graduate / professional school . . . . the structured TRQ / Exam Questions approach tips the scale.
      • Students that just "get it" on their own. You won't see them in office hours, but they are the top performers in the class.
    • The professor (like a coach) sets the schedule for learning new information and maintaining this information in the "learned state". I like the coach analogy - coaches "help", referees "penalize". Proir to implementing this "game plan," I was both coach and referee. It turns out that athletes can have great relationships with their coach, but mostly non-existent relationships with referees. So, the more "coach" a teacher becomes the more a student will view them as "helping and helpful."

An added pleasure of this approach is that it makes homework help sites like Chegg, StudyPool, HomeworkDoer, Yahoo Answers, DoMyHomework123, etc. unnecessary. Students can keep their money because their classmates and school resources (tutors, TAs, professors) will be there to assist them . . . . now, that's a " true guide by the side. "