has_expr: Don't give the answer! #337

Activity

1. sumedh10

   changed the title [-]Don't give the answer when student submits incorrect code[/-] [+]Don't give the answer when users submit incorrect code[/+] on Aug 8, 2018
2. filipsch commented on Aug 8, 2018

   

   filipsch

   on Aug 8, 2018

   Contributor

   More actions

   @sumedh10 Some thoughts

   I completely understand the reasoning: you don't want to give away the solution. On the other hand, we also have to make sure that people don't get stuck, thinking "WHAT DO YOU WANT FROM ME?!" when faced with a feedback message.

   When doing exactly as you suggest, I fear two things:

   • I'm pretty sure there are tons of exercises out there that are not perfectly clear in their instructions. For these exercises, this extra "Expected X, but got Y" message is actually very helpful.
   • Decreasing the guidance is going to increase the 'duped metric', or in other words the helpfulness of the feedback messages.

   I suggest a meet-in-the-middle, i.e provide guiding feedback that does not give away the answer.

   Example 1: incompatible types

   Change:

   Check your call of plt.hist(). Did you correctly specify the first argument? Expected something different.

   to:

   Check your call of plt.hist(). Did you correctly specify the first argument? You specified a list, but it should be a DataFrame.

   Example 2: incompatible lengths

   Change:

   Check your call of sns.boxplot(). Did you correctly specify argument x? Expected origin, but got carrier.

   to:

   Check your call of sns.boxplot(). Did you correctly specify argument x? You specified a pandas Series of length 50, while it should be length 100.

   There are probably other things we can check before giving away the answer. If type, length, etc match, so nothing helpful without giving away the answer, I suggest we still fall back on the current messaging we have.

   WDYT?

   👍React with 👍1Reacted by Sumedh Panchadhar
3. sumedh10 commented on Aug 8, 2018

   

   sumedh10

   on Aug 8, 2018

   Last edited by sumedh10

   Author

   More actions

   I'm pretty sure there are tons of exercises out there that are not perfectly clear in their instructions. For these exercises, this extra "Expected X, but got Y" message is actually very helpful.

   I agree with your suggested changes.
4. filipsch

   changed the title [-]Don't give the answer when users submit incorrect code[/-] [+]has_expr: Don't give the answer![/+] on Aug 9, 2018
5. filipsch commented on Aug 9, 2018

   

   filipsch

   on Aug 9, 2018

   Contributor

   More actions

   @sumedh10 I can also include an extra option in has_equal_value(), e.g. include_expect or something, that specifies whether or not to append the expected x got y append. Anyways, I won't have time for this until next week or the week after, so we can let this simmer for a while!

   👍React with 👍1Reacted by Sumedh Panchadhar
6. filipsch

   added

   enhancement

   on Aug 29, 2018