The questions you ask start with a sentence that sets up a scene, like a caption to a photo. For example:
Question: A dog runs across a field as a frisbee flies through the air. The dog
Note the question includes a noun phrase that starts the next sentence. You then supply four possible completions of the second sentence, where based on common sense one of the four is clearly best. For example, a good answer to the above might be
leaps through the air and catches the frisbee in its mouth.
whereas a less good answer would be:
curls up near the fire.
Here is a complete example of the kind of question you might ask:
A crowd of runners are gathered near the starting line for a race. They
a) each wash their hands.
b) suddenly sprint backwards.
c) signal left and move into the passing lane.
d) turn toward the starting line and get set, as the race is about to start.
Built at Northwestern . Go 'cats!
Note: This question type was developed as part of the SWAG data set, see the following paper for more: Swag: A Large-Scale Adversarial Dataset for Grounded Commonsense Inference Zellers, Bisk, Schwartz, and Choi. ACL 2018.