April 2016: We have removed Qualtrics from our experimental workflow, and have instead chosen for a custom Drupal based survey engine. As time went by we needed more and more workarounds to get our code to run within Qualtrics (particularly, but not limited to, their JSE engine). At a certain point, it simply became more efficient to develop our own framework and work from there.
As Qualtrics seems to be the current de facto standard platform when conducting online psychological experiments and surveys, we have employed Qualtrics as the basic front end and survey flow engine for our live LiF Decoy Effect experiment.
When 95 participants were told to choose between two paper shredders that differed significantly in price and quality only 21% picked the the more expensive and efficient one (B). But that number markedly increased when a third model (C) was added that was somewhat inferior to one of the shredders (A), and inferior in every aspect to the chosen one (B). | Illustration by liravega/iStock & Tricia Seibold
Through use of some basic JavaScript glue, our LiF Qualtrics surveys bring together all of the elements we have been working on up until now:
- Test the Lock-in Feedback algorithm in simulations in R.
- Setting up and optimizing StreamingBandit, an Python Tornado based asynchronous server.
- Running Python based simulations to put LiF through its paces on StreamingBandit.
- Discussing which research areas might be most conductive to the application of LiF, decision to start on decoy effect mapping and optimization.
- Running simulations in R to ascertain that our assumptions on applying LiF to decoy optimizations were correct.
- Development of JavaScript snippets (and here I do mean basic JavaScript snippets, as that seems to be the only surefire way to make sure our scripts do not clash with Qualtrics’s troublesome new JavaScript engine) to make Qualtrics communicate to our python LiF algorithm, which is itself running on a StreamingBandit server.
- Creating a Qualtrics survey (see below) containing some previously successfully tested decoy scenarios.
When Davide, Maurits and myself have successfully tested this LiF / StreamingBandit / Qualtrics / Decoy Optimization stack on family and friends, the next step will be to run a survey on Amazon Mechanical Turk. We expect this latter survey to supply us with our first data set for an upcoming LiF paper.
[QUALTRICS EXAMPLE REMOVED]