I recently added an implementation of our Lock-in Feedback algorithm to StreamingBandit. StreamingBandit was originally developed by Maurits Kaptein and Jules Kruijswijk as

“a Python Tornado based asynchronous web server to quickly setup and evaluate solutions to contextual multi-armed bandit problems. StreamingBandit allows an administrator to create new online “experiments”, each with their own policy. It also discloses an API to evaluate the set policy in applications”

By making LiF available in StreamingBandit, are now able to test LiF in real time online scenario’s – one of the main goals of our LiF project. One of the first simulations I did was to run the attached LiF_StreamingBandit script. This Python 3.5+ script makes LiF search for a preset optimal value within a simulated data stream that is randomly sampled from a normal distribution with a dropout of 20% in ten asynchronously running parallel request loops.

As before, LiF proved to work like a charm. It approached the set optimum (here, 5.0) within two hundred iterations, withstanding noise, randomization, and irregularly returning asynchronous requests. As you can see it seems to jump from one to four, going on to five, almost immediately after our integration time of 100 iterations:

It also proved quite fast. After some optimizations to StreamingBandit (which I will submit later this month) I was able to do send up to 200 requests per second on a $8 a month Hetzner CX20 vServer running Apache, Redis, MongoDB and of course our Python Tornado based StreamingBandit server.

All of the code for the simulation, plus a current version of my StreamingBandit LiF library, are included below:

The library itself as implemented in apps/lif/lif.py:

The basic experiments are represented by the following decision, and subsequent summary step: