I was going to mention the very nice online reinforcement learning
simulator at Imaginary.org but I see Andreas Matt beat me to it.

Yifei Chen and I also have a reinforcement learning simulator that
offers a different set of features.  It's based on the REINFORCE.js demo
of Andrej Karpathy, but has a simplified interface and adds maze
editing, maze load/save, and stochastic actions.  You can try it out
here: https://www.cs.cmu.edu/~dst/TDDemo

I plan to write a tutorial on how to use it, but it's pretty easy to
figure out if you're already familiar with temporal difference learning
on gridworlds.  Just click on the "Run/Stop" button and you'll see.

-- Dave