Need more informations about lesson

[jsellam]

Hi,
Thank you for your job.
I'm working for fun with your lib, but I don't clearly understand the concept of lesson (lessonsQuantity, lessonsLength, and lessonsWithRandom) and how to use lessons callback.

Imagine I'm trying to train 50 agents to play to Flappy bird. How to setup my lessons please ?

Thank you,
Jeremie.

[Pravez]

Hello and thank you.

The idea of the lesson is that your agent is like a student in an academy. It has a teacher which regulates its learning. When your game is updating itself, the teacher is managing the agent so that he can learn properly at a certain "rate". Each lesson is finally hours of teaching (lessonsLength) where at the end the agent has "homework" (model training).

For instance with (lessonsQuantity = 50, lessonsLength = 1000 and lessonsWithRandom = 10), the teacher will regulate the agent so that it will "learn" during 50 lessons (meaning, will do 50x backpropagation to update model's weights), each lesson will be 1000 frames length (so after your world updated 1000 times, the current lesson ends, then the agent does some learning, then a new lesson starts), and during the 10 first lessons the agent will take completely random decisions (exploration/exploitation problem, with random you ensure you explored approximately every possible state of your world).

Your computer will have a hard time if you are trying to train 50 different agents at the same time, do not forget that this library is at its beginnings, without the optimal algorithms and running in Javascript, in your browser. It would be a good idea to prefer genetic algorithms (which I hope will be implemented later in this library).

Let me know if this helped

[clementpl]

Hello,

First, Thanks for your work !

I'm not an expert in reinforcement learning.
I'm actually trying to use your library correctly.

I'm wondering how do you handle simulation ending when you don't have fixed "update/step".
For example (Flappy bird);
config => lessonLength: 1000
Your bird make a bad action after 10 update and the game end.
How do you tell the teacher that the lesson is finish, and force him to train the agent ?
Should I continue replaying several games until the number of step reach the lessonLength ?

Thank you
Clement

[krumiaa]

Thanks for making this available, looks excellent. Is there any simple examples available online where you have implemented this? This concept of an academy and teachers etc. is intriguing, but the formulation is unique so it would help to have a concrete example where all the elements are interacting.