2
$\begingroup$

I am trying to learn reinforcement learning from an article on Wolfram.com, and after several days I still cannot quite get everything right. Now I wonder about the correctness of a piece of code given in this article. The article title is "Train an Agent in a Reinforcement Learning Environment ", with the link https://www.wolfram.com/language/12/neural-network-framework/train-an-agent-in-a-reinforcement-learning-environment.html.en?footer=lang.

In the article, after training is complete the article animates the trained policy using the following code which I wonder whether is correct (right after the sentence "Animate the environment with the trained policy net ..."):

DeviceExecute[env, "Reset"];
ListAnimate[
  Table[(DeviceExecute[env, "Step", 
     DeviceExecute[env, trained[DeviceRead[env]["ObservedState"]]]]; 
    DeviceExecute[env, "Render"]), 50]];

In this piece of code,

trained

is the trained neural network that give a policy choice between 'Left' and 'Right' given the current observed state of env. So trained[DeviceRead[env]["ObservedState"]] is equal to either 'Left' or 'Right'. My question is, shouldn't

DeviceExecute[env, "Step", 
     DeviceExecute[env, trained[DeviceRead[env]["ObservedState"]]

be simply

DeviceExecute[env, "Step", 
     trained[DeviceRead[env]["ObservedState"]]

instead?

The variable env was initialized by the code

env = DeviceOpen["SimulatedCartPole"]

and so in

DeviceExecute[env,"Step",_]  

the blank should be either 'Right' or 'Left'.

$\endgroup$
1
  • $\begingroup$ I believe you are correct. I just had the same issue myself. $\endgroup$
    – PatrickS
    Commented Apr 22, 2022 at 8:45

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.