PARL特征

• 可重复：算法可以稳定地再现许多有影响力的强化学习算法的结果


• 规模大：能够支持数千个CPU和多GPU的高性能并行训练


• 可重复使用：存储库中提供的算法可以通过定义前向网络直接适应新任务，并且将自动构建训练机制


• 可扩展：通过在框架中继承抽象类来快速构建新算法

模型

算法

智能体

示例

import parl

from parl.algorithms import DQN, DDQN



class AtariModel(parl.Model):

	"""AtariModel

	This class defines the forward part for an algorithm,

	its input is state observed on environment.

	"""

	def __init__(self, img_shape, action_dim):

		# define your layers

		self.cnn1 = layers.conv_2d(num_filters=32, filter_size=5,

			 			stride=[1, 1], padding=[2, 2], act='relu')

		...

		self.fc1 = layers.fc(action_dim)

	def value(self, img):

		# define how to estimate the Q value based on the image of atari games.

		img = img / 255.0

		l = self.cnn1(img)

		...

		Q = self.fc1(l)

		return Q

"""

three steps to build an agent

   1.  define a forward model which is critic_model is this example

   2.  a. to build a DQN algorithm, just pass the critic_model to `DQN`

       b. to build a DDQN algorithm, just replace DQN in following line with DDQN

   3.  define the I/O part in AtariAgent so that it could update the algorithm based on the interactive data

"""



model = AtariModel(img_shape=(32, 32), action_dim=4)

algorithm = DQN(model)

agent = AtariAgent(algorithm)

• Python 2.7或3.5+


• PaddlePaddle> = 1.2.1（我们尝试使存储库始终与最新版本的PaddlePaddle兼容）

NeurIPS 2018解决方案