新加坡南洋理工大学Hiroshi Makino近期取得重要工作进展，他研究开发了大脑分层行为组合的算术值表示。相关研究成果2022年12月22日在线发表于《自然—神经科学》杂志上。

据介绍，从预先获得的行为习惯中创造新技能的能力是生物智能的一个标志。尽管人工智能体从过去的经验中提取可重复使用的技能，并以分层的方式重新组合它们，但大脑是否也有类似的新行为，在很大程度上是未知的。

研究人员展示了深度强化学习智能体通过添加组合构成子任务的预先学习动作-价值的表征来学习解决新的复合任务。通过在预训练期间引入随机行为，进一步增强复合任务中的学习效率。这些理论预测在小鼠中进行了实验测试，其中子任务预训练增强了复合任务的学习。在皮质范围内，双光子钙成像揭示了联合作用值的类似神经表征，当行为的可变性被放大时，学习得到改善。

总之，这些结果表明，大脑通过对预先获得的具有随机策略的行动-价值表征进行简单的算术运算来构成一种新的行为。

附：英文原文

Title: Arithmetic value representation for hierarchical behavior composition

Author: Makino, Hiroshi

Issue&Volume: 2022-12-22

Abstract: The ability to compose new skills from a preacquired behavior repertoire is a hallmark of biological intelligence. Although artificial agents extract reusable skills from past experience and recombine them in a hierarchical manner, whether the brain similarly composes a novel behavior is largely unknown. In the present study, I show that deep reinforcement learning agents learn to solve a novel composite task by additively combining representations of prelearned action values of constituent subtasks. Learning efficacy in the composite task was further augmented by the introduction of stochasticity in behavior during pretraining. These theoretical predictions were empirically tested in mice, where subtask pretraining enhanced learning of the composite task. Cortex-wide, two-photon calcium imaging revealed analogous neural representations of combined action values, with improved learning when the behavior variability was amplified. Together, these results suggest that the brain composes a novel behavior with a simple arithmetic operation of preacquired action-value representations with stochastic policies.

DOI: 10.1038/s41593-022-01211-5

Source: https://www.nature.com/articles/s41593-022-01211-5