I investigated how different parts of rodent basal ganglia is engaged in the value-based reinforcement learning by recording neural activity in striatum and globus pallidus. At the first experiment, I specifically focused on the role of striatum in the temporal credit assignment. Temporal credit assignment is necessary to properly connect temporally discontiguous choices and their outcomes, because choice outcomes are often revealed after a delay in the real world. To elucidate how the striatum handles the temporal credit assignment problem, I examined neuronal activity in the dorsomedial and dorsolateral striatum (DMS and DLS, respectively) of rats performing a novel behavioral task in which a choice has to be remembered until its outcome is revealed for correct credit assignment. Choice signals appeared sequentially, initially in the DMS and then in the DLS, during the period in which the animal had to remember its choice for correct credit assignment. Action value signals were found in both structures, and were elevated further in the DMS, but not in the DLS, when the choice outcome was revealed. In a behavioral context requiring temporal credit assignment, the DMS and DLS might be sequentially engaged in transmitting choice signals, and the DLS might be in charge of updating the value of the chosen action by integrating choice, action value, and reward signals together.
In the second experiment, I focused on the role of globus pallidus. I recorded globus pallidus neurons in a dynamic foraging task on a modified figure 8-shaped maze. As the parts of the downstream structure of striatum, the globus pallidus (GP) showed similar activity with striatum in terms of choice, its outcome, and chosen value. I also founded persistent, but weak, previous choice signals. My results argue against serial processing of information from striatum to globus pallidus. Collectively, these results show that rodent striatum and globus pallidus are engaged in value-based reinforcement learning, and suggest sequential engagement of the DMS and DLS, is temporal credit assignment.
