## Artificial Intelligence & Learning

One of the most exciting new developments in interdisciplinary aspects of quantum research is the emergence of the field of quantum artificial intelligence. In this line of research we investigate the problems of learning and artificial intelligence (AI) from the perspective of quantum information and statistical physics. Aspects of AI are already influencing our everyday lives. Here we investigate the fundamental questions of the limits and potential of learning and intelligent machines, under the full scope of physical law, but also the positive influence AI will play in future quantum research.

We have developed notions of quantum and classical learning agents that operate in unknown or partially known environments, where both the environment and part of the agent may involve quantum degrees of freedom [1]. Certain schemes in quantum information processing (QIP), including e.g. the one-way quantum computer (also known as measurement-based quantum computation), can be cast in the form of an agent whose actions in a quantum environment are driven by sensory feedback. While these schemes are well understood within QIP, their formulation as an agent problem offers new perspectives and possibilities, e.g. by introducing learning rules that effectively modify the operation program through feedback from the environment. In recent work, we have shown that in some cases, quantum control allows for genuine and generic improvements in learning capabilities of agents [2]. Only one construction where improvements can be obtained is known, and understanding the extensions and the potential of our approach is one of the cutting-edge research lines we are excited about. Beyond such research, which has immediate applicability as quantum technologies emerge, a full quantum mechanical treatment of learning and intelligent agents offers many conceptual challenges. Our group investigates also such fundamental topics as, for example, the understanding of the very meaning of learning and action in the general case when the agent and environment become entangled.

Along a second line of research, we have recently introduced a model for learning based on projective simulation (PS) [3], which combines notions of simulation, memory, and random walks into a new physical framework for learning. Here, the agent is operating in a classical environment but it can use quantum mechanics to learn and process its experience. The key concept the notion of the episodic and compositional memory (ECM), consisting of a stochastic network of co-called clips, which are the units of episodic memory of the agent. If the agent encounters a new percept, this triggers a random walk through the clip network which results, through stochastic out-coupling, in some action. Through environmental feedback, the hopping probabilities of the transitions are modified. In addition, new clips can be created by random variation and composition of existing clips. Distinct from most existing models of reinforcement learning and AI, the model of projective simulation provides a straightforward route for quantization. In the model of quantum projective simulation, the deliberation dynamics of the agent corresponds to quantum random walks, allowing the agent in principle to call its episodic memory in superposition. Such quantized agents achieve quadratic improvements in deliberation speeds, and qualitatively improved behavior in so-called active learning settings [4].

We have developed notions of quantum and classical learning agents that operate in unknown or partially known environments, where both the environment and part of the agent may involve quantum degrees of freedom [1]. Certain schemes in quantum information processing (QIP), including e.g. the one-way quantum computer (also known as measurement-based quantum computation), can be cast in the form of an agent whose actions in a quantum environment are driven by sensory feedback. While these schemes are well understood within QIP, their formulation as an agent problem offers new perspectives and possibilities, e.g. by introducing learning rules that effectively modify the operation program through feedback from the environment. In recent work, we have shown that in some cases, quantum control allows for genuine and generic improvements in learning capabilities of agents [2]. Only one construction where improvements can be obtained is known, and understanding the extensions and the potential of our approach is one of the cutting-edge research lines we are excited about. Beyond such research, which has immediate applicability as quantum technologies emerge, a full quantum mechanical treatment of learning and intelligent agents offers many conceptual challenges. Our group investigates also such fundamental topics as, for example, the understanding of the very meaning of learning and action in the general case when the agent and environment become entangled.

Along a second line of research, we have recently introduced a model for learning based on projective simulation (PS) [3], which combines notions of simulation, memory, and random walks into a new physical framework for learning. Here, the agent is operating in a classical environment but it can use quantum mechanics to learn and process its experience. The key concept the notion of the episodic and compositional memory (ECM), consisting of a stochastic network of co-called clips, which are the units of episodic memory of the agent. If the agent encounters a new percept, this triggers a random walk through the clip network which results, through stochastic out-coupling, in some action. Through environmental feedback, the hopping probabilities of the transitions are modified. In addition, new clips can be created by random variation and composition of existing clips. Distinct from most existing models of reinforcement learning and AI, the model of projective simulation provides a straightforward route for quantization. In the model of quantum projective simulation, the deliberation dynamics of the agent corresponds to quantum random walks, allowing the agent in principle to call its episodic memory in superposition. Such quantized agents achieve quadratic improvements in deliberation speeds, and qualitatively improved behavior in so-called active learning settings [4].

The broad framework of projective simulation is far from fully explored and we are continuously investigating and further developing this model, both in the classical and the quantum regime. For instance, on the classical side, the PS model is being applied to problems in robotics (specifically problems of hierarchical haptic learning, where robots learn autonomously how to perform complex object manipulations), machine learning in quantum experiments (were artificial agents help design novel interesting quantum-many-body experiments in e.g. photonics), as well as the simulation of biological agents (where the PS model is used in a multi-agent setting to investigate emergent collective behavior). Furthermore, the model is also expanded on the conceptual level: aside from earlier work where the model was shown to be capable to handle problems of associative learning and generalization [5], and also self-improvement (learning how to improve its own learning parameters via so-called meta-learning) [6], the group is further pushing the envelope of the model. Recently we have begun investigating how the model can handle the most complicated learning scenarios where the correct behavior has explicit, long-range temporal correlations with the history of the actions the agent has performed — that is, the task environment has long-term memory.

Progress in the classical aspects of the model also influences our quantum research. On the quantum side, one aspect of our research focuses on this developing model for AI. Here we investigate the possibilities of using quantum information techniques and algorithms to help further improve the learning properties, and the time- and space- requirements of PS agents. A second addresses the role and function of the quantum PS in the context of the larger project: the establishing of a general framework for quantum learning agents from the perspective of quantum information. This includes both conceptual research and concrete proposals for their experimental implementation in quantum optical systems.

Progress in the classical aspects of the model also influences our quantum research. On the quantum side, one aspect of our research focuses on this developing model for AI. Here we investigate the possibilities of using quantum information techniques and algorithms to help further improve the learning properties, and the time- and space- requirements of PS agents. A second addresses the role and function of the quantum PS in the context of the larger project: the establishing of a general framework for quantum learning agents from the perspective of quantum information. This includes both conceptual research and concrete proposals for their experimental implementation in quantum optical systems.

[1] V. Dunjko, J. M. Taylor, and H. J. Briegel,
Framework for learning agents in quantum environments, e-print arXiv:1507.08482 [quant-ph] (2015).[2] V. Dunjko, J. M. Taylor, and H. J. Briegel, Quantum-Enhanced Machine Learning, Phys. Rev. Lett. 117, 130501 (2016).[3] H. J. Briegel and G. De las Cuevas, Projective simulation for artificial intelligence, Sci. Rep. 2, 400 (2012) [arXiv:1104.3787].[4] G. Paparo, V. Dunjko, A. Makmal, M. A. Martin-Delgado, and H. J. Briegel, Quantum speed-up for active learning agents, Phys. Rev. X 4, 031002 (2014) [arXiv:1401.4997].[5] A. A. Melnikov, A. Makmal, V. Dunjko, and H. J. Briegel, Projective simulation with generalization, Sci. Rep. 7, 14430 (2017) [arXiv:1504.02247].[6] A. Makmal, A. A. Melnikov, V. Dunjko, and H. J. Briegel, Meta-learning within Projective Simulation, IEEE Access 4, 2110 (2016) [arXiv:1602.08017]. |

Website administrator: N. Friis (e-mail), last update: 19 October 2016