| NOTE: If you had a poster at the workshop, and it's not listed here, please e-mail brian@tannerpages.com |
Posters
The Special Thing About Experience Is That It Has A Now
Anna KoopRecent experience can be more relevant than past experience. Most machine learning is oriented toward convergence to a single solution, which implicitly weights all experience equally. We show that it may be advantageous to overweight recent experience even in static worlds. Tracking and forgetting both give precedence to recent experience. These techniques can result in more accurate predictions if the environment is temporally coherent and approximation resources are limited. Attached Files: |
| Attachments:
experienceNowPoster.pdf
An Information Theoretic Approach for Building Approximate Predictive Models
Susanna Still, Monica Dinculescu, Doina Precup |
Learning Subjective Representations Through Dimensionality Reduction
Dana Wilkinson and Michael Bowling and Ali GhodsiThere are a variety of domains where one wishes to learn a representation of an environment defined by a stream of sensori-motor experience. In many cases, such a representation is necessary as the observational data is too plentiful to be stored in a computationally feasible way. In other words, the primary feature of a learned representation is that it must be compact---summarizing information in a way that alleviates storage demands.This admits a new way of phrasing the problem: as a variation of dimensionality reduction. There are a variety of well-studied algorithms for the dimensionality reduction problem, we argue that any of these can be useful for learning compact representations as long as additional constraints to the problem are respected---namely that the result is useful for reasoning and planning. Here, we formalize the problem of learning a subjective representation, clearly articulating solution features that are necessary for a learned representation to be ``useful''. Further, we briefly present a possible solution to the newly defined problem (Action Respecting Embedding) and demonstrate it's effectiveness. |
Social interaction through movement: concepts from perception-action interplay
Emilia I. BarakovaThe human brain has evolved for governing motor activity by transforming sensory patterns to patterns of motor coordination. Movement and action (action is understood as purposeful movement) are the primary expressions of behaviour. Tracing the evolution of species, movement takes incrasingly more complex and abstract forms. By humans, movement is grounding cognition, language, and even social interaction. Interaction through movement and its implications for creation of social agents are discussed in an attempt to outline a new design framework. After reviewing the main frameworks on perception and action and the design concepts they suggest, we choose the common coding paradigm as a basis for design of social interactions. The common coding is supported by the latest discoveries in neuroscience and experimental psychology. In particular, the discovery of the mirror neuron system in humans have given new dimension of understanding the sensorimotor system and its interaction to a complex environment, including the interactions with another agents. This design paradigm is illustrated by an experiment. |
Human State Estimation Through Learning Over Common Sense Data
William Pentney & Matthai Philipose & Jeff Bilmes & Henry KautzWe seek to tackle the problem of human state recognition, in which sensor-based observations are used to reason about the state of the generalhuman environment. Recent work [Pentney et al., 2006] has shown promise in using large publicly available hand-contributed commonsense databases as joint models that can be used to interpret day-to-day ob ject-use data. We discuss the development of a graphical model for reasoning over large amounts of commonsense information about human activity, and the use of Web-based information retrieval techniques to evaluate and enhance such information for more effective use. The large scale of this commonsense data creates issues of scale in inference over our graphical model; we present some means of efficiently performing inference over such a model. Additionally, we discuss how to improve the performance of our model through the use of learning techniques which can scale to the very large networks induced by this commonsense data. Finally, we present experiments to show how these techniques can be used to provide improved results in the prediction of everyday human state. |
State Similarity Based Approach for Improving Performance in RL
Sertan Girgin, Faruk Polat and Reda AlhajjIn most of the realistic and complex domains, the task that a reinforcement learning agent tries to solve contains various subtasks, each of which repeats many times at different regions of the state space. Although the solutions of the instances of the same or similar subtasks are almost identical, without any guidance, an agent has to learn related sub-policies independent of each other; this would cause the agent to pass through similar learning stages again and again, and as a result it will be harder to converge t to optimal behavior in reasonable time. The main reason of the problem is the lack of connections that would allow solution sharing. Based on the fact that states with similar patterns of behavior constitute the regions of state space corresponding to different instances of similar subtasks, the notion of state equivalence can be used as a mechanism to facilitate the sharing of solutions. By reflecting experience acquired on one state to all similar states, connections between similar subtasks can be established implicitly; this would reduce the repetition in learning, and consequently improve the performance. In this paper, based on the (partial) MDP homomorphism notion of Ravindran and Barto, we propose a method to identify states with similar sub-policies without requiring a model of the MDP or equivalence relations, and show how they can be integrated into the reinforcement learning framework. As the learning progresses, using the observed history of events, potentially useful policy fragments are generated and stored in a labeled tree. A metric, which is based on the number of common action-reward sequences, is devised to measure the similarity between two states, and the tree is utilized to apply the metric and determine states with similar sub-policy behavior. Eligibility requirements are imposed on the structure of the tree in order to keep its size manageable and focus on recent and frequently used sub-policies. Updates on the value function of a state are then reflected to all similar states, expanding the influence of new experiences. The proposed method can be treated as a meta-heuristic and it is possible to apply it to existing reinforcement learning algorithms. We demonstrate the effectiveness of the approach by reporting test results on sample problems. |
Five Basic Principles of Developmental Robotics
Alexander StoytchevThis paper formulates five basic principles of developmental robotics. These principles are formulated based on some of the recurring themes in the developmental learning literature and in the author's own research. The five principles follow logically from the verification principle (postulated by Richard Sutton) which is assumed to be self-evident. This paper also gives an example of how these principles can be applied to the problem of autonomous tool use in robots.Downloads: |
| Attachments:
NIPS_Workshop_2006.pdf
Neighborhood Components Analysis for Reward-Based Dimensionality Reduction
Nathan SpragueThere has been a great deal of research that attempts to explain the structure of biological receptive fields in terms of various methods for adapting basis vectors based on the statistical structure of visual input. These include principal components analysis (Hancock et al., 1992), independent components analysis (Bell & Sejnowski, 1997), non-negative matrix factorization (Lee & Seung, 1999), and predictive coding (Rao & Ballard, 1999), among others. Typically, such approaches are based purely on the structure of the visual input; there is no consideration of the role that visual information plays in the goal directed behavior of an organism. The motivation for the current work is to explore mechanisms of basis vector adaptation that are explicitly driven by the behavioral demands of a situated agent. Attached Files: |
| Attachments:
abstract.pdf
poster-2.pdf
POMDP Homomorphisms
Alicia Peregrin WolfeThe problem of finding hidden state in a POMDP and the problem of finding abstractions for MDPs are closely related. This work analyzes the connection between existing Predictive State Representation methods and homomorphic reductions of Markov Processes. We formally define a POMDP homomorphism, then extend PSR reduction methods to find POMDP homomorphisms when the original POMDP is known. The resulting methods find more compact abstract models in situations where different observations have the same meaning. Attached Files: |
1-9 of 9