Robotic Musicianship Group

The Robotic Musicianship Group aims to facilitate meaningful musical interactions between humans and machines, leading to novel musical experiences and outcomes. In our research we combine computational modeling approaches for perception, interaction, and improvisation, with novel approaches for generating acoustic responses in physical and visual manners. The motivation for this work is based on the hypothesis that real-time collaboration between human and robotic players can capitalize on the combination of their unique strengths to produce new and compelling music. Our goal is to combine human qualities such musical expression and emotions with robotic traits such as powerful processing, the ability to perform sophisticated mathematical transformations, robust long-term memory, and the capacity to play accurately without practice.

Research Projects

Current

  • Travis

    Travis is a smart-phone enabled robotic musical companion that can respond to and enhance your musical experiences.

  • Shimon

    Shimon is an improvising robotic marimba player that is designed to create meaningful and inspiring musical interactions with humans, leading to novel musical experiences and outcomes.

  • Swarm Robotics

    We are developing a musical swarm robot system in which small, cell-phone based robots communicate with humans and with each other and coordinate their movement in order to explore real time algorithmic musical composition and performance.

Archive

  • Haile

    Haile is a robotic percussionist that can listen to live players, analyze their music in real-time, and use the product of this analysis to play back in an improvisational manner. 

People

Faculty

Current Students

Alumni

Publications

2012

  • Cicconet, M., Bretan, M., and Weinberg, G. (2012) "Visual cues-based anticipation for percussionist-robot interaction" In HRI 2012, 7th ACM/IEEE International Conference on Human-Robot Interaction, Boston, Massachusetts

  • Bretan, M., Cicconet, M., Nikolaidis, R., and Weinberg, G. (2012). "Developing and Composing for a Robotic Musician Using Different Modes of Interaction" In Proceedings of the 2012 International Computer Music Conference (ICMC 12), Ljubljana, Slovenia.

  • Albin, A., Weinberg, G., Egerstedt, M. (2012), “Musical Abstractions in Distributed Multi-Robot Systems,” IROS, IEEE/Robotics Society of Japan International Conference on Intelligent Robots and Systems, Vilamoura, Portugal.

  • Sun, S., Malikarjuna, T., Weinberg. G (2012), “Effect of Visual Cues in Synchronization of Rhythmic Patterns,” accepted to the 2012 International Conference of Music Perception and Cognition (ICMPC 12), Thessaloniki, Greece.

2011

  • Hoffman, G., Weinberg G. (2011), “Interactive Improvisation with a Robotic Marimba Player” Journal Autonomous Robots, Vol. 31. Springer Press.

  • Weinberg G. (2011), “Gesture-based Human-Robot Jazz Improvisation” Extended Abstract in the Proceedings of the International Conference of Machine Learning (ICML 11), Seattle, USA.

2010

  • Hoffman, G. and Weinberg, G. (2010) “Gesture-based Human-Robot Jazz Improvisation”, in Proceedings of the 2010 IEEE International Conference on Robotics and Automation (ICRA 10), Anchorage, AK.

  • Hoffman, G., Weinberg, G. (2010) “Shimon: An Interactive Improvisational Robotic Marimba Player” in Extended Abstracts Proceedings of International ACM Computer Human Interaction Conference (CHI 10), Atlanta, GA.

  • Weinberg, G., Nikolaidis, R., and Mallikurjuna, T. (2010), “A Survey of Recent Interactive Compositions for Shimon – The Perceptual and Improvisational Robotic Marimba Player” The International Conference on Intelligent Robots and Systems (IROS 2010), Taipei, Taiwan.

  • Nikolaidis, R., and Weinberg G. (2010), “Playing with the Masters: A Model for Interaction between
Robots and Music Novices” The 19th International Symposium on Robotics in Music and Art (RO-MAN 10), Viarggio, Italy.

  • Hoffman, G., and Weinberg G. (2010), “Synchronization in Human-Robot Musicianship” The 19th International Symposium on Robot and Human Interactive Communication (RO-MAN 10), Viarggio, Italy.

2009

  • Weinberg, G., Mallikarjuna, T., Ramen (2009) “Interactive Jamming with Shimon: A Social Robotic Musician” in the Proceedings of the ACM/IEEE International Conference on Human Robot Interaction, (HRI 2009) San Diego, CA, pp. 233-234.

  • Weinberg, G., Blosser B. (2009) “A Leader-Follower Turn-taking Model Incorporating Beat Detection in Musical Human-Robot Interaction” in the Proceedings of the ACM/IEEE International Conference on Human Robot Interaction, (HRI 2009) San Diego, CA.

  • Weinberg, G., Blosser B., Mallikarjuna, T., Ramen (2009) “Human-Robot Interactive Music in the Context of a Live Jam Session”, in the Proceedings of International Conference on New Instruments for Music Expression (NIME 09), Pittsburgh, PA, pp. 70-73.

2008

  • Weinberg, G. (2008) “Extending the Musical Experience – From the Digital to the Physical and Back”, in Seifert W., Hyun Kim J. and Moore A. (Eds.) Paradoxes of Interactivity – Perspectives for Media Theory, Human-Computer Interaction, and Artistic Investigations. Bielefeld, Germany: Transcript Verlag Press.

  • Weinberg G., Godfrey M., Rea, A., Rhodes, J. (2008) “A Real-Time Genetic Algorithm In Human-Robot Musical Improvisation”, Lecture Notes in Computer Science, Springer Press.

2007

  • Weinberg, G. (2007) “Musical Interactions Between Humans and Machines” in Sarkar N. (Ed.) Human-Robot Interaction. Vienna Austria: Ars Press. pp. 423-444.

  • Weinberg, G., Godfrey, M., Rae, A., Rhodes, J. “A Real-Time Genetic Algorithm in Human-Robot Musical Improvisation”, Proceedings of International Computer Music Conference (ICMC 2007), Copenhagen, Denmark, pp. 192-195.

  • Weinberg, G., Driscoll S. “The Robotic Percussionist – Bringing Interactive Computer Music into the Physical World”, in Sick. A. and Lishca C. (Eds.) Machines as Agency – Artistic perspectives. Bielefeld, Germany: Transcript Verlag Press, pp. 66-82.

  • Weinberg, G. “The Design of a Perceptual and Improvisational Robotic Marimba Player”, Proceedings of IEEE International Workshop on Robot and Human Interactive Communication (RO-MAN 2007), Jeju, Korea, pp. 132-137.

  • Weinberg, G., Driscoll S. (2007) “Introducing Pitch, Melody and Harmony into Robotic Musicianship”, Proceedings of the International Conference on New Interfaces for Musical Expression (NIME 2007), New York City, NY, pp. 228-233.

  • Weinberg G., Driscoll, S. (2007) “The Interactive Robotic Percussionist: New Developments In Form, Mechanics, Perception And Interaction Design”, Proceeding of the ACM/IEEE international conference on Human-robot interaction (HRI 2007), Arlington, VA. pp. 97-104.

2006

  • Weinberg G., Driscoll S. “Towards Robotic Musicianship” Computer Music Journal 30:4, MIT Press, pp. 28-45

  • Weinberg G., Driscoll, S., Thatcher T. “Jam ’aa – A Percussion Ensemble for Human and Robotic Players” ACM International Conference on Computer Graphics and Interactive Techniques (SIGGRAPH 2006), Boston, MA.

  • Weinberg G., Driscoll S. “Robot-Human Interaction with an Anthropomorphic Percussionist” Proceedings of International ACM Computer Human Interaction Conference (CHI 2006). Montréal, Canada, pp. 1229 – 1232

2005

  • Weinberg G., Driscoll S., Parry M. “Haile – An Interactive Robotic Percussionist” Proceedings of International Computer Music Conference (ICMC 2005). Barcelona, Spain, pp. 622-625.

  • Weinberg G., Driscoll S., Parry M. “Musical Interactions with a Perceptual Robotic Percussionist” Proceedings of IEEE International Workshop on Robot and Human Interactive Communication (RO-MAN 2005) Nashville, TN, pp. 456-461.

Press