Digitalisation has literally flooded all areas of our lives in recent years. In education, at universities, in industry or in the tourism sector, digital media has become indispensable in all its forms and facets. The topic is also increasingly relevant in the cultural sector. For several years now, the Konzerthaus Berlin has been working intensively on new ways of conveying music digitally, and thanks to the use of modern technologies such as augmented and virtual reality, it is spearheading the field in Germany. In this blog post, Elena Kountidou and Annette Thoma from the Konzerthaus Berlin offer insights into their work, their approach and their convictions on how and why digital technologies must also be increasingly anchored in culture.
Immersive technologies at the Konzerthaus Berlin
The Konzerthaus Berlin has been placing an emphasis on innovative formats for years. It all began in 2015. Together with the HTW Berlin University of Applied Sciences, an initial 360° video was created with the Konzerthausorchester Berlin under its honorary conductor Iván Fischer. This production was shown in the vestibule of the Konzerthaus Berlin and was so well received that it led to a long-term cooperation with the HTW Berlin. The “Apollo” project, supported by the EU through the ERDF Structural Fund, got underway in 2016 and is still running today.
The focus of the project is enabling classical music to be experienced in an entirely new way – and not only by the general public, but also by classical music experts. The members of the interdisciplinary project chose VR and AR for this purpose to ensure a modern-day presentation using the latest technologies. The team, coming from a diverse range of fields – programmers, developers, designers, musicologists, dramaturges and Communication Department staff at the Konzerthaus Berlin – found a common language. This was not always a matter of course in what was then still quite a new interface between classical music and technology.
Programmers have to understand how the world of classical music works, musicologists have to learn what kinds of things are possible in AR or VR and what the limits of these technologies are. The mutual exchange, perspective shift and open communication have been basic requirements for the project having worked so successfully over the years.
In the following section, some of the projects introduced by the Apollo team will be described as examples of collaborative work:
Projects using augmented reality:
About a year after the project started, the Konzerthaus Berlin published the augmented reality app “Konzerthaus Plus” in 2017. The first augmentations could be found in the 2017/18 season brochure with an interactive sound map of the Baltic States, photo galleries, musical excerpts and 3D models. Digitally enhanced postcards, an orchestra puzzle and a virtual timeline followed. Probably the most extensive project, in addition to a high-quality 3D model of the Konzerthaus and five of its halls, is the “Virtual Quartet”.
Here, members of the Konzerthaus Quartet are transformed into virtual musicians. By scanning several playing cards with the app, the respective musicians appear on the smartphone screen. Users can then decide whether they want to listen to the musicians together or concentrate on a single instrument – once a playing card is moved out of the camera’s focus, not only the virtual musician but also his or her individual instrument disappears. The Virtual Quartet is particularly suitable for music lessons in schools, as well as music education in general. Users playfully learn how a string quartet is structured and what the individual instruments are called and sound like – and can decide for themselves how they would like to explore the work being played.
Projects using virtual reality:
To date, three major VR projects have been implemented as part of the Apollo project. The first was a multi-perspective 360° video with the Konzerthausorchester Berlin. The user takes a virtual seat on stage between the musicians and its honorary conductor Iván Fischer and can follow the interaction and non-verbal communication between the orchestra members from several perspectives. Each 360° sphere also has 3D sound, giving an authentic impression of the symphony being played.
The second project was a virtual tour through the Konzerthaus Berlin. By putting VR glasses on, the user learns more about the history and architecture of the building with the help of a narrator, pictures, videos and animations. The tour is based on 3D models of the exterior facade and the halls, which are already part of the AR app.
The third and most recent VR project is an interactive composition that was created with the support of #experimentedigital, a cultural initiative of the Aventis Foundation. Together with the American composer Mark Barden and the Mexican visual artist Julian Bonequi, Apollo members have created “Environments”, a musical score come to life that can be walked through and explored. The complex and highly interactive project encourages users to become composers themselves by searching out and activating individual sounds in a surreal visual world.