How technology is helping culture-lovers cheat time and space

Pop music fans who never had the opportunity to attend concerts featuring their favourite musicians may soon be able to do so, even if they died many years ago, thanks to the EU-funded REVIVOS project.

The project is developing a voice synthesiser which can analyse a singer's voice and then reproduce it in a way that retains their original character and expression.

Oscar Mayor from Voctro Labs in Spain, which co-ordinates the project, says the synthesiser can even be used to have the artist sing new songs or different styles of music.

'Imagine Frank Sinatra singing a modern Jason Derulo song but with the expressive style and timbre of Frank Sinatra,’ said Mayor.

Existing voice synthesisers require actors to record hours of words to create a full collection, but these are not available for deceased musicians. The REVIVOS synthesiser can extract the information from recorded albums and can keep the expression of the artist.

So far, the REVIVOS synthesiser has been able to produce completely new songs using the voice of a jazz singer, and Mayor says they are working with record labels to get the rights to use the voices of deceased singers.

http://voctrolabs.com/audio/revivosdemos/1-singing_synth_autumn_leves_mixdown.mp3

A rendition of Autumn Leaves produced by the REVIVOS synthesiser.

Blast from the past

‘In the future we imagine mixed reality live shows where real singers share the stage with legendary singers,’ said Mayor. This would open up the possibility of hearing one of today's most popular performers singing naturally in a duet with an artist from the past, bringing the music to a new live audience.

Researchers are also working on ways to bring the past to life via smartphones. The EU-funded c-Space project has developed a prototype app that allows users to experience medieval Trento in Italy as they walk around the city.

“

‘This will change the way people explore and interact with the physical world around them.’

The c-Space researchers are capitalising on the trend of online video-sharing by taking images and video submitted by the public and producing 4D models (3D models that change over time) of particular scenes, which can then be used as the backdrop for creative content such as augmented reality.

In the Trento app, for example, the 3D image is overlaid with historical information. By viewing their smartphones through virtual reality headsets such as Google Cardboard, people using the app could view a 3D image which changes as they move around the city to give the experience of exploring Trento in a different century.

Projectors

Dr Bruno Simões from Fondazione Graphitech in Italy, the c-Space project scientific co-ordinator, said that in the future many smartphones will include small projectors, known as pico projectors, which will allow them to produce 3D images.

A pico projector could be used in a museum with the c-Space application, Dr Simões said, to give interactive 3D information about exhibits and create an immersive experience.  When several visitors are using the pico projectors on the same exhibit, the system will even allow them to interact with each other and produce even more complicated content.

‘This will change the way people explore and interact with the physical world around them,’ Dr Simões said.

Digital innovation is also coming to classical music. The EU-funded PHENICX project is developing tools to make the experience before, during and after concerts more stimulating, with the aim of bringing classical music to a wider audience.

‘Younger audiences tend to be information intensive,’ said Cynthia Liem from Delft University of Technology in the Netherlands, one of the PHENICX project partners. ‘They like to learn about background information over the timeline of the performance.’

Liem, who is also a classical musician, said that the project has been working to develop prototypes such as a tablet computer app which can give more extensive programme notes as well as a lot of extra information about the music.

An example of the visualisations from the project. Video courtesy of the PHENICX project

Recently, the system was showcased at a technology conference with images projected behind an orchestra showing the conductor's gestures, the instruments that were playing at any given moment and real-time harmony and score information.

Existing classical music lovers may find this experience distracting in the concert, so Liem says that in the future there may be special-format concerts using the technologies, targeted at users who like to see the extra information. The technology can also be used for live-streaming and with special recordings to encourage a wider audience to enjoy the music.

‘The best thing is to see people getting surprised and discovering new things about the music which they didn't notice before,’ said Liem.