Mozart and the digital twin for music

Although Mozart has no living descendants, his music still has countless inheritors. The latest in the lineage — born just this summer — is a digital twin.

German tech giant Siemens conceived the prodigious clone. Last month, the company brought the system to Mozart’s hometown of Salzburg.

Located at the foot of the Alps, the scenic city is hosting the prestigious Salzburg Festival of music and theatre. Classical notes are soaring through the sultry air. They’re sweeping across outdoor stages and around opera halls. Some have even entered the digital twin.

Like every digital twin, the system is a virtual model of a real-world counterpart. But this one has an unusual focus: sound.  

The 💜 of EU tech

The latest rumblings from the EU tech scene, a story from our wise ol' founder Boris, and some questionable AI art. It's free, every week, in your inbox. Sign up now!

The technology replicates a building’s acoustics. By simulating sounds within the virtual domain, the twin reveals the sonic workings of the physical realm.

“We are around acoustics all the time, but this means we can really be in them,” Julia Müller, Salzburg Festival’s director of development, tells TNW. “It is such a different feeling.”

Siemens envisions abundant applications. Architects will design venues with peerless acoustics. Builders will construct homes with perfect soundproofing. Events will adapt convention centres for music, theatre, and dance. Conductors will customise opera halls for their music. Orchestras will rehearse in replicas of real concert halls.

But before those fantasies became reality, the digital twin was called for an audition. Salzburg provided two formidable tests: a fabled festival venue and the music of Mozart.

Digitising sound

Inaugurated in 1960, Salzburg’s Large Festival Hall is a storied setting for operas and concerts.

Built on old stables, the square auditorium has globally renowned acoustics. As the name suggests, the venue is also immense. The 100-metre stage is among the widest in the world. Around 2,200 people can attend each performance.

Their seats offer different music experiences. They also affect the music. Their materials, locations, and the people sitting on them all influence the acoustics that resound throughout the hall. So do the carpets on the floor, the musicians on the stage, and the instruments they play. 

The architecture also dramatically impacts the sound. When the building pushes echoes and reverberations into cacophonous directions, acoustic panels are installed on walls and ceilings to soothe the strains.

The digital twin had to model each of these effects. Siemens assigned the task to Simcenter, a software portfolio for developing and testing simulations.

The software first analysed data on the shape, structure, and contents of the hall. All the underlying materials were then precisely defined. Each can cause sound waves to behave differently. 

Soft curtains absorb the waves, while metal furnishings reflect them. When they bounce off concrete walls, they generate a distinct acoustic reaction. If they hit a seat, they will create another reaction.

Siemens analysed these effects through two core techniques: impulse response measurements and ray tracing. Together, they measured and simulated the flow of sound through the hall.

A virtual stage for Mozart

Engineers first installed 12 microphones around the auditorium. On the stage, they placed 11 high-quality speakers in the positions of the instruments. 

Each speaker then sent small signals around the room. For a set number of seconds, they transmitted a broad frequency band at a certain duration and precise volume.

As the signals circled the hall, microphones recorded their effects. The results were then sent for analysis.

“With that technology, we get the individual signature sound of the room,” Arnold Holler, an engagement manager at Siemens Simcenter, tells TNW.

Siemens then embedded that signature into the digital hall. 

With virtual microphones and speakers in the same positions as their physical twins, the sound tests were repeated. Computer models then examined the acoustic behaviour.

“We correlate the simulation with the measurements and map the model to reality,” Holler says. “With that, you get the digital twin. Otherwise, you just have a digital model.”.

Assured that their replicant was fully developed, the team released the virtual doppelganger. 

The finished twin replicates endless musical configurations. Users can then explore the impact on every seat in the hall.

From remote locations, they can experiment with sounds, materials, and layouts. All their tweaks could transform the acoustics. By simulating them in a digital twin, they will create evidence for decisions in the real world.

More acoustic digital twins are already in the works. But technology still has to convince sceptics. To dispel their doubts, Siemens has turned to Salzburg’s favourite son. 

Mozart enters the digital stage

Siemens built the digital twin into an XR application. Named the “Sound of Science,” the app transports us into a 3D model of Salzburg’s Large Festival Hall. We then explore the music playing in the venue, adjusting sounds and structures as we listen. 

We move instruments, expand orchestras, add acoustic panels, and listen to the effects from different seats in the venue. Every change has an audible impact on the acoustics.

Stephan Frucht, the artistic director of the Siemens arts programme, chose Mozart’s 29th Symphony for the experience. Blending transparent textures with sophisticated techniques, the music showcases the digital twin’s capabilities.

A conductor and musician in his own right, Frucht cut the track in a Berlin studio. One instrument after another was recorded onto individual files. All were integrated with the digital twin. The audio signals were then fed into the XR software.

The XR app opens a window into the potential of acoustic digital twins. It also shines a new light on Siemens.

Music’s digital future

Founded back in 1847, Siemens has a lengthy history of engineering excellence. The Munich-based firm is also Europe’s largest industrial manufacturer. But the current company has another focus: software. 

Digital twins are a central component of today’s business. They’re used to monitor and improve various industrial operations, from manufacturing plants to energy grids. 

The Salzburg system shows the twins can also optimise acoustic conditions. Frucht has even loftier ambitions for the project. He believes the music can influence industrial applications.

“After all, innovation can also come from culture,” he says.

He points to a precedent set by Salzburg’s second favourite musical son: Herbert von Karajan. A celebrated (and controversial) Austrian conductor, Karajan inaugurated the Large Festival Hall. He was also involved in developing the compact disc. One of his recordings was the first work ever pressed onto the plastic.

Legend has it that he also set the CD’s parameters. He reputedly demanded that a single disc must have space for Beethoven’s 9th Symphony. The outcome was a 74-minute capacity.

If music has a similar impact on digital twins, Mozart could have a whole new breed of descendants.