Physical Audio for Virtual Worlds

news  |  intro  |  demos read download contact main 





10 Dec 12

The project has been overhauled and is now hosted on Google Code @ phya-code 


20 Nov 09

Sam Bayless has made a complete Java port of Phya, JPhya, available from Google Code at code.google.com/p/jphya/


10 Oct 09

Two synthesized sound samples have been added to illustrate an efficient approach to non-linear resonance described at the ICE-GIC09 conference in London. Details to appear in an article.


24 March 09

Demos tweaked, with a few code improvements.




11 Feb 09

New demos added, including general improvements and loose surfaces based on synthesis and sample processing. Demonstrated at AES35, read more in the proceedings here.

Also introduced in the proceedings. VFoley is an Phya authoring application being developed with Gavin Bell and with support from the Technology Strategy Board. It allows physical scenes to be imported and enabled with Phya properties.  All Phya parameters can be interactively edited while trying out the behaviour of objects in real-time. As well as speeding development, this allows non-programmer sound designers to author Phya objects. Here is a snapshot of how it is looking:

VFoley image


9 June 08

Links to recent talk and papers added. Also an audio sample of synthesized 'loose particle surfaces' demonstrated at NIME08.


28 May 08

Phya code released under the ZLib license.
Also included is a new demo for the Bullet dynamics engine.



The precursor to Phya used a physics engine developed by Mathengine. People who left this company when it finished went on to develop the Ageia engine, now part of Nvidia. This engine is in some ways superior to Bullet for generating contact data for audio. Demos can be downloaded below and include a deformable teapot whose resonance parameters are controlled by the evolving deformable state.




Introduction

Audio for virtual worlds is often generated using simple sample-based techniques. These leave much to be desired in terms of sound realism, especially where the sound is closely linked with visual cues. There has been much research into modelling natural sounds, but this has not yet developed into a comprehensive methodology for producing modelled audio content in virtual worlds. Physics engines are now routinely used to interactively simulate the motion of rigid bodies, deformable bodies, flexible surfaces and liquids. This sophistocation only highlights the relative inadequacy of conventional audio techniques.

The Phya project provides a lightweight C++ library and tools to facilitate the addition of modelled audio into virtual worlds, using a physics engine to provide macro-dynamic information about contacts and impacts. The project also includes an ongoing effort to develop audio models. The aim is to generate a practical, flexible and efficient system that can be adapted to a wide range of scenarios, while making consistent compromises. Once object audio properties and their links to physical objects are specified, the system can generate audio without further intervention.

The properties describing the sound objects can be extracted from real recordings using analysis tools, a process sometimes called physical sampling. For instance a recording of an oil drum being hit can be analyzed, then used in a world where an oil drum was being rolled and hit. Instead of playing back that same sample again and again, we hear the variation in collision sound that matches its detailed motion. Another advantage, is that the memory footprint for the physical sample, is a small fraction of one short audio sample. Physical samples can also be edited in interesting ways not possible with direct samples.

Sometimes modelled or generative audio is criticised for costing too much machine time. In practice rich audio can be generated at low cost, with high benefit. Traditional methods still have their place, and part of the sound design process is to find an appropriate balance. Phya automatically minimizes its use of resources, and limits can be set to prevent excessive resource useage.

The original Phya source compiles for Windows, and some examples use the Bullet dynamics engine, which is cross platform. A Java port (platform independent) has been made. System functions are wrapped to ease porting.

Phya is not directly intended for sound designers. If Phya were integrated into an interactive 3D design studio, then audio properties could be edited and trialed quickly by sound designers. Metadata containing the properties could then be loaded by the virtual world at run time. The VFoley project was created to demonstrate this concept. Further work is needed to build a studio-ready tool.


Feature summary:

  • Efficient dynamic body/surface/contact/impact/generator/resonator framework.
  • Scene management to minimize and manage cost.
  • Variety of surface models for fixed and loose surfaces
  • Resonator models, including non-linear distortions, rattling.
  • Modal analyzer, produces compact files to configure resonators.
  • Utility functions for interfacing with Physics Engines, geometry etc.
  • Look-ahead limiter class for output streams.






Phya distribution download

Phya is hosted on Google Code @phya-code  under the zlib license

https://code.google.com/p/phya-code/


Please inform us of projects using Phya, and report any issues. 

To reference Phya in an academic article please cite the an article below.

rdmg [@] dmu.ac.uk



Terms of the zlib license:

This software is provided 'as-is', without any express or implied
warranty. In no event will the authors be held liable for any damages
arising from the use of this software.

Permission is granted to anyone to use this software for any purpose,
including commercial applications, and to alter it and redistribute it
freely, subject to the following restrictions:

    1. The origin of this software must not be misrepresented; you must not
    claim that you wrote the original software. If you use this software
    in a product, an acknowledgment in the product documentation would be
    appreciated but is not required.

    2. Altered source versions must be plainly marked as such, and must not be
    misrepresented as being original software.

    3. This notice may not be removed or altered from any source
    distribution.










Demos

The executables are all for Windows.

phyaDemos090324_mp4.zip (12.0M) Movies. For Youtube version see here
phyaDemos090324.zip (5.0 M) Demos for improved Phya - loose surfaces using particle synthesis and dynamic sample processing, to create effects including water, leaves, gravel, foil. 
CcdPhysicsDemo_phya.rar (251 K) First demo using the Bullet dynamics engine. 
CcdPhysicsDemo_phya.mov (576 K) Movie.
bigGong.mp3 A nonlinear gong with upwards frequency migration.
chineseGong.mp3 A nonlinear gong with downward pitch glide.
looseSurfaces.mp3 (692 k) Preview of loose particle surface synthesis, before they were integrated into Phya.
demos.zip (563 K) Old demos using the original physics engine, featuring the deformable teapot, and spheres in spheres demos. Note frame rate not fixed.
demos.mov (3.4 M) Movie of the above demos.








References

This article provides an overview, with some details about specific processes.

Menzies, D. ‘Physically Motivated Environmental Sound Synthesis for Virtual Worlds’, EURASIP Journal on Audio, Speech, and Music Processing, December 2010.

The following article describes experiments in an approach to diffuse resonance using banks of filtered noise. Diffuse resonance using feedback systems is relatively expensive to do well and hard to control. This approach is designed to overcome this.

Menzies, D. 'Perceptual Resonators For Interactive Worlds', AES 22nd International Conference on Virtual, Synthetic and Entertainment Audio, June 2002.



For more information, report problems or to be kept informed of updates please contact -

phya [@] zenprobe.com



news  |  intro  |  source  | demos | readcontact | main