SoundToolKit  1.5
SoundToolKit Overview
stk_main.PNG

Description

SoundToolKit is a state-of-the-art audio engine using the newest scene scanning methods. It enables creation realistic spatial sound in 3D environment, providing the best auditory experience. SoundToolKit is based on real-time simulation of acoustic phenomena that occur in real life due to geometry and scientifically measured materials used in particular scene. Thanks to advanced degree of automation, SoundToolKit allows game developers to shift their attention to more crucial matters while getting immersive, dynamic 3D sound generated out-of-the-box by a dedicated audio engine.

SoundToolKit comes with a multitude of features:

  1. Real-time audio processing based on actual game scene geometry.
  2. Real-time simulation of various acoustic phenomena:
    • Occlusion
    • Reflection
    • Diffraction
    • Diffusion
    • Distance attenuation
    • Transmission
    • Reverberation
  3. Automatic and dynamic reverberation based on provided scene geometry
  4. Different acoustic materials which boost overall sound experience
  5. 3D sound with a HRTF capability
  6. Ability to affect acoustic medium properties

On top of that SoundToolKit.Unity extends that with a set of useful features for game developers:

  1. Numerous methods of creating various types of highly customizable Sound Sources, including Spatialized Source, Ambient Source, Streaming Source
  2. Over 100 predefined types of Acoustic Materials based on scientifically measured, real-life materials, such as wood, concrete, glass etc.
  3. Dedicated GUI for easy creation of custom Acoustic Materials
  4. Extensive dynamic settings available for runtime configuration
  5. Automated process of creating Samples which are compatible with a SoundToolKit audio engine
  6. Advanced log creation, diagnostics and extensions which allow flexible and easy to use engine management, e.g enabling rendering of sound paths calculated by the engine in real-time

Basic plugin setup

The basic workflow to setup the scene utilizing SoundToolKit.Unity plugin:

  1. In order to have control over StkSettings add a SoundToolKitManager prefab to the scene - located in Assets/SoundToolKit/Plugins/AudioEngine/Prefabs
  2. Add the SoundToolKitListener component to your PlayerController or where you want the AudioListener to be placed.
  3. Create the AcousticScene by placing SoundToolKitMesh components on scene Meshes or create simplified version of visual geometry for better performance and attach the StkMesh components to it. (This is essentially optional but required for the full experience)
  4. Place materials of your choice in the SoundToolKitMesh component's Material field (this is optional as default material will be assigned otherwise)
  5. Use the menu Tools->SoundToolKit->Geometry->SynchronizeDefaultGeometry to set up StkMeshClusters properly.
  6. Add SoundSources to the scene and assign SoundToolKitSamples to them. Samples should be created automatically from AudioClips in project, you can always use the menu Tools->SoundToolKit->Samples->Synchronize to synchronize AudioClips in project with StkSamples in project.
  7. Run and you should hear sound from STK.

Components

Sound Sources

SourceComponent

This is a base class of SpatialComponent and SoundToolKitAmbientSource.

Here we can control basic features such as Mute, Volume and PlayOnAwake. IsPlaying states whether the SourceComponent will emit sound.

SpatialComponent

This is a base class of SoundToolKitSpatialSource and SoundToolKitStreamingSource. Here we can set 3D sound properties such as Attenuation, Radius, Max Distance, HRTF Spatialization and Simulation Quality.

It also has basic control features such as Play, Stop, Pause.

SoundToolKitSpatialSource

This component inherits from SpatialComponent and has List of Playbacks (SoundToolKitPLayback). Each Playback may have one sample which can be controlled by Looped and AutoPlay options.

Using the sctipting API you can access multiple methods of playing and stopping the sound coming from a given SpatialSource, such as playing a random Playback from the list, playing a Range of Playbacks from the list, stopping only a specific Playback etc.

OnDisable: Pause Sound OnEnable: Resume Previous state OnDestroy: Release resources


SoundToolKitStreamingSource

This component inherits from SpatialComponent and it uses streaming samples which can be enqueued only by code.

stk_stream.PNG

OnDisable: Pause Sound OnEnable: Resume Previous state OnDestroy: Release resources

SoundToolKitSourceSpawner

This component enables quick spawning of Sound Sources in a given location, that are supposed to play a given sample once and then be destroyed. It's perfect for example for creating sounds of bullets impacting on surfaces - it will spawn a sound of impact in location of the impact and handle the release of resources automatically after the sound has played.

If not specified otherwise, the sound is spawned at the transform of GameObject owning the component.

stk_source_spawner.PNG

OnDestroy: Destroy all SoundSources that were spawned by this component immediately to avoid memory leaks.

SoundToolKitAmbientSource

This component inherits from SourceComponent and has List of Playbacks (SoundToolKitPLayback). Each Playback may have one sample which can be controlled by Looped and AutoPlay options.

The purpose of AmbientSoundSource is to use for sound effects that should be unaffected by the acoustic geometry, such as music, general ambience track or UI sounds.

Similar to SpatialSource it facilitates a lot of different play/stop/pause methods.

stk_ambient_source.PNG

OnDisable: Pause Sound OnEnable: Resume Previous state OnDestroy: Release resources

Acoustic Scene & Geometry

SoundToolKitMesh

StkMesh should be placed on components that contain MeshFilter. It automatically converts the triangles of that Mesh into AcousticMesh used by SoundToolKit engine. StkMesh component has three fields: StkMaterial reference, defining how sound will behave on contact with a given Mesh. Enabled As Acoustic Obstacle control, allowing the mesh to become transparent for STK engine on demand. MeshCluster reference, controlling which StkMeshCluster this StkMesh is part of.

stk_mesh.PNG

OnDisable: Makes that mesh transparent acoustically for the Stk Engine (works the same as EnabledAsObstacle property) OnEnable: Set Previous state

OnDestroy: Release resources

SoundToolKitMaterial

[ScriptableObject]

StkMaterial component defines how sound will behave on contact with a StkMesh that it is assigned to based on the material definition in .json format located in StreamingAssets. Each SoundToolKitMaterial object has one corrseponding .json configuration file. If StkMaterials and .jsons get desynchronized, use menu Tools->SoundToolKit->Materials->Synchronize

There are three effects defined by the material, each in 8 octave frequency bands: Reflection Scattering Transmission

The effects can be turned on and off on a given material.

The VisualMaterial field determines how the Material will be displayed when using AcousticMeshRenderer StkExtension.

There are over 100 predefined material configuration files coming with SoundToolKit.Unity plugin. On top of that, there is a custom material editor available, which allows easy creation of user-defined material files via corresponding GUI note: custom material creation is only available in playtime

stk_material.PNG

SoundToolKitMeshCluster

[ScriptableObject]

This ScriptableObject is an abstraction of a group of AcousticMeshes. If the streaming of AcousticScene was needed by a game, this component defines which meshes are loaded and unloaded as one entity into the SoundToolKit engine. MeshCluster can also be serialized in order to load and unload it's content much faster and thus boost the initialization time of a scene. It is done via StkGeometry component. If contents of MeshCluster are mutated after serialization, it needs to be performed again.

stk_mesh_cluster.PNG

SoundToolKitGeometry

This component governs which MeshClusters are currently loaded into the AcousticScene - only StkMeshes that are parts of MeshClusters loaded into the scene take part in sound processing. The list of MeshClusters in that component determines which MeshClusters are loaded to the scene on the scene start - all the other loading/unloading has to be coded using the component's API.

If only the standard behaviour is needed - all meshes being loaded at once and taking part in the sound processing - the menu option Tools/SoundToolKit/Geometry/SynchronizeDeaultGeometry automates the process, setting up the StkGeometry component with all meshes stored in one DefaultMeshCluster, which can be then serialized**

The SerializeGeometry button serializes all the MeshClusters currently on the list. This enables much faster loading of AcousticScene upon scene start. Note that whenever a MeshCluster's content is mutated, it's serialized geometry is invalidated and must be serialized again.

stk_geometry.PNG

General

SoundToolKitManager

This component is an access point for the high-level internal Stk settings. It facilitates volume controls of STK output and allows controlling some of the STK engine's performance via it's API (pausing the audio output for instance). On top of that it holds the reference to StkSettings ScriptableObject, that manage some of the more low-level performance details of Stk engine.

In the current implementation StkManager component is a singleton, which means it is persistent between scene changes.

stk_manager.PNG

OnDisable: Stop Audio Engine

OnEnable: Set Previous state

OnDestroy: Release resources because AudioEngine is gone so all resources should be gone

OnSceneChange: Singleton / Provider [design thing]

SoundToolKitListener

A wrapper for StkReceiver that communiacates the transform of that component from Unity into SoundToolKit engine. Is equivalent of Stk's version of AudioListener.

OnDisable: Global sound should be muted OnEnable: Global sound should restore the previous state OnDestroy: Stop the updates of transform to StkReceiver

SoundToolKitSettings

[ScriptableObject]

StkSettings allow controlling some of the more low-level STK engine parameters. They are used by placing them in proper field of Stk Manager. This means multiple instances of settings can coexist and be swapped at runtime. Note: newly created settings need to be placed in Manager and then PlayMode must be entered in order for them to initialize.

stk_sets.PNG

Debugging utilities

SoundToolKitStatus

StkStatus provides extensive information about the state of STK engine via Indicators and Logs (below the StatusIndicators there are Logs). Note: The component only works properly when SoundToolKit is initialized - in PlayTime

stk_status.PNG

SoundPathsRenderer

Enables dispalying of Sound Paths generated by SoundToolKit Engine. Sound Paths simulate how sound behaves in interaction with Acoutic Scene. They are composed of segments, each segment being generated by one of the acoustic effects taking place on contact with a given mesh. Paths that are discovered by STK:

Direct path (no effect), Diffraction, Transmission, Invalid, Reflection DiffuseReflection

This component allows contolling which segments should be visualised. The Sound Path Visualisation is turned on/off by Update Sound Paths control.

This component is a part of SoundToolKitExtensions prefab**

stk_sound_path_renderer.PNG

AcousticMeshRenderer

This utility allows visual rendering of AcousticMeshes that are currently utilized by STK. To display them one should enable the checkbox RenderAcouisticMeshes. Take note that this is the static viewer, in order to see the current state in Engine you have to set it off and then on to see the changes (if there were any).

This component is a part of SoundToolKitExtensions prefab**

stk_acoustic_mesh_renderer.PNG

Build target & Platforms

SoundToolKit right now supports x64 builds and Windows platform.

In near future, the platform range will be extended.