
## Animation

Animation is what makes your game or app come alive. There are two main ways of producing an animation in BABYLON.js. You can change the properties of an object within a scene during the rendering cycle. We will take a look at that after describing the *Animation* method.



## Audio

The Babylon.js sound engine is based on the [**Web Audio specification**](https://webaudio.github.io/web-audio-api/) and to use it you need to use a Web Audio compatible browser. 

The sound engine offers ambient, spatialized and directional sound.


## Behaviors

Babylon.js v3.1 introduced a new component based tools: the behaviors.
A behavior is a simple class that can be attached to a target where it will provide a specific set of features. Features will be triggered by defined events.

## General

A behavior is defined by the following interface:

-   `name`: Return the name of the behavior
-   `init()`: This function will be called when a behavior needs to be initialized. This is before the attachment to a target.
-   `attach(target)`: This function will be called when a behavior is attached to a target. This is where the behavior will hook into useful events. Babylon.js will make sure that the scene is not currently loading when this function will be called.
-   `detach()`: This function will be called when a behavior is detached from a target. The behavior must clear any associated resources and unhook all events

If behaviors rely on animation, the following **static** properties will be available:

-   `EasingFunction`: Define the easing function used animations
-   `EasingMode`: Define the easing mode used by animations

You can add behaviors to any object implementing the IBehaviorAware interface (lights, cameras or meshes for instance). Every IBehaviorAware provides the following entry points:

-   `addBehavior(behavior)`: Use this function to attach a behavior to a given target. If the scene is currently loading, this code will be delayed until scene completion.
-   `removeBehavior(behavior)`: Use this function to detach a behavior from a target
-   `getBehaviorByName(name)`: Return a behavior with the given name or null if not found
-   `behaviors`: This read-only property returns the list of behaviors associated with the target

Most of the time, behaviors are designed to deal with a specific kind of targets.

There are both camera and mesh behaviors.


## Cameras

There are a wide range of cameras to use with Babylon.js, Probably the two most used are the universal and arcRotate cameras and increasingly virtual reality ones.

* _Universal Camera_ - the one to choose for first person shooter type games, and works with all the keyboard, mouse, touch and gamepads. With this camera you can check for collisions and apply gravity.
* _Arc Rotate Camera_ - which acts like a satellite in orbit around a target and always points towards the target position.
* _Follow Camera_ - this takes a mesh as a target and follows it as it moves. Both a free camera version _followCamera_ and an arc rotate version _arcFollowCamera_ are available.
* _Anaglyph Camera_ - extending the use of the Universal and Arc Rotate Cameras for use with red and cyan 3D glasses. 
* _Device Orientation Cameras_ - this is a designed to react to a device being tilted forward or back and left or right. 
* _Virtual Joysticks Camera_ - on-screen 2D graphics that are used to control the camera or other scene items. 

* _Virtual Reality Camera_ a range of cameras for VR devices

Methods are available to change how a camera reacts to inputs and to use multiple cameras.


# Controls

Babylon.js Controls are a set of regular web controls that are super-charged with hardware accelerated rendering through Babylon.js to provide blazing fast dedicated controls.

Babylon.js provides an unified API on top of **WebGL**, **WebGL2** and **WebGPU** that controls can leverage to unleash the raw power of your GPU.

![Resizer](/img/features/controls/resizer.png)


Sample https://github.com/sebavan/BabylonjsVideoProcessingSample


## Realistic Environments

Of course working on games or apps the level of realism achieved will depend on your design decisions. Babylon.js can help you achieve whatever level you wish and in this section we will cover some of the ways to do this. For example lighting the background, lens flare, adding sky and fog or working with HDR.

Other ways need a section to themselves, such as infinite terrains, post processes and shaders.


## Event Handling

Naturally the main source of events are through the mouse, keyboard, touch, device movement and game controllers. In this section we will consider managing some of these. In terms of coding there are two features of Babylon.js that handle events, actions and observables. It is also possible to use plain JavaScript.


## Graphical User Interface

There are a number of options for adding a GUI to Babylon.js. The **Babylon.GUI** is covered in this section.

It allows you to place buttons and labels within in 3D space as well as a 2D front of screen GUI.
When you want a GUI that works in VR or within the 3D space it's the only option.

It is integrated within the playground. For your own projects it has to be loaded as well as Babylon.js.

<Playground id="#NGS9AU" title="Simple GUI Slider Example" description="Simple example of adding a GUI slider to your scene." image="/img/playgroundsAndNMEs/features/divingDeeperGUI1.jpg"/>

Other possible GUIs are:
1. CastorGUI, a Babylon.js community extension which overlays the scene
1. Dat.GUI, an external interface
3. HTML GUI


## CastorGUI

An alternative 2D GUI is the extension [CastorGUI](/toolsAndResources/castorGUI). It has to be loaded both for the playground and for your own projects.

It can be found on [Github](https://github.com/dad72/CastorGUI).

<Playground id="#S34THY#14" title="CastorGUI Example" description="Simple example of using the CastorGUI system in your scene." image="/img/playgroundsAndNMEs/features/divingDeeperGUI2.jpg"/>

## Dat.GUI

The external [dat.GUI](https://github.com/dataarts/dat.gui) is integrated within the playground. For your own projects it has to be loaded as well as Babylon.js.

<Playground id="#NGS9AU#1" title="dat.GUI Example" description="Simple example of using the dat.GUI system in your scene." image="/img/playgroundsAndNMEs/features/divingDeeperGUI3.jpg"/>

## HTML GUI

Since Babylon.js is in JavaScript it is possible to use HTML and CSS to overlay the Babylon.js scene.

<Playground id="#1AHPN5" title="HTML GUI Example" description="Simple example of using HTML GUI elements in your scene." image="/img/playgroundsAndNMEs/features/divingDeeperGUI4.jpg"/>

## Comparison of GUI Options

Here's a list of the pros and cons of using different types of GUIs. It’s worth noting that these options aren’t mutually exclusive - they can be used together, depending on requirements, for instance:

* Use a simple HTML GUI for initial rapid protoyping, before transitioning to Babylon 2D or 3D GUI
* Use an HTML GUI overlay for complex, dynamic, text-heavy info panels but Babylon 2D GUI for everything else

The sky’s the limit! Go forth and GUI in whatever way takes your fancy!


### HTML GUI

#### Pros

* Use familiar HTML, CSS and front-end frameworks like Bootstrap, Tailwind, React, Vue, Svelte & Angular etc
* Near unlimited flexibility & mobile responsiveness
* High performance (as rendered by native browser rather than 3D engine)
* Easier to make WCAG accessibility compliant

#### Cons

* Looser integration with 3D scene
* Can’t have GUI elements directly within 3D scene (e.g. applied to meshes)
* Can’t apply 3D post processing effects to overlaid HTML GUI elements
* Can’t be used for fullscreen/native VR

### Babylon 2D GUI

#### Pros

* There’s now an awesome [GUI Editor](/toolsAndResources/tools/guiEditor) to make interface creation easier!
* Tight integration with 3D scene
* Ability to optionally apply scene post processing effects to GUI as well
* Ability to apply/link GUI elements and meshes
* Some unique, useful capabilities like nine-patch stretching and sprite-sheet animation that aren’t available with a raw HTML GUI

#### Cons

* Less comprehensive/flexible than HTML GUI (but still more than enough for most requirements)
* Depending on advanced dynamic texture resolution, GUI may look a little blurry
* Possibly some performance considerations
* Can’t be used for fullscreen/native VR

### Babylon 3D GUI

#### Pros

* Supports fullscreen/native VR
* Tight integration with 3D scene
* Ability to optionally apply scene post processing effects to GUI as well

#### Cons

* Less comprehensive/flexible than both Babylon 2D GUI and HTML GUI


# Importing Assets

The built in file type is `.babylon` and Babylon.js can load these without a plugin. All other file types require a plugin as described in this section.

Possible file types are gLTF, obj, stl

To help you with imported assets there is a manager for them.

**Note:** Since meshes you import can have a _rotationQuaternion_ set before applying a rotation set the _rotationQuaternion_ to _null


# Inputs

As well as the usual keyboard, mouse and touch input methods you can also use a gamepad, device orientation and a virtual joystick.


## Lights and Shadows

Lights are of course necessary to see the meshes and affect brightness and colour. Although lights show meshes they only produce shadows when shadow generation is set on the mesh. The default number of lights allowed is four but this can be increased. 

There are four types of lights that can be used with a range of lighting properties.

* The Point Light - think light bulb.  
* The Directional Light - think planet lit by a distant sun.  
* The Spot Light - think of a focused beam of light.
* The Hemispheric Light - think of the ambient light.

Color properties for all lights include _emissive_, _diffuse_ and _specular_.

Overlapping lights will interact as you would expect with the overlap of red, green and blue producing white light. Every light can be switched on or off and when on its intensity can be set with a value from 0 to 1. 

For shadows a _shadowGenerator_ object is needed. There is also an extension, **shadows only**,  that allows shadows on a transparent mesh.

To begin to understand how meshes react to light you need to know about the normals of a mesh since they form part of the calculations in determining the color and illumination.



# Materials

Materials give your meshes color and texture.  One material in Babylon.js can be used to cover as many meshes as you wish. They can be displayed as a wireframe, scaled and offset across a mesh, have degrees of transparency and be blended. You can apply multiple materials on one mesh and use a dynamic texture as a material that you can draw and write on in real time. Do you need to make your material to appear more textured or bumpier and reflect and refract? The first section shows you how.

Node materials allow you create a material based on custom shaders but without having to deal with shader code.

The advanced section considers more technical aspects of materials and ways of customizing them.



## Meshes

In the 3D virtual world shapes are built from meshes, lots of triangular facets joined together, each facet made from three vertices.

![A mesh](/img/features/scene/custom1.jpg)

Babylon.js provides for the creation of predefined meshes, of your own custom meshes and for importing meshes created by 3D design software. For all meshes the underlying data structure is exposed and can be updated.


## Introduction

Babylon.js v3.1 introduced a new feature: Occlusion Queries.
Occlusion Queries detect whether a Mesh is visible in the current scene or not, and based on that the Mesh get drawn or not. Occlusion Queries is useful when you have an expensive object on the scene and you want to make sure that it will get drawn if it is visible to the camera and it is not behind any opaque object.
BabylonJs provides an implementation for Occlusion Queries using property occlusionType in AbstractMesh Class

## How Occlusion Queries works behind scenes

Babylon.js engine draw a light transparent bounding box on the targeted Mesh before drawing the object and create a query to check with WebGl engine if the bounding box is visible or not. if the box is visible, the object gets drawn if not the object is not drawn, Occlusion Queries is asynchronous and usually the query result of the object is not available in the current frame and because of this the object is drawn based on a query result of previous frame, the user wouldn't notice the difference unless your FPS is too low.

<Playground id="#QDAZ80#5" title="Occlusion Query Example" description="Simple Example of using occlusion queries in your scene."/>

## Basics

To use the Occlusion Queries on a Mesh

```javascript
var sphere = BABYLON.Mesh.CreateSphere('sphere1', 16, 2, scene);
sphere.occlusionType = BABYLON.AbstractMesh.OCCLUSION_TYPE_OPTIMISTIC;
```

For more about occlusionType and the supported algorithm check AbstractMesh class [documentations](/typedoc/classes/babylon.abstractmesh).

If your object by default is behind an opaque object you can set property `isOccluded` to true so Babylon.js engine won't take a decision to render or not until the query result is retrieved from WebGl engine.

```javascript
sphere.isOccluded = true;
```

## Advanced

As described earlier the Occlusion Queries result is asynchronous and it may take some time to get the result and because of this the object would take many frames to be loaded waiting for the query result. In this case you can use property `occlusionRetryCount` to set the number of waiting frames before query get broken. Once a break happens you will need to decide whether to draw the object or to maintain its state, property `occlusionType` is used for this reason as you have 2 options
1) OCCLUSION_TYPE_OPTIMISTIC: this option will render the mesh if a break is happened.
2) OCCLUSION_TYPE_STRICT: this option will restore the last state of the object whether visible continue as visible or hidden continue as hidden.

As a scenario of using restrict and optimistic if you have 2 expensive objects in your scene one of them is a must render object so you could set `occlusionRetryCount` and set the occlusionType to optimistic so your object will be rendered in case the query result is not available. If your object can wait till the query is available, don't set the `occlusionRetryCount` or set the property while use occlusionType as Strict so if the object was rendered in the last scene re-render it again in the current scene else hide it.

```javascript
sphere.occlusionRetryCount = 10;
sphere.occlusionType = BABYLON.AbstractMesh.OCCLUSION_TYPE_STRICT;
```

In Babylon.js you can also set the WebGl Occlusion Queries algorithm type using property `occlusionQueryAlgorithmType` for more into check AbstractMesh Class [documentations](/typedoc/classes/babylon.abstractmesh).

You can find a live demo here: <Playground id="#QDAZ80#3" title="Advanced Occlusion Queries" description="Example of using advanced occlusion query methods in your scene."/>


## Particles

Babylon.js supports three types of particles. These are based on sprites, meshes and points.

The particle system uses small 2D sprites which may be animated and always face the camera to simulate effects such as smoke. The clouds of particles produced can be controlled with a wide range of properties. This differs from the other two systems as it has parameters to control its behavior whilst the others have to have behavior coded. From Babylon.js V3.2, where the browser supports WebGL2, GPU particles are available. These make full use of the GPU though there are some restrictions in its use. When WebGL2 is not available the *GPUParticleSystem* will fall back to the *ParticleSystem* automatically.


The solid particle system (SPS) combines one or more base meshes into a single mesh requiring a single draw call per frame. There are many features available that allow you to control the behavior of the individual mesh particles.

A points cloud particle system (PCS) is a single updatable mesh and uses a material property to display its vertices as pixel squares of a size you set. The points cloud system provides some methods to manage the individual particles.


# Support for Physics Engines

Physics engines are third party external programs that can be plugged into Babylon.js. They, depending on their features, emulate "real-life" interactions between objects, which can be meshes, solid particles from the solid particle system or some cameras. One feature of a Physics Engine is the addition of gravity to a scene.

There are plugins for 3 physics engines:

1. Cannon.js - a wonderful physics engine written entirely in JavaScript
2. Oimo.js - a JS port of the lightweight Oimo physics engine
3. Ammo.js - a JS post of the bullet physics engine

All need to be enabled before use.

Interactions between objects are achieved by imposters, simple objects that are attached to any complex objects with a scene. The imposter can be assigned physical attributes such as mass, friction, a coefficient of restitution, impulse, and linear and angular 

Two imposters can be connected using joints such as a hinge or ball and socket.

Ammo also allows the creation of soft bodies.


# PostProcesses

Postprocesses allow you to create 2D effects on top of your scene.
A postprocess is linked to a camera and can be part of a chain of postprocesses where each postprocess uses the result of the previous one as input for its own processing.


# The Scene

The scene is somewhat like the stage on which all the meshes are placed to be seen and the cameras and lights are placed that make them viewable. Of course there are more than meshes, cameras and lights that make a scene a good experience for your users, for example the user interface. Also there are special effects to be had to make the environment more realistic. Want to overlap two or more scenes that is possible as well. 

Want to build a scene with the minimum of code then fast build is for you. 

What else might you expect of your scene? You might expect to be able to

* have the scene react to clicks and touches;
* have your own design displayed when your scene is loading;
* optimise your scene to maintain a good speed;
* output images of particular views of your scene.


# Sprites

Sprites are 2D images and always face the camera. They can used to display animated characters, and particles, and to simulate 3D complex objects like trees.  

For animation individual sprite images are collected together in a single file called a spritesheet.

* A uniform spritesheet is one where all the sprites are exactly the same size and arranged in order in the file. When you read the term spritesheet in the documentation you can usually assume that it is referring to a uniform spritesheet. A uniform spritesheet is overseen by a *Sprite Manager*.
* A packed spritesheet is one where the sprites can be of different sizes and often packed in such a way as to minimize the overall size of the file. Usually the full term of _packed spritesheet_ will be used for such a spritesheet. A packed spritesheet is overseen by a *Sprite Packed Manager*. * available from BJS version 4.1*

For sprites the use of one of these managers is mandatory, even for one sprite. They optimize GPU resources by grouping in one place multiple instances of a sprite. 

Also available, for any grid like implementation (in a 2d or 2.5d game level for instance) that require thousands of sprites to be animated and rendered, is a special system called a *Sprite Map*, *available from BJS version 4.1*


# WebXR

The WebXR API enables developers to create VR and AR experiences for the web. XR (Extended reality) has unified both Augmented Reality features and Virtual Reality features into a single API that works across all supported devices.


## Diving Deeper

We hope you have enjoyed splashing around in the Getting Started section and are now ready to plunge into understanding more of what is possible with Babylon.js. Looking for something specific and cannot find it within the navigation menu or landing page selections then head for the search bar and the results will cover the all the documentation including the API. Below shows you the surface of the vast ocean that is this intermediate section of Babylon.js, dive in to get a deeper understanding.



# Introduction - Going Virtual

## See Our Virtual Village World in XR

It could not be simpler - we remove the current camera and replace it with

```javascript
const xr = scene.createDefaultXRExperienceAsync();
```

This will enable WebXR **in VR immersive mode**, including session init, input sources, the camera, teleportation and scene interactions. All using our [WebXR Default Experience Helper](/features/divingDeeper/webXR/webXRExperienceHelpers#the-basic-experience-helper).


Diving Deeper

Diving Deeper

In This Category

Animation

Deep dive into the wonderful world of Animation in Babylon.js.

Audio

Dive into using the Babylon.js sound engine for simple to advanced audio.

Behaviors

Dive into the rich world of behaviors in Babylon.js.

Cameras

Learn all about the different types of cameras available in Babylon.js.

Controls

Babylon.js Controls are a set of regular web controls that are super-charged with hardware accelerated rendering through Babylon.js to provide blazing fast dedicated controls.

Environment

Learn all about creating beautiful environments in Babylon.js.

Events

Learn all about events and event handling in Babylon.js.

GUI

Learn all about the Babylon.js GUI systems.

Input

Learn all about user input in Babylon.js.

Importing Assets

Learn how to import assets to use in your Babylon.js scenes.

Lights

Learn all about lights and shadows in Babylon.js

Materials

Jump into the exciting world of learning Babylon.js materials.

Meshes

Dive into the rich world of understanding meshes in Babylon.js.

Occlusion Queries

Learn all about leveraging occlusion queries in Babylon.js.

Particles

Dive into the rich and exciting world of particles in Babylon.js.

Physics

Learn all about the powerful physics capabilities of Babylon.js.

Post Processes

Learn Post Process effect in Babylon.js.

Scene

Learn all about the core "scene" object in Babylon.js.

Sprites

Learn all about sprites in Babylon.js.

Tags

Learn all about tags in Babylon.js.

WebXR

Learn about creating immersive web experineces with WebXR in Babylon.js.