External Streams / Mixing
Since Webcaster version 5.11.0 you can use sources based on HTML5 canvas or other stream sources to enable user-generated live sources, separate to live camera or device screens.
To use this feature, a new API call was added, setMediaStream(). It works without using video and audio sources from startPreview().
notes:
If you simply intent to utilize a combination of camera, microphone or screen share sources:
- please use the startPreview() call.
use cases
- mixing of different video sources into one stream
- PIP (picture in picture)
- mixing of different audio sources together
- preprocessing of video and audio in the browser
- video overlays, eg. rendering animations, drawings, watermarks and logos
- HTML5 canvas as a live source
general workflow
- acquire different video/image and/or audio sources
- mix the sources together into one MediaStream
- video: in a canvas, write your render loop where you mix the sources together via HTML canvas drawing
- audio: with the WebAudio API
- pass the resulting stream to the webcaster API
restrictions
When utilizing this API call, application developers are responsible for the externally created stream.
- Maintaining the provided framerate is critical for end to end latency of the webcast.
- Tab throttling. When the current tab gets out of focus, most browsers will enable tab throttling. Timers, like
setInterval
, will be updated less frequently. You should check wether the tab left focus, and warn your users accordingly. E.g. with the DOM visibilitychange event - Video dimensions (width & height) must not change during a webcast.
- Under certain circumstances the browsers encoder will adapt the video resolution, for example when CPU overusage is being detected.
resources
- access audio and video devices
- capture a screen share or an application window
- creating a MediaStream from a HTML canvas
- create MediaStreams with the WebAudio API
samples & documentation
online samples
Please see the online samples (replace XXX with your bintu API key):
- webcast_external_mixing_sample.html: https://webrtc.nanocosmos.de/release/webcast_external_mixing_sample.html?bintu.apikey=XXX
- webcast_external_stream.html: https://webrtc.nanocosmos.de/release/webcast_external_stream.html?bintu.apikey=XXX
usage sample
HTML canvas as video source
// pass an MediaStream to the API
// rtcUser: instance of RtcUser
var canvasWidth = 1280;
var canvasHeight = 720;
var framerate = 30;
// access your HTML canvas (<canvas id="my-canvas"/> element)
var canvas = document.getElementById('my-canvas');
var context = canvas.getContext('2d');
// create a MediaStream from the canvas
var canvasStream = canvas.captureStream(framerate);
// optionally, add an AudioStreamTrack
var myAudioTrack = ...;
canvasStream.addTrack(myAudioTrack);
*
// your render loop, the loop calls itself 30 (framerate) times per second
// as an example, we simply draw the string "Hello World"
function renderLoop() {
context.font = '50px serif';
context.fillText('Hello world', 50, 90);
setTimeout(loop, 1000 / framerate); // drawing at 30fps
};
*
var config = {
stream: canvasStream,
width: canvasWidth,
height: canvasHeight,
framerate: framerate
};
// start the render loop
renderLoop();
rtcUser.setMediaStream(config);