Vectorly AI Filters Library

Vectorly AI Filters Library

Class

BackgroundFilter

new BackgroundFilter(input, params)

BackgroundFilter initialization

Parameters

  • input MediaStream | MediaStreamTrack | HTMLMediaElement

    Input to the BackgroundFilter; Either a MediaStream object from getUserMedia or a MediaStreamTrack which is a video track or a video tag

  • params object

    Properties

    • token string <optional>

      Token used to fetch models from server; Signup on Vectorly dashboard to get the token

    • model string <optional>

      Model to use. Options are
      "selfie" : Mediapipe segmentation
      "selfie_v2": Mediapipe segmentation version 2
      "webgl" : WebGL implementation
      "webgl_v2" : WebGL implementation version 2

    • background string | HTMLImageElement | HTMLCanvasElement | ImageBitmap | ImageData <optional>
       https://files.vectorly.io/demo/videocall/virtual-background.png

      For background blur, provide the string "blur".
      For Virtual Background Images, provide the String of URL of background image to use, or any type of Image source supported by createImageBitmap
      For transparent background provide the string "transparent"

    • blurRadius number <optional>
       5

      Value of blur radius to use, typically set as a value between [1, 10];

    • frameRate number <optional>
       30

      Framerate used for running the virtual background filter

    • segmentationFrameRate number <optional>
       15

      Target frame rate for running segmentation

    • passthrough boolean <optional>
       false

      If set to true; calling disable will pass the input directly through to the output MediaStream so that you can call disable/enable without changing the output MediaStream object. Default is false, in which case the output MediaStream stops when disable is called. segmentationFrameRate is set to the nearest

    • debug boolean <optional>
       false

Methods

static

isModelSupported(model) → {boolean}

async isModelSupported: Check if the passed model name is supported by the browser or not

Parameters

  • model string

    The name of model such as selfie, webgl, selfie_v2, webgl_v2

Returns

  • boolean
    • true if model can be run and is supported; false otherwise
static

isSupported() → {boolean|object}

async isSupported - Get whether current browser is supported and if so, list of features supported by the current browser

features object returned has following attributes:

  • wasm: Is WebAssembly supported? WebAssembly support is required for BackgroundFilter to work
  • simd: Is SIMD supported in WebAssembly? SIMD support reduces CPU usage but isn't necessary
  • webgl1: Is WebGL1 supported?
  • webgl2, Is WebGL2 supported?
  • offscreen: Is OffscreenCanvas supported? OffscreenCanvas support lets the camera feed process in the background even when the current tab is hidden
  • gpu: Gets the details of GPU that will be used
    • gpu.hwaccel: status of hardware acceleration; true/false
    • gpu.tier: Tier of the GPU; higher is better. GPU is classified into either tier: 1 (>= 15 fps), tier: 2 (>= 30 fps) or tier: 3 (>= 60 fps)

Returns

  • boolean object

    Returns false if the background filter can not be run at all. Returns current browser's support for a list of features if it can be run.

async

changeBackground()

async changeBackground - Change the background used in the filter

Parameters

  • https://files.vectorly.io/demo/videocall/virtual-background.png string | HTMLImageElement | HTMLCanvasElement | ImageBitmap | ImageData <optional>

    For background blur, provide the string "blur".
    For Virtual Background Images, provide the String of URL of background image to use, or any type of Image source supported by createImageBitmap
    For transparent background provide the string "transparent"

async

changeBlurRadius(blurRadiusopt)

async changeBlurRadius - Change the radius to increase or decrease blur strength

Parameters

  • blurRadius number <optional>
     5

    Value of blur radius to use, typically set as a value between [1, 10];

async

changeInput(input, stopPreviousStreamopt)

async changeInput - Change the input to the background filter

Parameters

  • input MediaStream | MediaStreamTrack | HTMLMediaElement

    Input to the BackgroundFilter; Either a MediaStream object from getUserMedia or a video tag

  • stopPreviousStream boolean <optional>
     false

    When set to true; stops the previous input stream that was set

async

destroy()

destroy.- terminates the WebWorker; call this to cleanup resources

async

disable() → {MediaStream}

disable - Disables background filter processing. Returns input stream as a new MediaStream object

Returns

  • MediaStream

    Outputs the input MediaStream object with no filter applies

async

enable() → {MediaStream}

enable - Enables background filter processing. Returns processed output stream as a MediaStream object

Returns

  • MediaStream

    Outputs MediaStream object which is the processed input stream with the background filter applied

async

getOutput() → {MediaStream}

async getOutput - Get the output MediaStream

Returns

  • MediaStream

    Outputs MediaStream object which is the processed input stream with the background filter applied

async

getOutputTrack() → {MediaStreamTrack}

async getOutputTrack - Get the output MediaStreamTrack

Returns

  • MediaStreamTrack

    Outputs MediaStreamTrack object which is the processed input video track with the background filter applied

async

requestFrameRate(setting) → {BackgroundFilter~frameRateSetting}

async requestFrameRate - Try to Change the framerate used in the filter. The return value gives the actual frameRateSetting values set, based on constraints of the track.

Parameters

  • setting object

    Framerate to be used for running the virtual background filter

    Properties

    • frameRate number <optional>
       30

      Framerate used for running the virtual background filter. If set to higher than the input MediaStreamTrack's frameRate, it defaults to the track's frameRate

    • segmentationFrameRate number <optional>
       15

      Target framerate for running Segmentation. If set to higher than frameRate; it will be default to frameRate.

Returns

stop()

stop.- stops the input track and the output track. Make sure to call disable before this

Type Definitions

frameRateSetting

Type

  • object

Properties

  • frameRate number

    frameRate at which the filter is run

  • segmentationFrameRate number

    frameRate at which the segmentation is run