Develop

Chapter 13: Camera Access

Updated: Jul 22, 2026

The IWSDK provides a camera access system that enables browser and XR applications to access device cameras for video streaming, photo capture, and computer vision tasks.

What You’ll Build

By the end of this chapter, you’ll be able to:

Set up camera access with automatic device selection.

Display live camera feeds in your XR experience.

Capture video frames for photo capture or computer vision.

Handle camera permissions and device switching.

Configure camera resolution and frame rate.

Overview

The camera system leverages the browser’s MediaDevices API. The system automatically manages the camera lifecycle while the world is visible, including non-immersive browser mode and immersive XR sessions.

Key Components

CameraSystem - Manages camera stream lifecycle.

CameraSource - Component holding camera configuration and output (texture, video element).

CameraUtils - Static utilities for device enumeration, permissions, and frame capture.

CameraState - Enum for camera lifecycle states (Inactive, Starting, Active, Error).

Quick Start

Here’s a minimal example to get camera working in your XR scene:

import { World, CameraSource, CameraUtils, SessionMode } from '@iwsdk/core';

// Optional: Request camera permission early for better UX
CameraUtils.getDevices()
  .then(() => console.log('Cameras ready'))
  .catch((error) => console.warn('Camera unavailable'));

World.create(document.getElementById('scene-container'), {
  xr: {
    sessionMode: SessionMode.ImmersiveAR,
  },
  features: {
    camera: true, // Enable CameraSystem
  },
}).then((world) => {
  // Create camera entity
  const cameraEntity = world.createEntity();
  cameraEntity.addComponent(CameraSource, {
    facing: 'back',
    width: 1920,
    height: 1080,
    frameRate: 30,
  });

  // Store for later access
  world.globals.cameraEntity = cameraEntity;
});

System Setup

Step 1: Enable Camera Feature

World.create(container, {
  features: {
    camera: true, // Registers CameraSystem and CameraSource
  },
});

Step 2: Request Permissions Early (Optional)

CameraUtils.getDevices()
  .then(() => {
    // Permission granted - camera will start quickly in XR
  })
  .catch((error) => {
    // Show UI warning that camera won't be available
  });

Why request early?

Avoids permission prompt interrupting XR session.

Caches available cameras for instant access.

Step 3: Create Camera Entity

const cameraEntity = world.createEntity();
cameraEntity.addComponent(CameraSource, {
  deviceId: '', // Empty = auto-select based on facing
  facing: 'back', // 'front' | 'back'
  width: 1920,
  height: 1080,
  frameRate: 30,
});

Important: The camera activates while the world is visible. It works in browser-only worlds created with xr: false and in visible XR sessions, and it stops streams when the page/session is hidden.

Understanding the Components

CameraSource

Holds camera configuration (input) and output (texture, video element, stream).

Input Properties

deviceId - Specific camera device ID (default: '' for auto-selection).

facing - Camera facing: 'front'

'back' (default: 'back').

width - Ideal video width in pixels (default: 1920).

height - Ideal video height in pixels (default: 1080).

frameRate - Ideal frame rate (default: 30).

Output Properties (Read-only)

state - Current state: CameraState.Inactive | Starting | Active | Error.

texture - VideoTexture for rendering (null until Active).

videoElement - HTMLVideoElement for advanced access (null until Active).

stream - MediaStream (internal, null until Active).

// Get texture from CameraSource
const texture = cameraEntity.getValue(CameraSource, 'texture');
const state = cameraEntity.getValue(CameraSource, 'state');

// Check if ready
if (texture && state === CameraState.Active) {
  material.map = texture;
}

CameraUtils

Static utility class for camera operations.

`getDevices(refresh?: boolean)`

// Get cached devices (fast)
const devices = await CameraUtils.getDevices();

// Force re-enumeration (slow)
const devices = await CameraUtils.getDevices(true);

// Each device: { deviceId, label, facing: 'front' | 'back' | 'unknown' }

findByFacing(devices, facing)

const devices = await CameraUtils.getDevices();
const backCamera = CameraUtils.findByFacing(devices, 'back');

hasPermission()

const granted = await CameraUtils.hasPermission();

captureFrame(entity)

const canvas = CameraUtils.captureFrame(cameraEntity);

if (canvas) {
  // Canvas at full video resolution
  const texture = new CanvasTexture(canvas);

  // Or export as image
  canvas.toBlob(
    (blob) => {
      const url = URL.createObjectURL(blob);
      // Download or upload
    },
    'image/jpeg',
    0.95,
  );
}

CameraSystem

Automatically manages camera lifecycle:

Starts cameras when XR visible.

Stops cameras when XR hidden.

Retries failed cameras.

Cleans up resources.

You don’t need to interact with the system directly.

Camera Configuration

Auto-Selection by Facing

cameraEntity.addComponent(CameraSource, {
  facing: 'back', // System picks best matching camera
});

Manual Device Selection

const devices = await CameraUtils.getDevices();
cameraEntity.addComponent(CameraSource, {
  deviceId: devices[0].deviceId,
});

Resolution and Frame Rate

cameraEntity.addComponent(CameraSource, {
  facing: 'back',
  width: 1920, // Ideal (may be adjusted by browser)
  height: 1080,
  frameRate: 30,
});

Switching Cameras

cameraEntity.setValue(CameraSource, 'facing', 'front');
cameraEntity.setValue(CameraSource, 'deviceId', '');
cameraEntity.setValue(CameraSource, 'state', CameraState.Inactive); // Restart

Common Patterns

AR Camera Viewfinder

class ViewfinderSystem extends createSystem({}) {
  private viewfinderPlane: Mesh | null = null;

  update() {
    if (!this.viewfinderPlane) this.createViewfinder();
  }

  private createViewfinder() {
    const cameraEntity = this.globals.cameraEntity;
    if (!cameraEntity) return;

    const texture = cameraEntity.getValue(CameraSource, 'texture');
    const videoElement = cameraEntity.getValue(CameraSource, 'videoElement');

    if (!texture || !videoElement) return; // Not ready

    // Calculate aspect ratio
    const aspectRatio = videoElement.videoWidth / videoElement.videoHeight;
    const width = 0.24;
    const height = width / aspectRatio;

    // Create plane with camera texture
    const geometry = new PlaneGeometry(width, height);
    const material = new MeshBasicMaterial({ map: texture });

    this.viewfinderPlane = new Mesh(geometry, material);
    this.viewfinderPlane.position.set(0, 0, -0.4);
    this.player.head.add(this.viewfinderPlane);
  }
}

Photo Capture

class PhotoCaptureSystem extends createSystem({}) {
  update() {
    if (this.input.xr.gamepads.right?.getSelectEnd()) {
      this.capturePhoto();
    }
  }

  private capturePhoto() {
    const canvas = CameraUtils.captureFrame(this.globals.cameraEntity);
    if (!canvas) return;

    // Create texture
    const texture = new CanvasTexture(canvas);
    texture.minFilter = LinearFilter;

    // Save photo
    canvas.toBlob((blob) => {
      if (!blob) return;
      const url = URL.createObjectURL(blob);
      const link = document.createElement('a');
      link.href = url;
      link.download = `photo-${Date.now()}.jpg`;
      link.click();
      setTimeout(() => URL.revokeObjectURL(url), 100);
    }, 'image/jpeg', 0.95);
  }
}

Digital Zoom

private capturePhotoWithZoom(zoomLevel: number) {
  const canvas = CameraUtils.captureFrame(this.globals.cameraEntity);
  if (!canvas || zoomLevel === 1.0) return canvas;

  const ctx = canvas.getContext('2d');
  if (!ctx) return null;

  // Create temp canvas with original
  const temp = document.createElement('canvas');
  temp.width = canvas.width;
  temp.height = canvas.height;
  temp.getContext('2d')?.drawImage(canvas, 0, 0);

  // Calculate crop for zoom
  const sourceWidth = temp.width / zoomLevel;
  const sourceHeight = temp.height / zoomLevel;
  const sourceX = (temp.width - sourceWidth) / 2;
  const sourceY = (temp.height - sourceHeight) / 2;

  // Draw zoomed region
  ctx.clearRect(0, 0, canvas.width, canvas.height);
  ctx.drawImage(
    temp,
    sourceX, sourceY, sourceWidth, sourceHeight,
    0, 0, canvas.width, canvas.height,
  );

  return canvas;
}

Device Switching UI

class CameraSwitcherSystem extends createSystem({}) {
  private availableCameras: CameraDeviceInfo[] = [];
  private currentIndex = 0;

  async init() {
    this.availableCameras = await CameraUtils.getDevices();
  }

  update() {
    if (this.input.xr.gamepads.right?.getButtonDown(0)) {
      this.switchCamera();
    }
  }

  private switchCamera() {
    if (this.availableCameras.length === 0) return;

    this.currentIndex = (this.currentIndex + 1) % this.availableCameras.length;
    const next = this.availableCameras[this.currentIndex];

    const cameraEntity = this.globals.cameraEntity;
    cameraEntity.setValue(CameraSource, 'deviceId', next.deviceId);
    cameraEntity.setValue(CameraSource, 'state', CameraState.Inactive);
  }
}

Troubleshooting

Common Issues

Camera not starting:

Verify features: { camera: true } is set.

Check camera permissions granted.

Ensure XR session is active.

Check console for errors.

Black screen:

Check state === CameraState.Active.

Verify texture is not null.

Check videoElement.videoWidth > 0.

Permission denied:

Request early with CameraUtils.getDevices().

Provide UI fallback.

Wrong camera:

Verify facing value.

Manually specify deviceId.

Poor quality:

Increase width and height.

Check actual resolution: videoElement.videoWidth/videoHeight.

Debug Tips

// Log camera state
const state = cameraEntity.getValue(CameraSource, 'state');
const deviceId = cameraEntity.getValue(CameraSource, 'deviceId');
console.log({ state, deviceId });

// Check devices
const devices = await CameraUtils.getDevices();
console.log('Available cameras:', devices);

// Monitor video
const video = cameraEntity.getValue(CameraSource, 'videoElement');
console.log({
  width: video.videoWidth,
  height: video.videoHeight,
  readyState: video.readyState,
});