Generic		FreeBSD/Linux		Solaris		Windows

This page gives advice on connecting audio and video devices to a computer for use with Open Mash. The advice given reflects our testing and experience with various chips, boards, and drivers.

Generally speaking, Open Mash looks for audio and video devices connected to computer and creates objects for them. Applications can request a list of devices (see note on video encoding) and open a device. Open Mash has a glue object that mediates between the device operations required by the toolkit and the device operations supplied by the hardware and drivers. This glue object includes operations to open/close the device, set parameters, and request a GUI control for manipulating the device parameters. Open Mash tries to use generic audio and video device abstractions supplied on the various platforms (e.g., Video-for-Windows) but sometimes a device will have features not supported by the generic device abstraction. In this case, the toolkit defines a hardware/device-specific glue object that provides both the general abstraction required by Open Mash and operations to allow use of the specific hardware capabilities. A good example is a video capture board that also contains an on-board codec (e.g., the Linux LML33 board which has an MJPEG codec, etc.).

The remainder of this page gives general information about audio and video device and data abstractions.

Audio Input/Output Devices

Most audio devices support both capture and playback of audio. The vat Mbone tool which formed the foundation of the Open Mash audio abstractions supported full-duplex 8KHz 8-bit audio. The representation used internally for an audio block is an array of PCM samples. Audio devices are block-oriented - with a default of 20 msecs per block, which is 160 samples. Recently, the audio abstractions were modified to support 16-bit samples (L16 representation) and sampling rates up to 48 KHz although support for high quality audio has not been integrated with all applications yet. The abstractions can be used to create, manipulate, and play multiple channel audio.

A major problem with audio is support for full-duplex operation. Many early PC sound boards do not support full-duplex operation. N-way video conferencing, which was the first application of the Mbone Tools assumed full-duplex operation so that a speaker could be listening to audio from other participants when speaking. If the system you are using does not support full-duplex sound cards, Open Mash applications include a "push-to-talk" command which switches between capturing and playing audio.