2001 Project Plans

Lawrence A. Rowe
(August 1, 2001)

This note describes our development plans for the third year of the NSF grant. Our plans include tasks that must be completed and tasks that are stretch goals. And as with all development projects, there will be numerous small projects completed along the way.

[must] H.263 codec.
Open Mash currently supports a number of video codecs including MJPEG, H.261, CELLB, NV, LDCT, etc. Some of these codecs are deprecated (e.g., CELLB, NV, and LDCT), and they will be removed from the code. MJPEG and H.261 are the primary codecs used with video streaming applications. MJPEG can be used for images up to full-sized D1 images (i.e., 720x480) with and without interlacing but every block is coded which makes it relatively slow. H.261 does block encoding so it is more efficient but the H.261 standard supports only QCIF- or CIF-sized images (i.e., 176x144 and 352x288, respectively).
Many applications want larger-sized image block encoded video streams. H.263 is an improved version of H.261 that supports more coding tactics and supports larger images. A series of extensions have been made to H.263 (e.g., H.263+, H.263++, etc.) leading to MPEG4. While MPEG4 has numerous algorithms for coding images including region coding, the baseline MPEG4 codec is based on H.263++. We plan to implement a simple H.263 codec that will support larger images. High quality commercial codecs can produce excellent CIF coded streams at 250 Kbs. Our plan is to produce good quality full-sized image coded streams at 500-750 Kbs.
[must] 16-bit audio support and audio codecs.
Open Mash, and for that matter vat on which the code is based, only support 8-bit 8 KHz sampled audio. Modern streaming audio applications require improved audio capture and playback (e.g., multiple channels, 16-bit digitizing, and sampling rates up to 48 KHz). We are implementing new audio abstractions, specifically an AudioAgent and codec architecture similar to the video abstractions, that will support improved audio. We are also incorporating an MP3 codec so that high quality audio can be streamed and played.
[must] RGB capture abstractions.

Many applications require the ability to capture and stream material taken from a computer display (e.g., presentation slides, demonstrations, web pages, etc.). The typical approach to solving this problem is to use a scan converter to convert the RGB signal from the computer to an NTSC or PAL video signal and then digitize the video signal, compress and transmit it. This solution works, but the quality is poor due to the lower resolution video signal (i.e., a 1024x768 RGB image has to be mapped into a 720x480 video image) and the errors inherent in digitizing an analog video signal.

We have found a low cost (approx. $1K) RGB digitizing board from a company in England (Datapath Limited) that can be used with Windows PC's. We are implementing a glue routine that will allow images captured from the board, called the VisionRGB board, to be passed to a video codec for streaming. Using this board, it will be possible to capture full-sized RGB images (i.e., xga and sxga) and stream them. Moreover, since the images will not transformed through NTSC or PAL, they will be much higher quality.
[must] Resurrect MediaBoard and AS/1 Services.

During the transition to Open Mash and the significant code clean-up done in 2000 (e.g., eliminating all compiler warnings and removing cruft from the Mash source tree), several important applications stopped working. The MediaBoard and AS/1 Services are two applications that researchers have asked us to clean-up and support. Consequently, we will get this code working and continue to support it. Note: the AS/1 Services are important for many other Open Mash applications including the Director's Console/Virtual Director, the MARS archiving/replay system, and the Video Explorer, an ad hoc video browsing system currently under development at Berkeley.
[must] Improved remote video capture application (rvc/rvic).

Several applications (e.g., Access Grid and Berkeley Internet Webcasting System) require server applications to capture and stream media. For example, rvic and rvc are two such applications that stream video. At present these applications use TclDP for the remote procedure call protocol (RPC). TclDP is a simple text-oriented RPC that is used by all our applications. More details are available at TclDP Project. However, that project is now defunct, and since sockets were added to Tcl, it is possible to implement TclDP functionality using Tcl-only as shown by the following example.

Recently there has been a lot of interest in a portable, text-based RPC package based on XML, sometimes called XML RPC. This protocol is gaining widespread interest particularly after Microsoft elected to use it for the .NET initiative.

We plan to modify the rvic application to use XML/RPC and develop a new client-side library for accessing and controlling the application.
[must] Upgrade to recent versions of Tcl, Tk, oTcl, and TclCL.

Numerous changes have been made to the various packages used in Open Mash. We need to upgrade to these new packages to incorporate bug fixes and solve some problems in Mash.
[stretch] Combined audio/video player with synchronization.

The Mbone tools have always operated as separate applications, for example, vic captured/played video streams and rat/ vat captured/played audio streams. Audio/video synchronization between streams has always been problematic. Many modern applications need high-quality audio/video synchronization (e.g., RTPtv). Moreover, all commercial streaming software uses one application to play both audio and video. A simple audio/video player that implements high-quality synchronization is needed.
[stretch] Memory leak code clean-up.

Several people have suggested that Open Mash has numerous memory leaks. This year we are using some memory tracking tools (e.g., Purify) to identify possible problems and fix them.
[stretch] Distribute Internet Webcasting System Software.

A number of tools have been developed at Berkeley to support production of Internet Webcasts. A brief overview is given in the workshop paper Streaming Media Middleware is more than Streaming Media. Other research groups have asked to use these tools. The current software is coded with too many specific assumptions about the Berkeley environment. We are developing a new audio/video infrastructure using XML descriptions and AS/1 services which will be used to support both the video explorer and the webcasting system. Once that infrastructure is complete, we can distribute the Webcasting System for others to use.
[stretch] MacOS X Port.

The original NSF grant specified that we would do a Mac port of Mash. One reviewer of the proposal noted that it would be foolish to port to MacOS 9 or earlier since MacOS X was going to be released soon and it would be based on Unix which should simplify the port. We have investigated this project and discovered that the Tcl/Tk port for MacOS X is not yet available. We hope it will be completed soon at which time we'll do the Mash port. In the meantime, many changes being made to clean-up the code and move to new versions of Tcl and related packages will simplify the port when we do it.