Introduction

General Considerations

Multimedia Programming

Timing controls are common in visual environments like VB and Delphi. Their purpose is to raise an event at a user-specified interval. The programmer can then set up a routine to process this event. This is particularly useful for sprite animation. A simple example is to have your timer event call a routine to display the next sprite frame. Reasonably smooth animation can be achieved by having a timer that will fire an event at least 10-15 times a second. As timers use system resources and as the number of timers present in an environment are limited, it is a good idea to have each timer control the activity of as many sprites as possible. Ideally, you would have no more than one timer active at any one time.

In applications that have a lot of activity, the use of timers is often inadequate. A situation to avoid the use of timers, for example, is when your application uses a scrolling background or has a large number of sprites. Timer events can be replaced with a loop that handles the animation. Besides the graphic processing, the loop must also make sure to process system messages. This ensures that the 'non-sprite' components of the application will continue to function normally. A typical animation loop would be coded as shown in the following psuedocode:

	while animation = TRUE do
		process system messages
		process next graphics frame
	end while

The previous discussion holds for animation-intensive applications that do not involve video. The choice is not as clear when video is to be integrated with large amounts of graphic processing. Like sprite animation, video spends a lot of time processing (decompressing) the next frame. When video and animation run concurrently, the struggle for CPU time becomes critical. Video playback is driven by it's own timer. Using timers to control your own graphics can lead to severe video problems. Depending on the amount of graphic processing that each frame performs, a timer interval of about 40ms can stop the video from playing altogether. That is, your animation timer effectively stops the video timer's messages from being processed.

Adopting the processing loop approach, however, is not always better. Since this approach alternates between system messages and graphics processing, spending too much time in either step will degrade the performance of the other. The success of the processing loop depends on the ability to quickly process each step. Without this, the result is poor video, choppy animation, or both.

Depending on the approach taken, animation or video playback will suffer to varying degrees. This is not to say that it can't be done successfully. Typically, a judgement will be made given the available hardware, the desired effects of the application, and the performance of initial prototypes. Also, as video codecs improve (and move to hardware) the CPU time spent processing video will decrease. Such improvements in video technology, as well as improvements in processing power, can only improve the performance of both approaches.

Media Control Interface (MCI) Issues

Another restriction placed upon video is display location. For optimal playback, the video must be horizontally aligned on a 4-byte pixel boundary. (Meaning that the distance from the left edge of the display should be equally divisible by 4). If this is not the case, the playback rate could slow 2 times or more. Windows 95 and some applications actually shift the video during run-time to ensure that it lies along an appropriate boundary, but this can not always be relied on. For best performance, the programmer should ensure that this is always the case, instead of leaving it up to the system.

Visual Basic 3.0

Despite its convincing attributes, VB 3.0 is not the perfect development environment. Due to the fact that VB is interpreted, performance concerns can arise when comparing VB to a development tool that can build true executables. VB 3.0 code also depends on the VB run-time DLL (vbrun300.dll). This DLL is just under 400K and adds some bulk to distributable VB programs, especially simple utilities. VB 3.0 also lacks the benefits of an object-oriented tool. Common OO concepts like encapsulation, inheritance, and polymorphism are not part of the environment. This is not to say that VB doesn't enjoy code reuse or other OO benefits. VB is better thought of as object based and not object oriented. A lot of good insight into the object world can be gained from VB experience, but the true OO functionality is not present.

As relates to multimedia development, VB 3.0 is a solid tool. Despite is interpreted nature, the tests conducted showed VB 3.0's graphical performance always at least slightly ahead of Delphi. A possible reason for this fact might have to do with Microsoft's understanding of Windows95 and how to really push the graphic limits. Another reason might be that the graphic calculations performed in the demos were minimal. More computationally intensive programs would likely shift the performance edge to Delphi.

VB 3.0's standard timer control lacks the resolution to make it an effective tool for animation control. The availability of high resolution timer VBXs, though, as well as the ability to skip the use of timers altogether, makes this a non-issue when comparing to Delphi. Numerous VBXs also exist to enable other multimedia functionality such as transition effects and animation.

The tests conducted found VB 3.0 to excel at handling a lot of graphic activity. This reduces to the ability to handle a large number of BitBlts (bit block transfers) to on- and off-screen device contexts. The one major drawback to graphic intensive work in VB 3.0 is the familiarity the programmer must have with the Windows API. VB 3.0 provides an easy way to call API functions, but the programmer must still know what to do with them. The programmer must also contend with palette control. This becomes a problem if you need to integrate external content into your multimedia application. These aspects of VB 3.0 add a bit to the learning curve. The important thing to note is that these facts do not restrict the end result. Multimedia development in VB 3.0 adds some complexity over standard VB 3.0 development but the compensation comes in the resulting performance.

Visual Basic 4.0

VB 4.0 adds ease to OLE development. It is simple to integrate OLE servers (i.e. an Excel spreadsheet, a Word document) into an application. Microsoft's view of object orientation (and code reuse) seems centered around OLE objects. VB 4.0 also introduces the capability to easily create OLE servers. Minimal time was spent investigating OLE server development, but a simple example was created in little more than an hour. The environment lends itself well for these tasks.

Other improvements pertain to inclusion of some useful object oriented features. Class modules are introduced to provide encapsulation for code and data. Also added are collections. Collections enable objects to be grouped together. Collections simplify code as actions can be performed on the collection as a whole. These features help extend the nearly non-existent object orientation of VB 3.0. They provide good tools to aid in code simplification and reuse. Despite the improvements, lack of direct support for inheritance and polymorphism make VB 4.0's object orientation incomplete.

Multimedia performance and capabilities appear unchanged in VB 4.0. If anything, performance of VB 4.0 applications was slightly slower than their VB 3.0 counterparts. The lack of VBX support excluded the demos with timers from being run under the 32-bit version. No high resolution timer OCXs were found, though no considerable effort was spent to find them. As in VB 3.0, the programmer still has to get down into the Windows API for most graphic functionality. The results still make this a worthwhile effort.

Delphi 1.0

The choice of object Pascal as the base language also aids the development process. Object Pascal is fully object oriented (inheritance, polymorphism, encapsulation) without some of the complexity of C++. It is a powerful language, yet is relatively easy for the novice programmer to pick up. With the assistance of integrated debugging facilities, problems are efficiently found. Additionally, support for OLE 2.0 and VBXs, extend the inherent functionality of Delphi.

Perhaps the most useful feature separating Delphi from the versions of VB is the ease of extending the visual component library (VCL). New controls can be coded (in Delphi) and added to the development environment. This process is surprisingly easy and straightforward. This is where the true object oriented nature of Delphi outshines the object based concepts of VB. Custom components (objects) can be developed in the environment and reused as easily as any other visual component. For the multimedia tests performed, a stationary sprite component (class) was built and added to the VCL. Later, a mobile sprite component (class) was added as a subclass of the stationary sprite class. This too was added to the VCL. This provides an efficient way to reuse code, as well as a simple way to develop powerful applications with the speed of a visual development tool.

Besides the simplicity of building new objects, Delphi has other features that assist multimedia development. The standard timer component has better resolution than that provided in VB. The resolution, though, is still not as good as available VBXs. Delphi also does an excellent job hiding a lot of the lower level graphics chores. For the tests developed in Delphi, palette switching was controlled without annoying palette flashes. Colors were not mangled as they were in VB (ex. if a video had a different palette than a bitmap). In addition to handling palette functions, Delphi wraps the Windows API calls in its own functions. The programmer can make calls to the API if necessary, but the demo applications showed minimal performance benefit in doing so.

Handling off-screen bitmaps is a good example of the difference between Delphi and VB in this regard. VB requires numerous calls to API functions to get device context handles, build and realize palettes, and set bitmap bits. This requires an in-depth knowledge of the API as well as system operation. The same process was handled in Delphi by declaring an object of the bitmap class and telling the object to load the appropriate bitmap file.

Unfortunately, the resultant graphic performance of Delphi (as experienced in the tests performed) was surpassed by VB. Most of the Delphi tests produced acceptable results, but when compared to VB, the difference was apparent. Despite this finding, Delphi's integrated visual environment, true object nature and extensibility make it a solid programming tool for all Windows application development.

Delphi32

ability to share .obj files between Delphi and Borland C++ projects

full OLE support including:

OCX support (and the ability to subclass OCXs as well as create your own in Delphi)

automation control

automation server construction

improved compiler speed

faster, smaller executable files

Conclusions

For purely performance reasons, VB 3.0 tested the best. It is by no means the best programming environment but for producing smooth animation it does the job. The wealth of multimedia VBXs is also a definite plus. The downside of VB 3.0 pertains mostly to development environment. Lack of OO support and the need to use the Windows API (for graphics work) make VB 3.0 less desirable in the long run. VB 3.0 would be the choice when developing multimedia applications that are light on processing content (i.e. demos). Simple decision logic and event handling can be handled very well with a fair amount of graphic activity.

VB 4.0 performed slightly slower than VB 3.0 throughout the demos. It brings along the benefits of VB 3.0 with the major addition of OLE. Depending on the need for OLE integration, this is a good or bad thing. The OLE extensions allow OLE servers to be coded in VB fairly easily. The ability to use external OLE servers is even easier. The down side is that if the use of OLE in your multimedia application is not needed, you still carry around supporting OLE architecture, which adds considerable bulk to distributable applications.

The OO extensions added to VB 4.0, specifically class modules, attempt the enhance the reusability of code but the object nature is still lacking. The lack of access to VBXs (in the 32-bit version) is perhaps, the largest drawback. Many multimedia developers have adopted certain VBXs as part of the programming toolkit. Until these VBXs can be brought over into the new OCX format, multimedia programmers might find it best to stick with VB 3.0.

If it were not for its slower (and sometimes erratic) graphical performance, Delphi would be the clear choice for multimedia development over VB. The object Pascal base and the excellent, extensible environment make Delphi a capable development tool. Delphi also lightens the load on the programmer, handling palette operations and providing useful graphic functions the eliminate the need to call Windows API functions. Delphi has to be given serious consideration if the final application will involve a large amount of processing. For large applications needing basic interactive multimedia, Delphi is a very solid tool.

Demo Outline

1) Bitmap Scrolling

The scrolling effect was created simply by initially loading the bitmap to an off-screen device context. To scroll, a section of the off-screen bitmap is copied to the display for each frame. Shifting the starting point of the copy routine creates the scrolling effect. This approach was preferred over creating the entire bitmap within a form (as a control) and then shifting it back and forth on the form.

VB comments:

Loading in the off-screen was not as trivial as it should have been. Code was used that decomposed the bitmap file, built and realized the new palette, and then set all of the bits in the device context appropriately. Knowledge of the Windows API was essential for this code.

Delphi comments:

Overall, scrolling large bitmaps in the two applications was comparable, but VB definitely has the edge on consistent smoothness. Creating the off-screen bitmap in Delphi was considerably more straightforward. The process involved creating a bitmap object and calling a method to load the file into it.

2) Scrolling w/ Off-window Video

VB comments:

Delphi comments:

3) Scrolling w/ On-window Video

VB comments:

Delphi comments:

4) Scrolling w/ On-window Video

VB comments:

Delphi comments:

5) Multiple Videos

For this test, VB and Delphi perform similarly. Playback of both videos isn't of great quality but it is acceptable. When transitioning between the two video windows, the video losing focus will suffer a pause of a second or more. The video playing in the window receiving focus appear unaffected by the change.

6) Multiple Videos

As in demo 5, performance between VB and Delphi was similar. Running two videos at one time produced adequate results. Playing three or four at one time resulted in noticeable frame skipping, particularly in the larger video windows.

7) Stationary Sprites

VB comments:

Delphi comments:

8) Moving Sprites

VB comments:

Delphi comments:

9) Integrated Sprites

VB comments:

Delphi comments:

The test built without a timer was significantly faster than using a timer. Additional sprites were handled by this version with reasonable sprite performance as well as solid scrolling. Palette problems existed in this version. When the sprites are not animated, the colors appear normally. Whenever the sprites are animated, the background bitmap loses the proper mapping of its colors. This problem was a surprise as Delphi had handled prior palette conflicts without a noticeable problem.

10) Integrated Sprites and Video

VB comments:

Delphi comments: