Our application is called "funanbulle". The goal of the application is to allow families to create their own micro virtual world and gather. We wanted to show what such virtual worlds would look like. The idea was to enable people to share and chat in real time with a fun and engaging user experience.

funanbulle, Aerys' #air24h challenge contribution

The application is nothing more than a proof of concept. If we had enough time, we would have added lots of feature like:

• Audio chat
• Photos and videos sharing
• Interactive objects to trigger applications (games, sharing applications, etc...)

In the end, we had just enough time to build a 3D chat. But I think it was a lot of fun and it looks really nice! Here is a quick video to show what funanbulle is about and how it works:

This video was made by Michael Chaize to show the 14 applications created during the contest.

To install the application, just click on the following picture to go to the install badge:

When the application starts, you will be asked to "join" or "create" a room:

• If you don't have a room number: select "create a room" to create a new room. When the room is created, you'll join in automatically join it. The room number will be visible in the top right corner. You can share it with your friends and family to enable them joining your room.
• If you have a room number (because a friend already created a room and gave you its number for example): select "join a room", type your room number and click "Go". And voilà!

You can use the bottom text input to chat with your friends. But you can also launch special actions by typping one of the following commands:

• /jump
• /wait
• /sit
• /use
• /wave

We hope you'll like it and have fun with it!

In this post I will:

• Explain how any 3D scene is built when using Minko
• Explain how pixel shaders are integrated in the 3D scene
• Explain how pixel shaders are built using Pixel Bender
• Show you a very simple demo of the kind of effects pixel shaders will provide
• Explain how the demo was built

Here are a two screenshots to show the results:

Phong shading + spheric environment mapping on a 2700+ polygons Lamborghini

Phong shading + spheric environment mapping demo

The Scene Graph API

This API is used to build and manage the 3D scene. Scene graph is the most common data-structure used to build 3D scenes. It is simple to understand and both memory wise and CPU efficient. To explain what the scene graph is, start thinking about Flash's display list. In ActionScript 3.0, the display list is actually a tree. The root of the tree is the Stage and you can add multiple children to it. Each child can be a leaf, such as a Bitmap object, or a container such as a Sprite. One of the limitations of a tree is that every node can have multiple children but each child can have only one parent. This property is what makes it look like a tree: branches (and leaves for that matter...) have only one parent branch.

A direct consequence of that paradigm is that if you want 20 red circles, you will have to create 20 red circles and add them to the display tree. Despite the fact that those 20 red circles will be exactly the same you will still have to create and add all of them to the display tree. Thus, you will use 19 times more memory than what is actually needed. That's where the graph kicks in...

A graph is just like a tree except every node can have multiple parents. You can see graphs as a generalization of trees. But you can also say that trees are particular graphs where each node can have only one parent. Anyway, we can now create one red circle and use it as many times as we need! To understand how this apply to a 3D scene, lets consider the following rendering:

And here is the corresponding scene graph:

There are five things to notice:

1. The figure doesn't show the whole graph (I removed the camera and materials).
2. The Scene Graph API is all about interfaces. You can add new features to the API and still extend the classes you need/want (ie. EventDispatcher).
3. The CubeMesh.cubeMesh, SquareMesh.squareMesh and SphereMesh.sphereMesh nodes, which contain the primitives geometry, exist only once each but are used by many IDisplayObject3D. Therefore there is no useless memory consumption! The same can be done with any node in the graph: containers, display objects, materials, etc...
4. The orange circled nodes apply a (3D) transformation to their children, just like Flash containers apply (2D) transformations to their own children.
5. Z-Sorting is here enabled by two different technics: the drawing order (which is pretty much the same as in a classic-2D-Flash display tree) and the DepthSortContainer3D node. This node takes care of z-sorting by sorting its children using their average depth. This way, z-sorting occurs only when needed and only on relevant geometry.

There a lot to explain about the Scene Graph API! But the main point is to understand how it compares to Flash's display tree and what benefits it provides: lower memory consumption and powerful scene building. You are likely to need to find a way to fit in the Scene Graph API anytime you want to extend Minko to add a new feature. In this precise case I will talk about how pixel shaders fits in.

Pixel shader integration

The first step is to find "where" in the graph your new feature will be used. Close to the scene "root" you will more likely find containers. And the "leaves" are often more tangible objects such as a materials or meshes. A pixel shader is some kind of a material so that's where we should start looking!

In the Scene Graph API, there is a special way to handle materials: the "material stack". It's a part of the graph - a subgraph really - that looks very much like a stack because every node uses an "underlaying" material node. Lets consider the orange box from the previous example:

This figure shows the orange cube subgraph. On this graph I added the material nodes to demonstrate how the material stack works. In this precise case, the stack is made of a WireframeMaterial node and a SolidMaterial node. The first one provides a wireframe rendering and the second one is a solid fill. When rendering, the resulting graphics operations will be the list of all the graphics instructions outputed by each node of the stack. Thus, we will here obtain an (orange) solid fill with a (purple) wireframe. So where does the pixel shader fits in the material stack? Well you might have noticed there are two different kinds of nodes in the material stack:

1. the bottom of the stack is more likely to be a "base material", like a solid fill or a texture
2. and the rest of the stack will be mainly "modifiers". In the case of the orange cube, the WireframeMaterial node acts as a modifier for the SolidMaterial node

Our pixel shader material will be a modifier because it will work on an underlaying texture. Now we know our pixel shader material should be on the top of the material stack we have to know what data it needs and how to provide it!

Pixel shader creation using Pixel Bender

A pixel shader works at the pixel level (true story). Therefore, we need per-pixel data. But because we use Flash's drawing API we don't have a direct access to the actual pixels (I'm oversimplifying but that's the idea). The trick is to work on the material BitmapData itself rather than the screen pixels. Thus, I will now talk about "texels" (texture pixels) rather than pixels (so I guess it should be called a "texel shader" then...). Anyway, our shader will then most certainly always need three inputs:

1. The color of each texel, also called a "diffuse map" or... a texture. Any material will do the trick (we can get a BitmapData out of any material).
2. The position of each texel, also called a "position map".
3. The normal of each texel, also called a "normal map".

The position map will be generated at runtime. It is mainly a BitmapData where each pixel stores the (x, y, z) values of the position of the corresponding texel instead of an (r, g, b) color value. The normal map can be generated at runtime. It can also be any material as long as it provides a relevant BitmapData. This BitmapData stores the (nx, ny, nz) values of the normal of the corresponding texel. Here is an example:

In this example both the normal and the position maps have been generated using Minko's NormalMap and PositionMap classes. This picture is actually a screenshot of a demo Flash application. I wont talk too much about this generation step for now, but its done using Pixel Bender and its lighting fast!

In order to make this data available to any pixel shader, the diffuse, normal and position maps are stored in a one-and-only scene graph node: the PixelMaterial. This node will then provide per-texel data access to its modifiers. We can now implement a real pixel shader on top of any PixelMaterial! Everything we need now is a Pixel Bender kernel that takes our 3 maps (diffuse, position and normal) in input.

The following Pixel Bender filter performs Phong shading:

void
evaluatePixel()
{
        float2 o = outCoord();
        float3 position = sampleNearest(positionMap, o);
        float3 normal = sampleNearest(normalMap, o) * 2. - ONE;
        float3 light = normalize(light - position);
 
        // ambient
        float l = ambient * intensity;
 
        // diffuse
        l += max(0., dot(light, normal)) * diffuse * intensity;
 
        dst = sampleNearest(diffuseMap, o) * float4(l, l, l, 1.);
}

As you can see its really straight forward. This is due to the Scene Graph API which makes it possible to do every computation in local space. Therefore we don't have to care about where those computations happen in the 3D scene.

Demo

Phong shading or reflection mapping are great examples of pixel shaders. The following demo uses both (the white diamond stands for the light source):

Phong shading + spheric environment mapping demo

Phong shading + spheric environment mapping scene graph

And to put it in a nutshell, a few performances considerations:

• Everything is done in real time.
• Thanks to the Scene Graph API, every computation is done in local space. It avoids matrix multiplications by the thousand and gives a significant performance boost.
• The application takes less than 20Mb of memory and weights less than 100Kb.
• There are a total of 6 pixel shaders run at each frame (only the visible faces materials are computed and there are 2 shaders per face).
• Those shaders work on 128*128 pixels diffuse/normal/position maps
• That's a total of 98,304 pixels per frame
• Each frame is "processed" in an average of 10ms. (Core2 Duo U9400 @ 1.4Ghz, Flash 10.1 RC7, Chrome 5)
• That's 9,830,400 pixels per second!
• There is a ~30% performance boost with Flash 10.1 RC7
• The number of triangles does not impact the performances
• Do not hesitate to post your own results (with your configuration of course)!

]]> http://blog.promethe.net/2010/06/09/pixel-shader-demo-using-flash-10-pixel-bender-and-minko/feed/ 11 http://blog.promethe.net/2010/06/01/video-of-the-minko-presentation-at-the-ttfx-barcamp/?utm_source=rss&utm_medium=rss&utm_campaign=video-of-the-minko-presentation-at-the-ttfx-barcamp http://blog.promethe.net/2010/06/01/video-of-the-minko-presentation-at-the-ttfx-barcamp/#comments Tue, 01 Jun 2010 12:42:12 +0000 Promethe http://blog.promethe.net/?p=747 Here is a link to my first official presentation of Minko, the 3D library I am working on (and which was previously called "DirectFlex"):

This video was shot the 23rd of March by the people from BAAO. A very big thank you to Yann Chevalier, who's leading the TTFX effort, Michael Chaize, our awesome french Flash platform evangelist and Olivier from BAAO who gave me the link to the video.

You can find the slides here.

Anyway, as the whole HTML5 versus Flash battle is raging, it appears Flash relatively poor performances are indeed criticizable at best. Beside poor developers, the Flash Platform suffers from a very very slow software renderer. Or at least much slower than the rest of the platform. It's no secret: hardware acceleration is a key feature for the future of the Flash Player. It's even hard to believe it is not available yet!

As you must already know, Flash 10.1 will support OpenGL ES 2 on mobile devices to leverage the lack of CPU horsepower. While hardware HD video decoding will be available on the desktop too, the drawing API will only be accelerated on mobile devices.

Anyone knowing a bit about OpenGL ES knows it is a subset of OpenGL. Thus if it works with OpenGL ES, it should work with OpenGL. With this in mind, a few questions:

1. Why isn't Flash 10.1 drawing API hardware accelerated on any OpenGL capable platform, including the desktop?
2. How will it work?
3. Will Pixel Bender be hardware accelerated?

1. Hardware Accelerated Desktop Flash Player

Let's face it: there must be an hardware accelerated desktop Flash Player in the works. As I said previously, OpenGL supports all features of OpenGL ES so this is not far fetched at all. Yet, it is neither released nor announced. Why?

My first guess is OpenGL provides with a lot of features that would make the desktop experience a lot smoother than just using what offers its little brother OpenGL ES. So at some point Adobe had to make a choice :

• release a fully hardware accelerated Flash 10.1 on both mobile and desktop platforms, with the last one being very far from what desktop hardware is actually capable to handle
• or release Flash 10.1 focusing on mobile devices and announce hardware acceleration for the desktop just after its the final release... and I'm guessing it might be a key feature of Flash 11

When I was at the French Flash User Group (TTFX - les TonTons FleXeurs) a few months ago, I spoke with Lee Brimelow and Mike Chambers about the just announced microphone raw-data access. I asked them why it was announced only for AIR 2.0 and not Flash 10.1. The answer was something about the "quality guys" making sure the feature was well suited on both the roadmap and the logic of the incoming updates. And this feature eventually made its way into the Flash Player! I think that's what is happening with hardware acceleration on the desktop.

But if you don't believe me, you don't have to take my word for it! What about Adobe's word? Cnuuja, one of the engineer working on the Flash Player, posted this very message on the Flash 10.1 Forum:

Can OpenGL 3.3/4.0 improve Flash 10.x ?

"Yes....  with a lot of work.   We have spent the last year writing new code which allows OpenGLES2 to render flash content on mobile devices.  Performance varies significantly from one gpu to the next, with some gpus being slower than the software renderer.   What Flash does is significantly different from the 3d triangles+shaders GPUs were designed to support. Its a lot of work to make OGL/D3D usable as our renderer, but we're working on it 

-chris"

2. How will it work?

Just like in Flash 10.1 for mobile devices. But much faster thanks to OpenGL and Direct3D.

The internals of such feature is very important: developers must know and understand how it works to make the best out of it. Adobe already talked about how the drawing API is accelerated in Flash 10.1 on mobile devices:

"When a GPU renders vector graphics, it breaks them up into meshes made of small triangles before drawing them, a process called tesselating. There is a small cost to doing this, which increases as the complexity of the shape increases. To minimize performance impact, avoid morph shapes, which must be retesselated on every frame."

It's straight forward and I think it's actually the best (and only...) way to do it. Tesselation will create triangles by computing sets of vertices/indices (also called Vertex and Index Buffers) and push them to the graphics hardware. The end of the quote suggests such data is cached and should not be recomputed if no redraw occurs.

Something very important though: z-sorting. People might think hardware acceleration implies z-sorting. But it doesn't. When you know how 3D hardware and APIs work, you know it will be very tricky to make it work with something as general purpose as Flash. If Adobe wants to use the z-buffer, they will have to cut the compatibility with the software renderer. And I don't think this will happen anytime soon.

3. Hardware Accelerated Pixel Bender

Pixel Bender is already hardware accelerated pretty much everywhere except the Flash Platform. I'm not sure why. Still, it's hardware accelerated in other products of the Creative Suite so I guess that an OpenGL/Direct3D shader languages compliant intermediate represenation of Pixel Bender kernels does exist.

This said, it's just a matter of how to make it work with the very general purpose Flash Player. Considering Flash 10.1 is using tesselation, my guess is pixel shader should follow quite easily.

Optimization is always important. But when it comes to 3D for the Flash Platform, it's an everyday battle. The first ideas that come to mind are to avoid:

1. redrawing the same regions : each pixel value must be set once and only once
2. rendering invisible objects : objects that are out of sight still take a lot of CPU horsepower

While Flash takes care of the first one in its very renderer, the second one is not handled. But that is something we can easily address!

The Technic

The method is called "frustum culling". The big picture is that every mesh is approximated using a bounding volume (typically a sphere or a box). If the bounding volume is visible, the corresponding mesh is rendered. The two following pictures show the frustum culling caught in action:

The mesh is visible: TPS counter indicates 18900 triangles per seconds

The mesh is out of sight: frustum culling is acting and TPS counter indicates 0!

1. The frustum

First things first: before performing frustum culling, you need to build the frustum! The frustum is the 3D shape that contains everything the camera can see. It's made of 6 planes and looks like a truncated pyramid:

The view frustum

Each plane is defined by a plane equation: ax + by + cz + d, where (a, b, c) is the normal of the plane and d the distance on that normal. The 6 planes must be extracted from one of the Matrix3D object used to define the transform from one space to another (world to camera for example). Here are the 6 plane equations:

• left: (a = m14 + m11, b = m24 + m21, c = m34 + m31, d = m44 + m41)
• right: (a = m14 − m11, b = m24 − m21, c = m34 − m31, d = m44 − m41)
• bottom: (a = m14 + m12, b = m24 + m22, c = m34 + m32, d = m44 + m42)
• top: (a = m14 − m12, b = m24 − m22, c = m34 − m32, d = m44 − m42)
• near: (a = m13, b = m23, c = m33, d = m43)
• far: (a = m13, b = m23, c = m33, d = m43)

The source matrix you chose defines the coordinates systems of the extracted planes:

• the projection matrix provides planes in view (or camera) space
• the view * projection matrix provides planes in world space
• the world * view * projection matrix provides planes in object (or local) space

2. Bounding volumes

Testing every polygon against the frustum is a very expensive task. In order to speed things up, we will rather test a primitive shape that bounds the mesh: a "bounding volume". Common bounding volumes are spheres or boxes. I'll focus on the first one in the rest of the article.

A bounding sphere is pretty simple to implement: you just need to store the center of the sphere and it's radius. That being said, a BoundingSphere class would just contain the center as a Vector3D and the radius as a Number. When you have all the vertices of a mesh, building the bounding sphere is easy :

public function computeBoundingSphere(myVertices : Vector.<Number>) : BoundingSphere
{
	var min : Vector3D = new Vector3D(Number.MAX_VALUE, Number.MAX_VALUE, Number.MAX_VALUE);
	var max : Vector3D = new Vector3D(Number.MIN_VALUE, Number.MIN_VALUE, Number.MIN_VALUE);
	var length : int = myVertices.length;
	var center : Vector3D = null;
	var radius : Number = null;
 
	for (var i : uint = 0; i < length; i += 3)
	{
		min.x = myVertices [i] < min.x ? myVertices[i] : min.x;
		min.y = myVertices [uint(i + 1)] < min.y ? myVertices[int(i + 1)] : min.y;
		min.z = myVertices [uint(i + 2)] < min.z ? myVertices[int(i + 2)] : min.z;
 
 		max.x = myVertices [i] > max.x ? myVertices [i] : max.x;
		max.y = myVertices [uint(i + 1)] > max.y ? myVertices[int(i + 1)] : max.y;
		max.z = myVertices [uint(i + 2)] > max.z ? myVertices[int(i + 2)] : max.z;
	}
 
	center = new Vector3D((myMax.x + myMin.x) / 2.0, (myMax.y + myMin.y) / 2.0, (myMax.z + myMin.z) / 2.0);
        radius = Math.max(myMax.x - center.x, myMax.y - center.y, myMax.z - center.z,
		          center.x - myMin.x, center.y - myMin.y, center.z - myMin.z);	
 
	return new BoundingSphere(center, radius);
}

3. Point to plane distance

The plane equation (ax + by + cz + d) can be stored as a Vector3D object. Indeed, the Vector3D implements x, y and z but also a w Number attribute. This last one is supposed to store the homogenous coordinate. Nonetheless, we can use those 4 Number attributes to store the (a, b, c, d) values that defines our plane equation.

In order to test whether a bounding volume is inside or outside the frustum, we must be able to compute the distance of this object to the plane. To do so, we use the dot product of the point with the plane normal (which must be normalized first) :

public function planeToPointDistance(myPlane : Vector3D, myPoint : Vector3D) : Number
{
	return myPlane.x * myPoint.x + myPlane.y * myPoint.y + myPlane.z * myPoint.z + myPlane.w;
}

4. The test function

Now that our frustum and our bounding volumes are ready, we can add this little test function that checks whether an object is visible or not:

public function testBoundingSphere(myPlanes      : Vector.,
                                   mySphere      : BoundingSphere) : Boolean
{
	var length : int = myPlanes.length;
 
	for (var i : uint = 0; i < length; ++i)
	{
		var d 	: Number = planeToPointDistance(myPlanes[i]);
 
		// bouding sphere is out of the frustum
		if (d + mySphere.radius < 0.0)
			return false;
		// bounding sphere is spanning
		else if (d - mySphere.radius <= 0.0)
			return true;
	}
 
	// bounding sphere is inside the frustum
	return true;
}

The Experiment

The following experiment demonstrates frustum culling in action (click on the picture to open the application) :

Frustum culling experiment (use the arrow keys to move)

Use the arrow keys to move the robot across the scene. You can notice the TPS counter drops down to zero whenever the character is out of sight (on the near left or near right side of the screen). When the TPS indicates 0, no triangles are rendered and CPU usage drops down to 0%. It actually means that the rest of the rendering process has a minimal overhead!

And it's even more awesome...

This experiment also introduce my brand new mini-debugger inspired of Mr. Doob's work. I replaced the maximum memory footprint counter with a per-second rendered triangles counter (or TPS counter). CPU usage or framerate alone are not absolute measurements but the TPS counter reflects the true performance of the application the old fashioned way: the more polygons per second the better. I also updated the timer to display the processing time and the overall rendering time.

You can compare this experiment to the "old" one I made up a few weeks ago: the displayed 3D model features 30% more polygons but the CPU usage is the same.

The rendering process is done in such a way that absolutely no computation occurs on invisible objects, not even the slightest animation or z-sorting operation.

I enabled dynamic bounding volumes for animated meshes: if you place the mesh on the edges of the viewing space, you might notice that the TPS counter sometimes goes down to zero to raise up again to a few hundreds/thousands of triangles per second. The explanation is simple: the visibility test is done dynamically and follows the mesh animations. If the animation drives the mesh out of the viewing frustum, culling operates and it is not renderered.

Augmented reality (AR) is a term for a live direct or indirect view of a physical real-world environment whose elements are merged with (or augmented by) virtual computer-generated imagery - creating a mixed reality.

Sounds a bit blury? Well... I'll try to make it clearer with a demo...

Demo

First, you will have to print a little black and white "marker". The AR software scans the webcam picture and look for this very marker. When it is found, its 3D position, rotation and scale are computed and used to embed a 3D object. You can found the marker here and it looks like this:

FLARToolkit Marker

Then, launch the following application and place the printed marker in front of your camera:

FLARToolkit demo application (webcam required)

What it does...

• Load a MD2 file and a JPEG texture
• Loop through the animation of the model
• Define multiple materials (press "w" to switch)
• Use the FLARToolkit library for the AR
• Translate FLARToolkit matrices to fit the 3D API

How it works

• The viewport and a dummy camera are initialized using the camera parameters and the size of the capture
• The webcam image is binarized using a threshold filter: aeras that are darker than a threshold value end up completely white on a black background. Areas that are too small or too big are rejected.
• Each remaining area is detected and its bounding box and position are computed
• For each area that "looks like" a square, the data of what should be the pattern is extracted
• This data is then compared to the actual pattern
• If the match confidence is above a certain value (80% is this case), a matrix that contains the position/rotation/scale of the center of the marker is computed
• This matrix is converted into a native Matrix3D object that can be used as the world transform for the 3D objects we want to display

What it does not do...

• Use FLARManager to handle the AR. Lee "The Flash Blog" Brimelow juste made this excellent tutorial about FLARManager: thank you Lee!

Known Issues

The FLARToolkit library is very CPU intensive! I looked into it and beside the fact that it is poorly engineered, it looks like the blob extraction algorithm is the bottleneck. Sadly, there is not much to do about it...

It was ported from a Java port of a C++ library called ARToolkit. I never read the original C++ code or its Java port, but the AS3 version seems to be just a straight forward rewriting without much thinking. AS3 native types are totally ignored, if not re-implemented, and the design clearly perspires C++ (and not only because the original code is embeded in comments all over the place...). FLARManager apparently leverages the lack of a proper API.

To put it in a nutshell, FLARToolkit is an impressive tool but it needs a big cleanup!

Credits

• Framerate counter by Mr. Doob
• Augmented reality using FLARToolkit by Saqoosha

Details, pictures and a demo application right after the jump...

MD2 file format

Quake 2 uses the MD2 file format  to store animated 3D models. It is a quite simple file format: each MD2 file stores a single mesh and its animations. Each animation is just copy of the original mesh with different positions for each vertex. The number of vertices and the indices are the same for each frame. Therefore, the MD2 file format is easy to adapt to the Flash Platform because no complicated shaders/bones are required to animate the model: you just have to display the right vertices!

But it is not THAT simple. Each animation is actually subdivided into frames (vertices really). Because storing a fluid animation would take a lot of frames (and a lot of memory), only a few keyframes are stored. When rendering, one must compute the interpolation coefficient between the current and the next keyframe and use it to get the interpolated vertices. The following code does it all:

var frame1	: Vector.<Number>	= _currentFrame.vertices;
var frame2	: Vector.<Number>	= _nextFrame.vertices;
var c2		: Number		= _frame - int(_frame);
var c1		: Number		= 1 - c2;
var length	: Number		= frame1.length;
 
for (var i : uint = 0; i < length; ++i)
	_vertices[i] = frame1[i] * c1 + frame2[i] * c2;

... and here is the demo application :

Flash 10 MD2 loader

What it does...

• load Quake 2's MD2 files
• basic z-sorting using faces barycenter
• add a "frame" property to the mesh to handle animations
• interpolate and update vertices only when necessary
• wireframe material (press "w")
• "free chase" camera (mouse click + drag'n'drop to turn around the model)

What it doesn't do...

• use BSP trees to handle z-sorting (1 BSP tree / frame takes a lot of time and memory...)
• loop through a specific animation
• read Quake 2's animation files

Known issue(s)

None. If you find one please let me know! Performance seems to be very good actually...

There are so many reasons why loading Quake 2 files inside the Flash player would be considered as just "impossible". Performance would be the first and main one. One would just consider that Flash is not fast enough to display complex 3D graphics, not even those of a game as old as Quake 2 (published in 1996!). But using Flash 10 latests features such as the Vector and GraphicsTrianglePath classes, it is actually quite doable!

Details, pictures and a demo application right after the jump!

Quake 2's maps file format

But the really big advantage here is inside the Quake 2 maps file format itself. Each map is a *.BSP file and the BSP file format might be the best ratio in terms of ease of implementation against performance optimization. As you might have guessed, *.BSP files actually store a pre-compiled Binary Space Partitionning tree of the map itself. For those who don't already know about BSP trees, just imagine that it is actually the best known method to handle a polygon-perfect z-sorting in Flash when dealing with 3D. But computing a good BSP tree of a large and complex 3D model such as a map takes a lot of time... that is why having a pre-compiled and ready to load BSP tree is just great!

You can find a lot of good documents on the Quake 2 BSP file format. The one hosted on flipcode is quite complete and easy to understand. It is actually the one I used to create my BSP loader. Reading the file formats specifications shows that *.BSP files also include "visibility information". This data is also very important because it makes the user able to know which area of the map is visible from any other area. This way, only the areas visible from the area where the player stands have to be displayed which is an incredible performance boost!

The following application is a proof of concept of a Flash powered Quake 2 *.BSP loader :

Click on the picture to launch the demo application.

Here are some screenshots of the same application loading much more complex maps (those screenshots have been made with the Flash debugger which explains the higher memory consumption) :

actcity2

aqnitro

What it does...

• load Quake 2's *.BSP files
• build the corresponding geometry (vertices, indices and UV data) using Flash 10 native types such as Vector and GraphicsTrianglePath
• frustum culling and clipping using bounding boxes and bounding spheres
• mouse controlled "free chase" camera (click + drag'n'drop to look around, mouse wheel to zoom in/out)

What's next...

• use the embed BSP data to handle z-sorting
• use visibility information to display only visible clusters
• load WAL textures (Quake 1 and 2 textures file format, already done just have to post it )  and use them properly to display the map
• load MD2 animated 3D models (again, already done and just have to post it!)

Known issue(s)

If you zoom enough to place the camera "inside" the map geometry, you will notice a severe framerate drop down. It is very logical since frustum clipping will have to check and eventually clip each polygon of the geometry against the relevant view frustum planes. Because visibility information tells us what is actually visible or not, handling it properly should offer a huge performance boost: instead of having to clip the whole geometry, only the visible areas will be treated.

• FPS: is it possible to display Half Life 1 graphics in Flash ? I don't know yet... but this experiment is an improvement of the one I posted a while back
• Earth: I always wanted to implement the famous Google Earth in Flash. This might be a good first step! I made this experiment in approximately 1 hour to test out a few new features

Please let me know what you think about those new experiments. Any feedback will be greatly appreciated!