funanbulle, our #air24h application!
The Adobe 24H Challenge was last friday and the application the 14 teams created are already online. You can see all the available applications on the official website.
Our application is called "funanbulle". The goal of the application is to allow families to create their own micro virtual world and gather. We wanted to show what such virtual worlds would look like. The idea was to enable people to share and chat in real time with a fun and engaging user experience.
funanbulle, Aerys' #air24h challenge contribution
The application is nothing more than a proof of concept. If we had enough time, we would have added lots of feature like:
- Audio chat
- Photos and videos sharing
- Interactive objects to trigger applications (games, sharing applications, etc...)
In the end, we had just enough time to build a 3D chat. But I think it was a lot of fun and it looks really nice! Here is a quick video to show what funanbulle is about and how it works:
This video was made by Michael Chaize to show the 14 applications created during the contest.
Pixel shader demo using Flash 10, Pixel Bender and Minko
I'm really excited to announce Minko (which is, by the way, the final name for my 3D library) has reached a new level: pixel shader integration! Pixel shaders are little programs that run on each pixel and can modify their final color. They are often written in C-like languages and in this precise case we use Pixel Bender, the shader language introduced with Flash 10.
In this post I will:
- Explain how any 3D scene is built when using Minko
- Explain how pixel shaders are integrated in the 3D scene
- Explain how pixel shaders are built using Pixel Bender
- Show you a very simple demo of the kind of effects pixel shaders will provide
- Explain how the demo was built
Here are a two screenshots to show the results:
Phong shading + spheric environment mapping on a 2700+ polygons Lamborghini
Phong shading + spheric environment mapping demo
Video of the Minko presentation at the TTFX barcamp
Here is a link to my first official presentation of Minko, the 3D library I am working on (and which was previously called "DirectFlex"):
This video was shot the 23rd of March by the people from BAAO. A very big thank you to Yann Chevalier, who's leading the TTFX effort, Michael Chaize, our awesome french Flash platform evangelist and Olivier from BAAO who gave me the link to the video.
You can find the slides here.
Flash 11 drawing API will be hardware accelerated
I love catchy titles. I know nothing about the next major version of Flash and all of this is just speculations.
Anyway, as the whole HTML5 versus Flash battle is raging, it appears Flash relatively poor performances are indeed criticizable at best. Beside poor developers, the Flash Platform suffers from a very very slow software renderer. Or at least much slower than the rest of the platform. It's no secret: hardware acceleration is a key feature for the future of the Flash Player. It's even hard to believe it is not available yet!
As you must already know, Flash 10.1 will support OpenGL ES 2 on mobile devices to leverage the lack of CPU horsepower. While hardware HD video decoding will be available on the desktop too, the drawing API will only be accelerated on mobile devices.
Anyone knowing a bit about OpenGL ES knows it is a subset of OpenGL. Thus if it works with OpenGL ES, it should work with OpenGL. With this in mind, a few questions:
- Why isn't Flash 10.1 drawing API hardware accelerated on any OpenGL capable platform, including the desktop?
- How will it work?
- Will Pixel Bender be hardware accelerated?
1. Hardware Accelerated Desktop Flash Player
Let's face it: there must be an hardware accelerated desktop Flash Player in the works. As I said previously, OpenGL supports all features of OpenGL ES so this is not far fetched at all. Yet, it is neither released nor announced. Why?
My first guess is OpenGL provides with a lot of features that would make the desktop experience a lot smoother than just using what offers its little brother OpenGL ES. So at some point Adobe had to make a choice :
- release a fully hardware accelerated Flash 10.1 on both mobile and desktop platforms, with the last one being very far from what desktop hardware is actually capable to handle
- or release Flash 10.1 focusing on mobile devices and announce hardware acceleration for the desktop just after its the final release... and I'm guessing it might be a key feature of Flash 11
When I was at the French Flash User Group (TTFX - les TonTons FleXeurs) a few months ago, I spoke with Lee Brimelow and Mike Chambers about the just announced microphone raw-data access. I asked them why it was announced only for AIR 2.0 and not Flash 10.1. The answer was something about the "quality guys" making sure the feature was well suited on both the roadmap and the logic of the incoming updates. And this feature eventually made its way into the Flash Player! I think that's what is happening with hardware acceleration on the desktop.
But if you don't believe me, you don't have to take my word for it! What about Adobe's word? Cnuuja, one of the engineer working on the Flash Player, posted this very message on the Flash 10.1 Forum:
Can OpenGL 3.3/4.0 improve Flash 10.x ?
"Yes.... with a lot of work. We have spent the last year writing new code which allows OpenGLES2 to render flash content on mobile devices. Performance varies significantly from one gpu to the next, with some gpus being slower than the software renderer. What Flash does is significantly different from the 3d triangles+shaders GPUs were designed to support. Its a lot of work to make OGL/D3D usable as our renderer, but we're working on it 
-chris"
2. How will it work?
Just like in Flash 10.1 for mobile devices. But much faster thanks to OpenGL and Direct3D.
The internals of such feature is very important: developers must know and understand how it works to make the best out of it. Adobe already talked about how the drawing API is accelerated in Flash 10.1 on mobile devices:
"When a GPU renders vector graphics, it breaks them up into meshes made of small triangles before drawing them, a process called tesselating. There is a small cost to doing this, which increases as the complexity of the shape increases. To minimize performance impact, avoid morph shapes, which must be retesselated on every frame."
It's straight forward and I think it's actually the best (and only...) way to do it. Tesselation will create triangles by computing sets of vertices/indices (also called Vertex and Index Buffers) and push them to the graphics hardware. The end of the quote suggests such data is cached and should not be recomputed if no redraw occurs.
Something very important though: z-sorting. People might think hardware acceleration implies z-sorting. But it doesn't. When you know how 3D hardware and APIs work, you know it will be very tricky to make it work with something as general purpose as Flash. If Adobe wants to use the z-buffer, they will have to cut the compatibility with the software renderer. And I don't think this will happen anytime soon.
3. Hardware Accelerated Pixel Bender
Pixel Bender is already hardware accelerated pretty much everywhere except the Flash Platform. I'm not sure why. Still, it's hardware accelerated in other products of the Creative Suite so I guess that an OpenGL/Direct3D shader languages compliant intermediate represenation of Pixel Bender kernels does exist.
This said, it's just a matter of how to make it work with the very general purpose Flash Player. Considering Flash 10.1 is using tesselation, my guess is pixel shader should follow quite easily.
Frustum Culling in Flash 10
Update: corrected a few glitches in the bounding sphere creation routine.
Optimization is always important. But when it comes to 3D for the Flash Platform, it's an everyday battle. The first ideas that come to mind are to avoid:
- redrawing the same regions : each pixel value must be set once and only once
- rendering invisible objects : objects that are out of sight still take a lot of CPU horsepower
While Flash takes care of the first one in its very renderer, the second one is not handled. But that is something we can easily address!
The Technic
The method is called "frustum culling". The big picture is that every mesh is approximated using a bounding volume (typically a sphere or a box). If the bounding volume is visible, the corresponding mesh is rendered. The two following pictures show the frustum culling caught in action:
The mesh is visible: TPS counter indicates 18900 triangles per seconds
The mesh is out of sight: frustum culling is acting and TPS counter indicates 0!
Augmented Reality in Flash 10
For those who have never heard of "augmented reality" (AR), here is Wikipedia's definition:
Augmented reality (AR) is a term for a live direct or indirect view of a physical real-world environment whose elements are merged with (or augmented by) virtual computer-generated imagery - creating a mixed reality.
Sounds a bit blury? Well... I'll try to make it clearer with a demo...
Demo
First, you will have to print a little black and white "marker". The AR software scans the webcam picture and look for this very marker. When it is found, its 3D position, rotation and scale are computed and used to embed a 3D object. You can found the marker here and it looks like this:
FLARToolkit Marker
Quake 2′s 3D models in Flash 10
Loading and rendering Quake 2's maps is a challenge because Flash 10 doesn't handle complex 3D geometry very well. But what about a smaller count of polygons ? Say a 3D model for example... already done! What about an animated 3D model then?
Details, pictures and a demo application right after the jump...
Load and display Quake 2′s Maps in Flash 10
The Quake-series is really awesome. Not only in terms of gameplay but also technically. Quake 1, 2 and 3 are especially impressive. No wonder why Quake's graphics engine, maps or models file formats have been reused in many many games such as Half Life, Call of Duty or Medal of Honnor! So why not in your own Flash 10 game?
There are so many reasons why loading Quake 2 files inside the Flash player would be considered as just "impossible". Performance would be the first and main one. One would just consider that Flash is not fast enough to display complex 3D graphics, not even those of a game as old as Quake 2 (published in 1996!). But using Flash 10 latests features such as the Vector and GraphicsTrianglePath classes, it is actually quite doable!
Details, pictures and a demo application right after the jump!
New DirectFlex experiments: ‘FPS’ and ‘Earth’
I just posted and blogged about a few new experiments built using DirectFlex on Direct Flex Labs:
- FPS: is it possible to display Half Life 1 graphics in Flash ? I don't know yet... but this experiment is an improvement of the one I posted a while back
- Earth: I always wanted to implement the famous Google Earth in Flash. This might be a good first step! I made this experiment in approximately 1 hour to test out a few new features
Please let me know what you think about those new experiments. Any feedback will be greatly appreciated!