Arno Gerretsen : Programming

LOD Creator improvements

arno — Sun, 18 Oct 2009 11:29:00 GMT

The functionality in ModelConverterX to create lower level of detail versions of a model was running very slow for complex objects. I have not fixed that completely (yet), but I have made some changes to this functionality that should at least make it easier to work with. The calculation that simplifies the object is now running on a different thread, so that at least the rest of the tool should remain more responsive. I have also added a progress bar, so that you can see how far in the simplification process the tool is already. See the screenshot of the new dialog below.

As you might see there are some more changes. I have removed the 3D preview from the LOD Creator dialog. Instead the mean 3D preview is used now to display the result of the LOD Creator dialog. You can still select the LOD you want to see from the LOD Creator dialog though.

Optimizing the simplication code further, so that it will run faster is still on my todo list. But at the moment I am also busy with preparing for the new gPoly ground polygon tool. So I am not sure which one will get my attention first. Now that I am talking about the gPoly tool, I would still be happy to receive your input on what you would like to see in a tool for custom ground polygons.

Unions

arno — Sun, 27 Sep 2009 17:28:00 GMT

You could say that recently unions have played a role in my life. First there has been my wedding one and a half week ago of course, forming a nice union between me and my wife. But that is not what I want to talk about in this blog post.

I have also been working on some code that allows me to do boolean operations between polygons. This functionality is something that will come in handy when I start working on my new ground polygon tool, It can for example be used to automatically slice the polygons into piece of no more than 100 meters, so that there are no problems with the curvation of the earth within FSX.

Below you see a screenshot of my test application, where I am testing out these boolean functions. The red polygon is polygon A and the green one is polygon B. The four small pictures below show in grey the union of A and B, the intersection of A and B, polygon A minus B and polygon B minus A respectively.

At first I looked around for some library that implements this functionality and although some exist, I still decided to code it myself in the end. The main reason for that was that I wanted to understand how the process works and that why learn a bit more from it (although it might not always be the easiest way). So in the end I used the algorithms described in Geometric tools for computer graphics and make my own implementation, building further onto for example the polygon class I had already made for ModelConverterX.

The boolean operations are not completely bug free yet, I have already found a few polygons that are not processed correctly. So there is still some more work to do and after that I will start with creating the tool for ground polygons that can make use of this logic. So I still have much more fun ahead...

Reading of textures

arno — Sun, 16 Nov 2008 12:33:00 GMT

A little while ago I already wrote about the fact that I was working on my own texture loaders for the ModelConverterX tool. I have now finished the first versions of the loaders I find most important for the tool. I say first, as I am sure some bugs will be surfacing when they are used by more users. So I have now made loaders for the DDS, BMP and R8 format.

The DDS format is common in FSX and I can now read DXT1, DXT3 and DXT5 variants. Figuring out what their difference on the binary level are was very interesting to understand how the compression of those files work by the way. I think I should make a little Wiki or blog article about the things I learned, as that would help a lot of other users as well in understanding which format to choose in which case.

For the BMP format I can read the normal 8, 24 and 32 bit BMP files, but also the ones with DXT compression that were used in previous version of Flight Simulator. I am sure there are some other compressions that are sometimes used for FS as well, but I did not yet encounter these in my own collection of test models.

Finally there is the R8 format. If you still remember these textures you are around for a while, as it is the format used by FS5. Basically it is a RAW file with indices into the FS5 colour palette. A wide range of extensions was used for these textures, not only R8, but also PAT, 0AF, 1AF, ..., or any other extension the designer could think of. I can now read those textures in ModelConverterX and display them in the preview for old API macros that still use them. Maybe it would be a good idea to convert these old textures to a more modern format when you export to FSX MDL again.

So that is an update on my texture reading progress, I am now looking forward to tackle some of the other wishes still on my wishlist for the next version.

Texture file formats

arno — Sun, 02 Nov 2008 14:25:00 GMT

Loading the textures files into the ModelConverterX tool is something that has given me quite some challenges already. I have used different libraries for it already and all of them gave me some trouble. At the moment I am using the library by Martin Wright and although this one can read all the files I want, it is not a very fast library. So this makes the loading of models relatively slow.

That is why I decided to have a look at how the different texture formats exactly work and see if I could create my own texture file loaders. At the moment I have been studying the BMP and DDS formats. Creating a loader for those formats was not so hard, so at the moment I can read these formats. This should allow loading most of the texture formats used by Flight Simulator.

Only the R8 format is still on my todo list, as this is still used by older API macros that the tool should be able to convert. Maybe it is a good idea then to convert the textures to the DDS format when exporting to FSX MDL files.

I still have to test my new loaders with some more models, to see if there are not some "exotic" variants in use that I can not yet read at the moment. But all together it has been quite interesting to study those formats and gave me a lot of useful background information on how they work.

ModelConverterX and x64

arno — Sun, 14 Sep 2008 18:27:00 GMT

Just after the release of the new ModelConverterX version, fellow MVP Lefteris reported the first bug to me. That in itself was not so weird, as there have been many bugs reported during the previous testing phases. But the bug that was reported was a bit unusual. It seemed the tool did not run on 64 bit Vista. I guess that was a combination that had not been tried before during the alpha testing. After some testing we were fortunately able to locate the problem and I have produced a fix for it now. The new version, that also works on 64 bit systems, is available from the tread at the forum.

This bug is the first time that I had problems with any of my tools on a 64 bit system. But I guess that was more lucky than whisdom, as I did not really consider what might go wrong on such a system. I guess from now on I will keep that in the back of my mind when working on the different tools. And if 64 bit systems get more common it might be time to have one installed for testing.

User interface and manuals

arno — Fri, 12 Sep 2008 20:02:00 GMT

It's already some days ago that I wrote that the next version of ModelConverterX is almost ready. One reason for the delay is that I got a cold this week and that reduced the amount of time I would like to spend behind the screen a bit.

But another reason is that I, and I guess more developers, tend to forget how much time is needed to write or update the user manual. That process or also the process of creating a graphical user interface that is user friendly (and idiot proof) seems to take up more time when programming a tool like ModelConverterX than writing the actual logic of the tool. For example adding the functionallity of attached objects was not that hard, but making the dialog to edit such objects in a user friendly way took me more time.

Of course I am saying that the user interface and the manual should get less attention to save some time, as without those two things it will be a lot more difficult for people to use the tool. And what is the purpose of making a tool when nobody can use it?

I hope I can find the time this weekend though to finish that next release...

Real-time rendering

arno — Sun, 24 Feb 2008 16:06:00 GMT

Recently I bought an interesting book, Real-time rendering. I have not completely finished reading this book, but it contains a lot of useful information. For example I have found there solutions for some problems I had while working on my ModelConverterX tool. But the explanation of how the rendering process works also give a better insight on what has an influence on the performance when rendering. This is not only interesting to understand better how FSX works, but that is also something I can use at work when dealing with our image generators there.

Triangulation

arno — Sun, 27 Jan 2008 21:22:00 GMT

Today I did some more work on the ModelConverterX tool again. There were some complex polygons that were not converted into triangles correctly, so I had to improve the triangulation algorithm. I did make such an algorithm a while ago for another tool, but that one was for 2D polygons only. So I had to convert it into 3D to work correctly with 3D objects.

This turned out to be a lot of fun in the end. With some nice vector and matrix manipulations it seems you don't even need that much code to triangulate a polygon. And it was a good reason to refresh some of my math skills again.

So that is another problem less, there are still some problems left with the normals and the sides of the polygons being drawn. Once these are solved I hope to release a new alpha version, as there have been quite some improvements by now.

Tango Desktop Project

arno — Sat, 29 Dec 2007 14:02:00 GMT

Today I was looking around on the web for some icons to use in the ModelConverterX tool. Of course there are a lot of places where you can buy very expensive icons to use in your application, but that was not really what I was looking for. In the end I came across the Tango Desktop Project, a project with the aim to help create a consistent graphical user interface experience for free and Open Source software. They have a nice set of icons and I think you will see some of them appear in my tools in the future.

Working with the GeoTIFF library

arno — Wed, 22 Aug 2007 17:42:00 GMT

I came across this interesting article on how to use the GeoTIFF library today. This might be of interest for FS scenery tool designers as well, since the GeoTIFF format can now be used with the FsX resample. So that makes the GeoTIFF format also interesting for background images in tools for example.

GFL SDK

arno — Thu, 05 Jul 2007 20:16:00 GMT

Loading in all the different kind of textures that FS supports in a tool can be a quite challenging task. Some scenery object use 8 bit textures, others 32 bit, or compressed DXT textures. Some objects even still use the old RAW textures with a palette. And in FsX the DDS format has been added to this long list.

For my MDL Tweaker tool I have used the graphical DLL files by Martin Wright to help me reading the different (compressed) textures that FS supports. But yesterday when working on ModelConverterX I can across another library that can also help me with this subject, GFL SDK. This is the library that is behind the XnView thumbnail viewer as well. Using this library made my job of loading textures a lot easier and the library also provides some functionality to edit the textures (in case I ever need that).

So if you are having trouble with loading all those different texture formats in your tool, you might want to check out this nice library as well.

COLLADA

arno — Mon, 07 May 2007 19:33:00 GMT

For the new model conversion tool I am working on at the moment, I took a closer look at the COLLADA format this weekend. COLLADA is an open standard Digital Asset schema for interactive 3D applications. It can for example be used as an exchange format to share models between different tools. That is also the role in which I want to use it in my tool. There are for example COLLADA plugins for tools like 3DSMax and Maya, but also Blender can work with it.

I think that it would be useful to have the ability to convert COLLADA files to FsX MDL files, so that more models can be used with FsX. And also the reverse conversion might be useful if that allows you to edit your models easier (but I have not looked deep into that yet).

So a COLLADA importer/exporter is certainly on the todo list for my new tool. But it does not have the highest priority at the moment. The first features I want to have working is importing old API macros and being able to save them again as MDL obect. As that is the main purpose of this tool. After that I'll see what other cool features can be added to it.

No math?

arno — Wed, 12 Apr 2006 21:35:00 GMT

The last two days I posted rather positive about those new animation macros I was writing. Today I am a little less positive, as it seems I have hit a limit of BGLC/MASM (as far as I know BGLC is build on MASM).

It seems I can only do very simple mathematical calculations in those macros. But to do fun stuff, I need more than a sum or a multiplication. I would like to calculate square roots and cosines as well. So I guess I need to do a little more research if such functions are available in MASM or if I can add them easily.

OGR

arno — Wed, 08 Feb 2006 19:43:00 GMT

The last few days I have been using the OGR library at work. As the website says, it is a Simple Feature Library. It allows you to rather easy read, manipulate and write vector features in different file formats (for example DXF or SHP).

This can be very useful, as you do not have to create your own file format parser, as I did for my DXF2SCA tool. And the manipulation of the vector data is really nice. At work we for example use the union function to make one big polygon out of different taxiway segments. And the buffer function is used to calculate a grass area around all features of an airport. This way we can rather easy make some generic airports for our visual databases.

I think I will also give this library a try for my next (updated) version of DXF2SCA. That could certainly save me some coding and allows me to focus on the scenery output for FS, instead of reading a certain file format.

OpenFlight

arno — Tue, 13 Dec 2005 22:11:00 GMT

I want to start with a little warning. I am afraid this post is going to be rather long, so if you are already desperately in need of coffee now, you'd better get a cup before reading the rest of this.

So what is this OpenFlight you are talking about you might think? OpenFlight is not some sort of open source Flight Simulator or so. No it is a very common format to store your scenery in, for professional flight simulators. In this article I want to talk about the similarities between this format and the BGL format of Flight Simulator. And also about my attempts to make a converter between the two formats. Let's get started.

The OpenFlight format has been create by MultiGen-Paradigm and is very common in real flightsimulators. At work we also make use of this format to store our visual databases (or scenery as most of you will know it). Rather soon after I started on my job, I had to learn more about the OpenFlight format, as I wanted to create a few tools to make our work easier. As I can almost dream the FS BGL/MDL format by now, it was rather logical that I compared this new format for me to what I knew. Slightly to my surprise a lot of things are almost the same. All materials are defined with the same properties (diffuse, ambient, specular and emissive colors for example) and the way to to store vertices and create triangles using them is also very similar.

The biggest difference for me was, that the OpenFlight format is a sort of a database. So each polygon has a few vertex nodes as children. And as a parent the polygon node can for example have a group, object or LOD node. This makes the structure of your object a lot clearer. A downside of this, is that the file format itself takes more diskspace. While learning about OpenFlight, I started to understand that MS has chosen a nice compact format to store the objects in.

There is another difference I should note. When I am talking about FS BGL/MDL files, I am talking about the object files mainly. The terrain has its own BGL format in FS, which has its own logic. In the OpenFlight format we usually create an entire database in one file (or a set of files that are linked together). So you have terrain, roads, 3D objects, airport all defined in the same file. As a result of this, the structure of your file is much more important, when trying to optimize your scenery. In FS scenery the MDL files usually contain only one object, so that makes their structure less important. All the placement is done with different (XML based) code and that has its own logic and structure again (I won't start to talk about that as well here).

At work we were working on a database for Schiphol, the airport of Amsterdam. And because I had already created a FS scenery for this airport (as part of the NL2000 scenery team), the question soon came if we could not convert some of those 3D objects for use in the real flight simulator. One of the other NL2000 members had already created a converter the could create OpenFlight files from SCASM source code. But as we had created these objects with Fs2004 MDL files, they could not be decompiled to SCASM code. I had also started with the new version of my MDL Tweaker tool (which contains a nice MDL loader) at that moment, I decided that it would be cool if I could also add an OpenFlight export function to it.

In theory this conversion is not that hard. You have an object, that contains of a collection of vertices, materials, textures and triangles and you need to read the MDL format and write the geometry to the OpenFlight format. And the logic behind the conversion was indeed finished rather soon. But then the trouble started for me. I had to code this logic. The API provided with the OpenFlight format provides functions to create an OpenFlight file, but unfortunately for me these are in C++. And because I still need to find an easy way to create a nice GUI in C++, I am coding my MDL Tweaker project in Visual Basic.

So I first tried to use the functions of this API in Visual Basic. I won't bother you with all technical details, but it proved to be more difficult then I had expected. Either the export function worked correctly, but crashed on big MDL files, or it worked on big files, but the properties in the OpenFlight format were not set correctly. So a few days ago, I decided that it would be easier if I wrote directly to the binary OpenFlight format from MDL Tweaker. A good discription of this binary format is luckily available.

So just this evening, I was able to finish that part of MDL Tweaker and the good news is that now everything works fine. So I can create OpenFlight files from my MDL scenery objects created with GMax. Now you also know the reason for the delay of a new alpha version of MDL Tweaker. The last few weeks I have mainly been busy to implement this new function (I could not accept that it did not work as I wanted).

After reading all this, you might think what can I do with all this? The answer for most of you is probably that the OpenFlight format is not of very much use to you. But if you want to play with it a little bit, I would advice you to give OpenSceneGraph a try. The OSGViewer that comes with OpenSceneGraph is able to load OpenFlight (FLT) files. So after I have released the next alpha version of MDL Tweaker, you can view your own objects in the OpenFlight format is you ever wish to do so :).