Arno Gerretsen

Slooooooooow compilation

In my last blog post I mentioned that the new FwTools version had solved most of the problems I had with my markings compiler. That is true, but it also turned out that the speed of the compiler had not improved.

Let me take the center lines of Schiphol airport as an example. I finally figure out how to compile them without errors into a good looking scenery. But I should mention that there are about 13000 line segments that make up the marking on the entire airport. The tool I wrote combines these segments into LOD grids and also gives the lines a width. So in the end that gives me one complex polygon for all the markings in a certain LOD grid.

This technique has a few advantages. For example I can draw all lines in that grid in a single reference point. This is important for the performance, because if I would draw them all with their own reference point it would bring down the frame rate too much. On the other hand if I draw all markings of the entire airport in one single reference point, the performance also suffers as markings are drawn on the other end of the airport (miles away) that you can't even see. So dividing it into tiles gives me the advantage of limiting the display of segments, but still with a good performance. Another advantage of combining the lines is that it allows me to make smoother connections between the different lines. This look much more realistic.

So, did I mention the word slow in the title of this post? I found out that when I performed all operations in the correct order, it takes something like 24 hours to compile the center lines of Schiphol. And of course there are a lot more lines to draw, like roadmarkings, parking lots, safety lines, etc. So you can figure out how much work my PC has to do.

But as the pictures below show, the end result looks very nice. So I guess it is worth the effort. And the good news is that I have a Core2Duo CPU, so that means I can run the compilation tool twice at the same time.

New FwTools version solves some of my trouble

In the past I have already written about the markings I was working on for the Schiphol scenery and at the moment I am finalizing them. The tool I made to convert the lines of these markings into polygons uses OGR as I wrote about in that earlier post as well.

This week I updated to the latest version of FwTools and to my pleasant surprise it has become a lot more stable. In that past my tool sometimes choked on the amount of line segments I was feeding it, but now they are processed fine (although it still takes some time to work through them all).

The OGR library provides some nice functionalty to for example combine, subtract or clip elements in shapefiles. These are the kind of operations I do on my line segments as well, to give them a width and chop them into LOD grids. But now that shapefiles are also used for the terrain tools, this library might also become useful for people working on tools for the mesh scenery. So check it out.

Of course there are always some trouble left. Now that my markings tool can process them correctly, the triangulate algorithm I use when writing the SCASM source files crashes on me. I guess I have to look a bit more into triangulation now... 

Europa Universalis

As a little exception, this blog post has nothing to do with Flight Simulator. Sometimes, when I am not designing scenery or developing new tools, I like to play a computer game as well. Ever since I picked up a copy of the game Europa Universalis II from a bargain bin, I have been a great fan of the strategy games made by the Swedish company Paradox Interactive.

In these games you start with a country in a historical setting and then you can try all sort of what-if scenarios to see if you can alter history a little bit. You could for example try to prevent Germany from taking the Netherlands in World War II (I never really succeeded in that) or try to battle with France and the UK in the colonization of the world during the 19th century.

And this week there was a little surprise in my mailbox, my copy of the new game Europa Universalis III had arrived. I haven't had that much time yet to play it, but it looks like great fun again. The only problem with these games is that they are quite addictive, so that will probably mean a little bit lack of sleep again...

The forgotten Douglas

Those who visited the SceneryDesign.org forums in the past might recognise my old avatar in the picture on the left. This aircraft is (one of) my favorite aircraft and as it is a little unknown I was thinking about writing a blog post about it for a while already. So here is that blog post.

For those that have not yet recognised hte aircraft from this little avatar, I am talking about the Douglas DC-5. Ever since I once borrowed the book "De Douglas DC-5" from Piet Kok from the library, I have liked this aircraft a lot. Currently I own the book as well, but I think it is only available in Dutch.

Why has such an book been published in Dutch? This is probably best explained by the fact that only 12 DC-5 aircraft have been build, of which 4 have been used the the KLM, the Dutch airline. Due to the outbrake of the second world war these aircraft have never been used in the Netherlands itself, they have been used in the Dutch West Indies (Curacao) and the Dutch East Indies (Java, Indonesia). The other 8 have been used by the US Navy as R3D. A funny note is that one of the aircraft has been the personal aircraft of William Boeing for a while, before it went to the US Navy as well. At that moment William Boeing was no longer working for the Boeing Company though.

So what went wrong for this aircraft? It must be called the forgotten Douglas now because it never really became a success. This is mainly due to the outbrake of the second world war. At the time the DC-5 was still a very new aircraft (first flight 20 february 1939) and some time would be needed to perfect the design. Due to the war this time was not there and it was decided that the DC-3 would become the standard transport aircraft. Besides that the El Sequndo division needed to be produce the Dauntless as well. So after only 12 aircraft had been build, the program was cancelled.

A last funny fact is that the DC-5 flew before the DC-4 as we know it know flew. This is because before the current design of the DC-4, Douglas was also working on a DC-4E. But as this aircraft proved to be too complex, it never went into production.
 

And a few extra bloggers again...

And again an ACES team member has entered the world of blogging, this time it is Mike Singer. Also it seems that Hal Bryan has moved his blog to the Live Spaces.

I must say that I am very excited about all these new bloggers, I am really looking forward to reading their backgrounds and stories about FS. As Paul Lange already mentioned, it seems the blog draught is over...

Am I a data junkie?

At work my collegue and I, we are the ones working on the visual databases for the simulators, are quite well known for our excessive harddisk usage on our development systems. When we are using satelite images in these visual database, it is not that weird to have a few GB of images that we need to process. For example to cover the Balkan area or the entire country of the Netherlands. And then I am not even talking about high resolution images (that would increase the amount of diskspace used even more).

During the processing of these images into the actual databsae, we usually have to reproject them a little or apply some other tweaks on them and of course the final tool creating the database also generates some  temp files worth a few more GB of data. So I guess you can imagine that we have to clean our harddisk quite often to keep the system running (from experience I know that 0 kB left of harddisk space does not really work well). And I guess the system administrator of our department has already gotten used to our questions for more diskspace.

And now the same seems the be happening at home with FsX. With the increased resolution of the terrain system in FsX the amount of data to process also goes up of course. I did a little test with high resolution images around Schiphol airport (16 cm resolution). Those photo alone were already a few GB worth of data, but when I added some intermediate files due to the conversion from the Dutch RD coordinate system to WGS84 and mosaicing of the different images into a few larger ones, the harddisk usage for this little test went up already. Now, after compiling the BGL files for FsX as well, the entire test project uses about 50 GB of diskspace (of which the final BGL files are only about 6 GB). Luckily my new PC at home has a big enough harddisk (for the moment).

With the changes to the FsX terrain system, I find it interesting to see that the process of scenery creation comes a step closer to the GIS data I have become familiar with at work already. The new shp2vec tool makes use of shapefiles and resample can process GeoTIFFs. Quite interesting developments and it shows that the difference between "professional" visual database and the FsX "entertainment" world is not that big.
 

Updating some links...

If you have been reading the blogs of Jon Patch or Nick this is not really news anymore, but I updated the links to MS blogs.

Tim Gregson has a new blog location, and I added the blogs of Brett Schnepf (community development manager),  Rick Selby (lead game designer trainsim) and  Pat Cook (design director ACES).

The FsX MDL format

On the FsDeveloper Wiki I have added a page that describes (part of) the FsX MDL format. At the moment mainly the sections involved in a simple scenery object are covered, as that is the part I mainly looked at for the ModelConverterX tool until now. Hopefully this page is useful to other programmers as well, when they want to read information from FsX MDL files.

But in this blog post I want to try to explain some of the changes in less technical terms, so that also "normal" scenery designers can understand what effects the new MDL format might have on their scenery. To do that, I will start with a short description of the Fs2004 MDL format.

The Fs2004 MDL format contains lists of the vertices, textures and materials used in the object. And there is one big section that determines what is actually drawn. In that section triangles are defined using the vertices for example. When drawing the object the scenery engine has to read this entire section from start to end, as conditions and jumps can be just in it to control what is actually drawn. Besides this there are also some sections for animations, attachpoints, etc but I will not consider these in detail here.

In the new FsX format still has lists of vertices, materials and textures. But there is no longer one big section that draws the object. It has been replaced by a couple of sections that for example defined a list of triangles and a list of object parts. So each object part uses a certain material and has a number of triangles related to it. These triangles are linked to the vertices that define them.

As you might already understand now, this structure is less flexible. The part lists are part of the LOD section, so you can define different parts for different LOD levels. But it seems no longer possible to add other conditions or jumps, to for example only show your objects during winter. I can understand that MS has chosen this new structure, as the more rigid structure makes it a lot easier to optimize the scenery engine to draw them. There are no surprises anymore because a designer has added a weird jump or complex condition somewhere.

The downside of this seems to be that we have lost the ability to make objects like docking systems that require conditions and jumps for the correct display or the ability to show different textures for each seasons.

As FsX is still very new, we might find a way to get past these problems later on. But for the moment it seems we are limited to making less complex objects (feature wise of course, I am not talking about the amount of polygons here).
 

Library Creator XML 2.0 alpha

I have made an alpha version of the new Library Creator XML version 2.0 available at the FsDeveloper forums. So if you want a GUI to create your FsX object libraries, give it a try! And as it still is an alpha version I can't promise that it will be bug free.

Scenery Shortcut

Both Owen and Nick blogged about the new Scenery Shortcut tool from Abacus in the last days. I must say that personally I don't really get all the enthousiasm about this tool, as it does not seem to really add something new that you can't do with the Mission Object Placement tool already.

You can already save your scenery from the Object Placement tool directly. Just use the "Save scenery" button and that will generate a XML file that can be compiled with BGLComp into a BGL scenery file you can use in your scenery. So all you have to do outside the Object Placement tool is compiling the BGL file.

I can understand that for some people compiling a XML file into a BGL file can be difficult, that is why I made my CompilerHelper tool in the past as well. So if you have trouble doing that yourself, maybe Scenery Shortcut is a useful tool for you. But I find a 30 MB download a bit big, for a tool that only does that. So I wonder what else you get for those 30 MB....

 

Library Creator XML 2.0

Today I started coding on the update to make Library Creator XML fully compatible with FsX. Or actually I should say that I started with this new version from scratch, using C# this time. As Library Creator is not a very complex tool, I have been able to get a lot done in just one day. And this was also a good exercise in making a GUI with C# (and I can say I like it a lot).

Compared to the current version of Library Creator XML expect the following new features:

• MDL files will be checked to see if they are valid (this also allows checking if they are for Fs2004 or for FsX)
• For FsX MDL files the GUID and friendly name are extracted, so that these can be shown in the GUI. This gives you an easy way to find the GUID of your MDL (looking it up in GMax is not always that easy).
• The user interface should be even easier to use. I have tried to get rid of all things that were not really needed.

Below you can see a first screenshot. All objects in this screenshot are FsX MDL files, so the name and GUID shown have been extracted from the MDL file. I hope to get a first working alpha version ready later this week and then I will post it on the forums so that people can test it.

ModelConverterX with textures

As I posted some screenshots yesterday of a church in ModelConverterX, I can not held back this new screenshots. I have now been able to load some textures as well.

It will still take some time before this tool can do anything useful, but at least I can now visualize the models that I read in and check if the reading of the MDL file worked OK. That is very useful while I continue to explore the FsX MDL format.

And although I can display some textures now, the performance of loading them should still be improved a lot. If someone knows a way to load DDS/DXT textures directly in C# I would also be happy to hear that. In the past I used the graphics library from Martin Wright for that, but I don't know if he also has some C# wrappers (mental note to self, ask Martin about that).

Is there a future for ObPlacer XML?

As you might already have read on Nicks blog, we are been playing a bit with the Object Placement Tool that comes with the FsX SDK. While this tool has probably been designed with missions in mind primarily, I think it can become a few powerful and useful tool for scenery designers as well.

The fact that you can place and move your objects from within FsX and that you can just move them around and see the effects instantly makes this a dream for every scenery designer as well. And when you have your own custom object library loaded, you are also able to place objects from them. In the screenshot below you can see how I placed three objects from one of my libraries (this was still a Fs2004 object library, so therefore some polygons show as black).

From within the object placement tool you can save your scenery. This is done as a BGLComp XML file. So if you compile that XML file you will get a BGL to make the placement of your objects permanent.

All these features are very similar to the EZ-Scenery tool of course. But the fact that you get this included with FsX now is great I think. And besides that the ability to move, scale and rotate your objects with the mouse as well makes it quite easy to use (I don't think EZ-Scenery can do that, but to be honest I have not used EZ-Scenery that much so I could be wrong). 

At the moment the mission object placement tool also has some problems. For example when I load in my objects again after placing them, I do not always see them. In those cases I only see the bounding boxes. And besides that the GUI could be made a little more user friendly as well. Finding objects is quite hard now for example, due to the huge amount of object available to be placed.

But I like the concept of this tool that much, that I don't know if I am going to update ObPlacer XML for FsX. And if I do, I guess it would be a much better idea to write it in the same form as the mission object placement tool, so as a plugin that runs in FsX.

Is it a gift to annoy people?

Over the last few days I found out that spammers have found a new way to annoy me. This time they are not spamming my blog or my email accounts (well, they are still spamming my email, but not more than usual). I noticed that on the Talk/Discussion pages of the FsDeveloper Wiki they have now started to post links to nonsense stuff.

Most of them don't show up when you look at the page, but when you start to edit it you see the huge amount of links. I try to delete this stuff, but usually within 30 minutes they are back again. I'll see how it develops in the next few days, but if the spam does not go away I guess the access to the Wiki needs to be restricted a bit. Like only allowing people who have registered to contribute. But if not needed, I would like to prevent such steps, as the Wiki should be as open to all users as possible. If we make it to hard people will not contribute that fast.
 

A nice morning GMax session

This morning I was chatting with Nick and it turned out that both of us had not been able to use the new FsX GMax gamepack as much as we would like to in the past weeks. So we decided to have a little session right away, while speaking to each other using Skype.

We went over some of the documentation, for example the documentation about the animated jetways and we exchanged some ideas on how we could use that on our own objects. And we also explored the new FlightSimX material type in more detail, to check all the cool options available on it.

But the most concrete result was that we also tested the platforms of the attachtool. Of course these work fine. But now the different types of platforms also work fine, while they did not in Fs2004. So if you set the type to GRAVEL you will see dust behind your aircraft, while there is none if you use the ASPHALT type. And when setting it to ICE you will certainly notice that the platform is more slippery.

In the end we both found it a very useful session, as it is a nice way to explore the new stuff and exchange ideas. Hopefully we have some time soon to do this again and I am sure we will be able to report over some interesting finds then.
 

Progress for today

As the screenshot of the new tool I posted earlier today is not that nice looking with all those weird colors, I thought it would be nice to post another one now at the end of my day to show you how far I have gotten.

As you can see the weird colors are gone now. This is because I have been able to figure out how to read most of the material information. So I can get the material color and texture now, just some of the more advanced material options I can't get yet. At the moment I can't read in the textures yet in the preview, although I can extract their names from the MDL file. That is why all textured parts show that checkerboard.

Besides that extra information can I can now read from the MDL file, I have also improved the preview picture in general, by applying the correct shading to all polygons and that short of stuff. So that all together makes it look more like a church now.

I will continue this weekend to see what other cool things I can read from the new MDL format and besides that I'll also try to create a new page on the Wiki about this MDL format. So that other people can help with discovering it and don't have to reinvent the same wheel again. 

Working on a new tool

The last few days I have been working on a new tool, it will be called ModelConverterX. In the end the purpose of this tool will be to allow you to convert 3D objects between different formats, for example loading old API macros and saving them again as FsX MDL objects. But I still have a long way to go before all that is ready.

At the moment I am mainly working on understanding the new FsX MDL format (and learning C# as a new programming language for my tools as well). The FsX MDL format has changed completely compared to the Fs2004 one, so that means a little work is needed again to understand it and be able to read and write it.

At the moment I can already read the shape of simple objects, but I still need to put some more work in reading the material section for example. Below you find a first screenshot of what will become the tool in the end. Hopefully you will be able to recognise a church in this picture. But I guess it is clear that I still have some work to do.

Happy new year

I would like to wish everybody a happy new year. Hopefully this will be a good year for you all, with a lot of Flight Simulator related fun of course.