Arno Gerretsen

New blog

I know this blog has not been very active in the last few months, but I haven't stopped blogging. I hope to be able to post some interesting updates soon.

I have however decided to move my blog to another site. So please update your bookmarks, from now on my blog is found at scenerydesign.org. All the old posts have been migrated already. So hope to see you there again!

Specular and fresnel effects in preview

The next development release of ModelConverterX contains improvements to the rendering of the object in the preview. The two biggest changes are that the specular hotspot and the fresnel effects are now shown in the preview as well. I am not convinced that the fresnel effect is exactly like in FSX, but I think it is rather close at the moment. Besides those two changes there are other smaller improvements as well, for example reflections work more like FSX now and also some additional material settings now have an effect in the preview.

Classify building footprints

I have been adding more built-in attributes to scenProc recently, to allow you to classify buidling footprints in different types. Then for each type you can make different kind of autogen. This work is not yet finished, but in this blog post I want to show what you can do already now.

Please read the scenProc manual to see which attributes are available and what they mean. Below is a scenProc configuration file I used to create autogen around the city of Amsterdam:

# Amsterdam autogen test
IMPORTSHP|buildings_adam.shp|NOREPROJ
#
ADDATTRIBUTE|*|BUILDCAT|5|1
ADDATTRIBUTE|FCONVEX=1|BUILDCAT|4|1
ADDATTRIBUTE|FNUMPERPANG>3;FNUMNOTPAR<2;FNUMVERT<10|BUILDCAT|3|1
ADDATTRIBUTE|FAREARAT>0.85|BUILDCAT|2|1
ADDATTRIBUTE|FNUMVERT=4;FNUMPERPANG=4|BUILDCAT|1|1
#
EXPORTSHP|*|bla
#
SPLITGRID|AGN
#
SETAGNBUILDINGHEIGHT|1.0;0.4;0.2;0.0
CREATEAGNGENBUILD|BUILDCAT<2;FWIDTH<20|{5ae04eb6-934c-4f63-bb48-5e7dee601212}|MAXRATIO=2
CREATEAGNGENBUILD|BUILDCAT<2;FWIDTH>20|{6089A0BD-CED1-4c47-9A9E-64CDD0E16983}
#
WRITEAGNFILES|c:\flightsim\nl2000\nl2k_v4_resample_NH\texture

So first I set all features to category 5. Then with the second ADDATTRIBUTE step is make sure all convex buildings are category 4. The FCONVEX attribute is used for this. All buildings that have at least 4 perpendicular corners and no more than 2 edges that don't have another parallel edge and have less than 10 vertices are step to category 3. All buildings where the difference between the footprint area and the bounding box area less than 15% are set to category 2. And finally all footprints with 4 perpendicular corners are defined as rectangle (category 1).

Now I use can this new attribute BUILDCAT to create the autogen buidlings. I use category 1 and 2 for the rectangular autogen. The other three categories are not used in this example. But my plan is to add another step that can split the footprints of category 3 into a couple of rectangles to represent the shape. For category 4 and 5 maybe the polygonal autogen can be used or else I think it would be a good idea to create MDL objects from those footprints.

Of course the classification I show now is not how it should be done, you are free to define your own rules in the ADDATTRIBUTE steps. Let me know if more attributes are needed for that.

Below is a screenshot that shows which footprints are classified in which category for part of the city. As you can see it is not perfect yet, so I will continue to do some more testing to refine my rules. Once I found a set that works well, I'll post an update of course. Especially for category 3 the results as not as good as I hoped, so I will try to find a better algorithm there.

Built-in help

In the last two days I have made scenProc a little more robust. Before any mistake in the configuration file would usually result in a crash of the entire tool. That has been changed now. The tool will now check if you provided the right amount of parameters and give a warning when that is not the case. Sometimes it will also check if the parameters provided are of the right type, but that is not yet supported for all available processing steps.

Another change I have made is that the tool now has built-in help. If you enter scenproc --help in your command prompt you will get a list of available processing steps. If you type scenproc --help stepname you will get additional information about the processing step with the given name. This includes information about the expected parameters and what the step does.

I hope these two changes make the tool a little bit easier to use for you. Oh, and one more thing. You might see there are more processing steps available than documented on the wiki. Those other steps are used to make ground polygons from dxf or shp files. But they are still under development and might be a bit buggy, so I would not advice you to use them on a serious project at this moment.

Some autogen thoughts

I am going to spend some time on the scenProc tool again the coming weeks. It's time to finish some of the features of that tool and make it even easier to make autogen. But of course there are some challenges left.

One of the main challenges is that most FSX autogen must be rectangular. So for a group of houses as shown above that will work. It is not so hard to define a rectangle for each of them. Some have a small extension, but even if you ignore those the result will look quite good. For the buildings shown below it becomes a little harder already, but you could represent their shape with 3 rectangles that partly overlap. An algorithm can be defined to derive those rectangles.

So, so far it seems quite possible. But now look at the buildings below.These are all very irregular shaped. If you would try to represent these with rectangles you would always get parts sticking out or parts missing. Not really a good approach thus.

You might be thinking, right, that's why we have polygonal autogen buildings in FSX as well. But unfortunately the polygonal autogen buildings must have an internal courtyard. This makes them very hard to use for this kind of shapes. I really wish Microsoft would have made them without the courtyards as well. But it seems that is not possible.

So my challenge is to find out how to classify the building footprints from the vector data. Which footprints can be represented by normal autogen buildings, which ones can be represented by row houses or by polygonal autogen. And which footprints can better not be represented by autogen at all.

For that last category it might be an option to create them as normal MDL scenery object. From the footprint it is possible to create a building directly. And by using drawcall batching it might not even be that hard on the performance. That's something I would like to test as well.

And finally I think I will try to analyze the autogen format of the polygonal buildings a bit more. Maybe, maybe, there is some undocumented bit to remove those courtyards? Let's keep dreaming....

All map images are from OpenStreetMap.

User interface change

I have changed the main user interface of ModelConverterX a bit, actually it has gone back to some of the concepts that the tool used in 2008 already. I did remove the tabs that were used for the preview, event log and options. Instead of those tabs the options are now on a separate form, which can be opened with the Options button at the top toolbar. The preview and the event log are both shown at the main screen. There is a splitter in between, so you can change the size of the preview and event log as you wish. There is also a button to collapse the event log completely, if you want to maximize the preview. Below is a screenshot of the new user interface.

Why did I change this? Because I think it would be better to have the event log visible while you are working with your object. That way I think it is less likely that users miss an error or warning in the event log.

Problem reporting

user: He, your tool does not work.

me: What do you mean? What are you trying to do and what goes wrong?

user: When I import my object it crashes. You need to help me, I have to convert this object.

me: What is the error message you get at the crash? Can you maybe send me the object to debug?

user: Never mind, I fixed it already.

OK, this is expurgated a little bit, but this is how a typical user interaction about a bug in one of my tools might go. Both for the user and for me it is not an easy process. So I have made some to ModelConverterX now to make the progress of reporting bugs easier.

Every time you have a crash in ModelConverterX you will see the error handler screen as shown on the right. To help me fix bugs in the tool, I would like to ask that you send in the error everytime you have a crash (of course only once for the same crash).

But what happens when you send in the error? The tool will then automatically make an entry in my bug tracking system. In that entry it contains the error message and the stack trace that helps me to find what was going on. If you select the checkbox to upload the file that is causing the error, also the object you were trying to load will be attached to the bug report. If possible, please send the object as well, since that helps me a lot with debugging. And that's all. No other information is being send about you or your computer.

So please use this function if you encounter crashes in the future. It will help me to improve the tool and it will help you to get a tool with less bugs quicker.

Correcting for the FSX curved earth

One of the main differences between FS2004 and FSX when it comes to rendering the earth, is that FS2004 assumes that the earth is locally flat while rendering the scene, while FSX uses the correct curvature of the earth as well. But this has some consequences for scenery designers. When you make ground polygons they are usually flat in the design tool, but putting them on the curve of the earth will mean that the ends will float in the sky. The ground polygon wizard of ModelConverterX does correct for this problem, by shaping your ground polygons to follow the curve of the earth.

But this problem not only affects ground polygons, also normal buildings or animated vehicles can have this problem. Therefore an experimental function to correct for this problem was in ModelConverterX for a while already. Over the last week this function has been expanded and improved and therefore it now has become a new editor: the Earth Curve Editor. You can load this editor by clicking on the icon with the picture of the round earth.

From this editor you can set the position of your object and choose what you want to correct. Entering the exactly altitude of the ground is very important to get the correct correction.

A problem in FSX noticed before is that attached effects are offset from the geometry, the further you get from the reference point. This is related to the curve of the earth as well. So there are two chooses for a correction:

1. You correct the geometry to follow the curve. This will also correct the path of the animations and attached platforms. Attached objects, like effects, are not corrected however. Once the geometry has been corrected they line up again.
2. You only correct attached objects, like effects. This correction will move them to line up with your geometry. But if your geometry is big, it will still float at the end. So in general it is probably better to use option 1 instead of this one.

This new editor is in the latest development release. I hope it proves to be useful and let me know if there is any feedback.

Bye, bye, performance indicator mode

ModelConverterX used to have a render mode called performance indicator mode. With this mode you could spot where the texture mapping or normals were not so optimal. The recent updates to the preview have removed that mode. But I have now added two replacements. One is called the Normal Inspect mode and the other TexMap Inspect mode.

In the Normal Inspect mode the colour of the object is determined by the normal, see the screenshot on the right. This means that as long as the normal is shared by different triangles smoothly you will see the gradually changing colour. If you see a sharp change in the colour, then the two triangles meeting there will have a different normal. Hopefully this helps to spot places where the smoothing is not as optimal as it could be.

The TexMap inspect mode works similar. It is meant to spot places where the texture coordinates are not shared optimally between triangles. The screenshot below shows an example. In this case the object is covered by a checkerboard with a varying colour. The checkerboard helps you to spot if texture mappings might be misaligned and the colour gives you an indication which part of the texture is used. So on the side of the church in the screenshot you see two types of blue, which means those polygons are textured with different parts of the texture.

The Normal Inspect mode is quite intuitive to use I think, for the TexMap inspect mode I am not sure yet. But I haven't come up with a better idea yet. Hopefully these modes help you in making optimized models.

A bit buggy

The recent changes I made to the ModelConverterX preview turn out to be a bit buggy. To be honest that's not a surprise to me. I am still learning the OpenGL shader language and also the fact that every graphics card might handle the shader code slightly different does not make the job easier. But I like the bug hunting and it helps me to learn more about the shaders as well.

So a big thanks to all the users who encounter these problems and report them to me. Your reports really help and I try to iron those bugs out as soon as I can.

And if the bugs really prevent you from doing what you want to do, please revert to the stable version 1.3 for the moment. The development has hardly any new functions, except for those preview changes. So you are only missing the bugs.

Updated ModelConverterX preview

The development release of ModelConverterX now contains an updated preview of the objects. It is now using OpenGL shaders, which as enabled me to add some cool new functions. Like showing bump maps and the reflection influenced by specular maps. Be aware that this is a big change, so it might have resulted a few new bugs here and there. Please let me know if you have any issues. And in case of big issues, you can always revert to the stable 1.3 release until I have fixed them.

Instant Object Studio - part 2

Since my previous blog post about the new Instant Object Studio tool I received a review copy of the tool (thanks for that Konstantin). So I am now able to answer some of the questions I still raised in my previous post.

When placing for example a roof on a box, the tool does not remove the top polygon of the box, altough it is not visible. So that means there are slightly more polygons than optimal, but for most objects that should not matter too much for the performance. Especially if you make sure it has the same material as other polygons in the model, in that case it ends up in the same drawcall.

Speaking about drawcalls, as long as you are careful about how many materials you use when modelling, the generated scenery objects can be very light on drawcalls. As an example I tried to model the house I did for my SketchUp tutorial again with this tool. This house only uses one texture sheet, so the generated model has only one drawcall.

I also mentioned in my previous post that I did not like the visually dragging and scaling of the textures, as that makes it hard to align exactly the part of the texture sheet you want. Especially when you put all your texture parts in one big sheet that can be tricky. But there is a function in Instant Object Studio to help you with that already.

In the material settings you can select that the texture should be stretched along the X and Y axes. With the Area button you can then choose the area of the texture you want. If you then apply it to a polygon, exactly that piece of the texture will be fitted on the entire polygon. Works quite neat I would say. It's not as flexible as the UVW Unwrap editor in GMax, but I feel it gives me better control over where the texture goes than in SketchUp. Maybe starting scenery developers won't appreciate this feature so much, but if you try to model with one texture sheet only I feel it comes in handy.

I'll continue to test this tool more, as it seems to be packed with interesting features. Make sure to read the manual to learn about all the shortcuts and restrictions that can be applied. Next thing I will probably test is see how easy it is to make a building when you have a photo scenery showing that building. The object I made now is in the middle of the sea. But I think this tool can be really handy to trace buildings from photo scenery. I'll share my experiences again after I have tested this.

Instant Object Studio

Flight1 has released a new tool, Instant Object Studio. This evening I had a play with the demo version and I must say I am surprised by the tool. After seeing the video I had the impression that the tool would be nice to make some quick buildings, but that it would be hard to make really detailed photo realistic objects with it. But after testing the demo version a bit I think I was wrong there. Since the demo version does not do export, I have not yet been able to see how optimized the output generated by the tool is.

Here's a list of the positive things I have noted:

• The concept of building your object inside FS is cool!
• Making the geometry is quite easy. The different shapes snap together and after reading the maniual (yes :)) I found out there are some easy ways to fix the movement of polygons or edges along an axis. All that together makes it rather fun to model the shape you want. It's a bit like SketchUp. Modelling geometry in SketchUp is also fun (more fun than in GMax at least).
• When making a new material you have access to all the FSX specific settings if you want, but they are hidden under advanced settings.
• It is cool how you can easily make your own texture using the generic templates. You can adjust colours and even overlay a second texture. Nicely done, only don't get carried away as you might end up with a lot of drawcalls.
• You have rather good control over the texture mapping. I think it is even easier than those clumsy pins in SketchUp. So even if you have all parts of your house on one texture (as you would try for good performance), it is not too hard to alter the texture mapping.
• You can attach effects and control the amount of detail in the crashboxes.
• The fact that your textures are directly saved in the format that FS needs and that your model is exported to BGL right away will be very useful for many people. It saves the steps to use BGLComp, ImageTool, etc. Especially for new developers that is very good.

Of course there are also some points I am not so sure about:

• Although the manual mentions it is important to use as few textures as possible, the way you can make generic textures could tempt a lot of people to use much more textures than is good for the performance. If you only build one of two houses it is not going to hurt, but on bigger projects it will.
• Although it is cool to work in FS directly, it also adds overhead. FS running uses more memory and CPU than the average modelling tool. And when making scenery you will often have a painting program and some other utilities open as well. I did not yet run into trouble, but I guess it could happen.
• I would like to have more control over the texture mapping, a bit like a UVW unwrap editor where you can move each vertex of the polygon to the correct texture position. But that could be me, I am a control freak and prefer to have my mapping very accurate.

Since I haven't been able to save yet, I can't judge yet how easy it would be to adjust your model after you made it. For example if you made a mistake in the shape of the texture mapping. I'll probably buy the tool so that I can have a look at that. And then I can check how optimized hte output is as well.

So all in total I think this is a very nice tool for people who want to build a few scenery objects or maybe their small local airport. Working with it is quite fun (which is also important). And from the functionality point of view I think it is similar to modelling with SketchUp. You won't get the fancy animations, levels of detail and very complex shapes that you can achieve with GMax or 3DS Max. But on the other hand it is much easier to learn and probably more enjoyable to use.

ModelConverterX 1.3 released

Just a quick note that ModelConverterX 1.3 (stable version) has been released. It is the same as the development version of yesterday, but now with a updated manual. It will be the stable release for the near future, since version 1.2 was getting very old. You can read some more information here.

Ground polygon wizard upate

I have updated the ground polygon wizard in ModelConverterX. Two new features have been added to it in the latest development release:

• You can now specify which layer and visibility are used by default in the options. This should mean that you have to change less values while working on your ground polygons.
• An option has been added to convert the textures used on the ground polygons. This means they are converted to DDS (when using FSX) or DXT BMP (when using FS2004). Before you would have to do this conversion manual. In the screenshot on the right you see the options for the texture conversion. In the text box you need to specify the location where the new textures will be stored.

Hopefully these changes make the wizard a little more easier to use!

Reading X files

I have added a new function to ModelConverterX. It can now also read X files. This can be a X file generated by the FS2004 or FSX gamepack, but also a X file generated by another tool that does not contain the FS specific information. The following features will be read from the X file:

• Geometry, normals and texture coordinates
• Material settings (including FSX specific material settings)
• LOD information
• Animations

This new functionality is in the latest development release. Let me know if you have any issues with this feature.

Yes this blog is still alive

Just to let you know this blog is still alive. Just recently I did not have too much time to post interesting things here. But that does not mean I haven't been (silently) working on new things. Here are some of the things I have been working on recently and once they are finished you will read more about them on this blog as well:

• Improving the reading of aircraft MDL files in ModelConverterX, especially when it comes to the animations of the aircraft.
• Improving the ModelConverterX preview. I have been experimenting with using shaders and that seems to be a great way to allow reflections, bump mapping and that kind of advanced features to show in the preview as well. I might even get the skin and bone animations working in the future. 
• I have also been doing some scenery design with SketchUp, working on some new objects for the NL2000 scenery.
• Oh, and last but not least, we have been preparing for the baby that can be born any day now.

That should be enough to dust off this blog for now. When I have finished some of the items listed above, you will be able to read all the details here again.

Animation Editor updated

I have update the Animation Editor in ModelConverterX. Before this form could only be used to remove all animations from your object, but now it can do more. The following features have been added:

• Select which animations are active and shown in the preview.
• Examine only sections of an animation, for example by looping only a subset of all frames.
• Remove the selected animations. These will then be "frozen" to the animation frame you have selected. This is similar to the function to remove all animations before, but now it can be done per animation.

I have made a quick video tutorial to show you how it works.

MDL Tweaker

I have just added a new feature to the ModelConverterX development release. It is a MDL Tweaker that you can find in the special tools menu.

Let me first explain how this tweaker differs from loading a MDL into ModelConverterX itself. When you import a MDL into ModelConverterX the object is read and stored in an internal representation. ModelConverterX then works with this internal representation when you make changes. In the end this internal representation is exported again to a MDL file.

The MDL Tweaker works differently, it directly manipulates the binary code of the MDL file, without decoding it to an internal representation first. This means that you can for example make small modifications without loosing animations or other elements that are in the MDL file. But it also means you are more restricted in the kind of manipulations you can make.

So what can this new MDL Tweaker do? At the moment it supports the following tweaks:

• Change MDL name
• Change MDL GUID
• Change radius as stored in the MDL file (this is similar to the function of the RADItor tool I made a while ago)
• Change the bounding box values of the MDL
• Add a custom shadow model

Below you see a screenshot of the form. After you have made the changes you want, you need to click Save MDL to saves the changes to disk. When you click the Insert custom shadow model button you will get a file selection dialogue where you can select the MDL file that should be used for the shadow model.

I would like to stress that at this moment the tweaker only works for FSX MDL files. I still need to test and adapt it to work with FS2004 MDL files as well. Also at the moment the tool might not be super robust yet, I am still working on that as well.

Custom shadow model

A recent question by Don Grovestine on the FSDeveloper forum made me look more at the shadows of FSX models. As everybody knows, turning on the shadows in FSX can have a considerable influence on the performance. But the FSX MDL format also does allow you to specify a custom model that is used to render the shadow. I did some testing today to determine how big the performance difference can be.

I started by making a rather complex object, with almost 6000 triangles. And I placed many instances of it at my favourite test location. The screenshot below shows how the scene looks with shadows. Without shadows I got a framerate of 22. When I turned on the shadows the framerate dropped to 15.

So next I made a simplified version of the model. Instead of 6000 triangles it uses only 250 triangles. And I inserted this model into my MDL as the shadow model. In this case I got 20 frames for the scene, so that is only a minor performance drop, especially compared to the original drop. The screenshot below shows the result. As you can see the shape of the spheres is less smooth in the shadow now.

In my test scene I placed a lot of objects and the object I  used was rather complex. I am not sure if the performance benefit would be that big for an airport as well. But at least it shows that using a simplified shadow model can help improve the performance.

To insert the shadow model into the MDL, I used an experimental function in ModelConverterX. I will develop it further to make it useful for general usage and then put it on the development release version. Hopefully it should be available in a few days.