Arno Gerretsen : GIS

QGIS

arno — Sun, 15 Mar 2009 18:05:00 GMT

Until now I was mainly using fGIS when I wanted to view or edit shapefiles, but since a year or two the updates to fGIS are no longer publicly available due to some license issue. Last week I bought an interesting book about opensource GIS and it brought the QGIS application to my attention again. I had tried an earlier build of this application some time ago, but then I found it more difficult to use than fGIS. Now I tried a more recent build and I am impressed, it is a really powerfull application and quite easy to use. So I will replace fGIS by QGIS as my default GIS application for viewing and editing vector data.

OpenStreetMap rocks!

arno — Sun, 10 Aug 2008 07:30:00 GMT

I had came across the OpenStreetMap project before, but only when I revisited it earlier this week the reail power of it struck me. Let me start with a little explanation about what OpenStreetMap is. It is a free Wiki world map as their slogan says. I has been created by people collecting GPS tracks and combining that with other copyright free GIS data to get a really free world map. On their website you can browse to all the data in a similar way as Google Maps.

But that is not all, the real powerful thing is that you can also download this data. That is what I took a look at earlier this week. I came across some shapefile version of the data and looked at their OSM XML format. Then I realized that this data could be very useful for FSX as well. Because it does not only contain vector data about roads and railways, but it also contains information about the location of bridges. And not the speak about all the point features that are included. If I would like to put mailboxes for the Netherlands in a scenery I could do that.

So I have started writing a little tool now that will take the OSM XML data and turn that into shapefiles with the correct information for shp2vec. And the tool also writes the XML style ExtrusionBridges for BGLComp. That way I am trying to get better looking rail and road data in FSX now. I am doing the tests on a small scale now (only one country), but I guess there is no limit later to extend it to entire Europe to improve the default FS scenery. And I am sure I will come across some other nice features in the dataset that can be used for FS as well.

Geoids, projections, datum, what?

arno — Wed, 10 Oct 2007 19:18:00 GMT

At work I came across this nice article that explains some basics of GIS data projections, datums, geoids, etc. Although the target audiance of it is more the oil industry, I think it would also be good reading for people making FsX terrain scenery and wanting to know a little more about what could be involved when using GIS data. Because knowing that the data is in the right projection, etc is very important for the best result.

Preserve those geo tags

arno — Sun, 10 Jun 2007 10:28:00 GMT

A while ago I wrote about using FwTools to create or manipulate GeoTIFF images for use with the resample tool. One common problem I did not mention right then is that the tags containing this geo information are not understood by every tool. If you for example open the image in a painting program, like PhotoShop or Paint Shop Pro, to make some changes to it, the image will be saved back to disk as a normal TIFF file. I have now extended the tutorial on the Wiki with a small section about how to save this geo information and later put it back. That allows you to easily restore it after editing the image in your favorite tool.

Working with FwTools

arno — Wed, 07 Mar 2007 21:20:00 GMT

I have added a new article to the Wiki that explains how you can use FwTools to create GeoTIFF files, which you can use for your photo scenery. There might be other tools to do such tricks, but I am familiar with FwTools as I use it at work. And besides that it is a open source project, which means it is free to use. So therefore I decided to make a little article about it.

The article describes how you can add positional information to an image and how you can reproject it to WGS84 if needed. Hopefully the article is useful to some of you and I will try to expand it with other useful information when I find some more.

Having fun with projections

arno — Wed, 14 Feb 2007 17:40:00 GMT

Using GIS data to create your scenery has become a lot more common in FsX. Vector data can for example converted from shapefiles and for new version of resample can process GeoTIFF images as well. If you have access to this kind of GIS data or can create it yourself, this makes creating the scenery a lot easier (and more fun as well).

But there are also aspects of working with GIS data that are hard to get for people not used to this. One of these is the use of projections. All FsX scenery design tools expect the data in the WGS84 projection. This means that you can usually not use any scanned map.

Today at work we came across a similar problem with the use of projections and I want to talk about it here as well, because it illustrates very nicely what a big influence a wrong projection can have.

For the flight simulator we have a visual database that contains the airport of Paris CDG and Sion, plus some terrain between these two airports. So the projection used for this database has been optimized for the area it covers, we are using a Lambert Conformal Conic (LCC) projection for this database. A few days ago they asked us if we can add an airport in Africa (Mali to be more specific) as well, because they wanted to fly from there to Paris for an experiment.

So our first thought was let's try to add the additional airport to the same database (which also means using the same projection). So we downloaded some Landsat images of Mali and started adding them to the database. But to our suprise (or maybe not really), the results were not very good. Below you can see a screenshot of the airport in the LCC projection of our Paris database (left) and a screenshot of the same airport when we use a UTM projection chosen for Mali (right).

As you can see, the heading of the runway differs something like 60 degrees. Which is of course much to much (how to explain the pilot that runway 06-24 is actually facing North). To be honest we already expect some trouble with the projection system, but we just want to try and see how worse it was. And it was much worse than we ever expected.

But I think this nicely shows that you can not ignore the projection you are using when working with GIS data. To prevent problems like the one we had, MS has chosen to use WGS84 for all scenery elements. So to get the correct placement, all you have to do as a scenery design is to make sure your data is in WGS84 or else convert it.

New FwTools version solves some of my trouble

arno — Fri, 26 Jan 2007 22:36:00 GMT

In the past I have already written about the markings I was working on for the Schiphol scenery and at the moment I am finalizing them. The tool I made to convert the lines of these markings into polygons uses OGR as I wrote about in that earlier post as well.

This week I updated to the latest version of FwTools and to my pleasant surprise it has become a lot more stable. In that past my tool sometimes choked on the amount of line segments I was feeding it, but now they are processed fine (although it still takes some time to work through them all).

The OGR library provides some nice functionalty to for example combine, subtract or clip elements in shapefiles. These are the kind of operations I do on my line segments as well, to give them a width and chop them into LOD grids. But now that shapefiles are also used for the terrain tools, this library might also become useful for people working on tools for the mesh scenery. So check it out.

Of course there are always some trouble left. Now that my markings tool can process them correctly, the triangulate algorithm I use when writing the SCASM source files crashes on me. I guess I have to look a bit more into triangulation now...

Am I a data junkie?

arno — Fri, 19 Jan 2007 21:09:00 GMT

At work my collegue and I, we are the ones working on the visual databases for the simulators, are quite well known for our excessive harddisk usage on our development systems. When we are using satelite images in these visual database, it is not that weird to have a few GB of images that we need to process. For example to cover the Balkan area or the entire country of the Netherlands. And then I am not even talking about high resolution images (that would increase the amount of diskspace used even more).

During the processing of these images into the actual databsae, we usually have to reproject them a little or apply some other tweaks on them and of course the final tool creating the database also generates some temp files worth a few more GB of data. So I guess you can imagine that we have to clean our harddisk quite often to keep the system running (from experience I know that 0 kB left of harddisk space does not really work well). And I guess the system administrator of our department has already gotten used to our questions for more diskspace.

And now the same seems the be happening at home with FsX. With the increased resolution of the terrain system in FsX the amount of data to process also goes up of course. I did a little test with high resolution images around Schiphol airport (16 cm resolution). Those photo alone were already a few GB worth of data, but when I added some intermediate files due to the conversion from the Dutch RD coordinate system to WGS84 and mosaicing of the different images into a few larger ones, the harddisk usage for this little test went up already. Now, after compiling the BGL files for FsX as well, the entire test project uses about 50 GB of diskspace (of which the final BGL files are only about 6 GB). Luckily my new PC at home has a big enough harddisk (for the moment).

With the changes to the FsX terrain system, I find it interesting to see that the process of scenery creation comes a step closer to the GIS data I have become familiar with at work already. The new shp2vec tool makes use of shapefiles and resample can process GeoTIFFs. Quite interesting developments and it shows that the difference between "professional" visual database and the FsX "entertainment" world is not that big.

Awesome

arno — Sun, 25 Jun 2006 14:05:00 GMT

I would like to point your attention to cool OpenSource project this time, OSSIM. At work we make use of this tool to mosaic different aerial images for example. But it can do a lot other image manipulating related tasks as well, like color matches between different images or reprojecting images to a different projection. So have a look at this awesome tool.

While reading the manual of OSSIM, I noticed that it should also be able to do image tiling. This could be a useful feature when trying to make the tiles for a high resolution ground layout. But I haven't been able to get that feature working completely, I'll make a new post about this once I got it fully working.

Markings

arno — Wed, 24 May 2006 21:32:00 GMT

It has been a little quite here lately, but that does not mean I am not doing anything. So time for a little update. Apart from the fact that I have been away for a few days for work (attended a workshop in Engeland), I have also been working on the apron markings for the Schiphol scenery. Let me start with a little introduction on what we want to achieve in that scenery.

We want realistic markings, so that means we need more than only a yellow center line. We also need the red clearance lines for example or the white tow lines. And as the cream on the pudding we would also like to put in the road markings on the airside of the airfield (these will probably be optional, to keep the users with a not so fast PC happy as well). As we want more than only some yellow markings, it is clear that the XML options to place markings are not sufficient for use. Although I hope that in the future these XML markings will become more flexible, so that they allow those red lines and other cool stuff as well.

Another things that I should probably mention, is that we have been lucky to receive some data about which markings are where. That means that we do not have to draw them all by hand, but that we can use this data and convert it to some scenery. In the past I have made a converter for this and it created Fs2000 style roads for the apron markings. I used these as they are the easiest way to place lines with a certain width in the scenery and assign your own custom texture to it. But they also have a few downsides (of course). For example they are not floating point, so to prevent gaps between them you should use a rather small scale of the reference point. And another negative point seems that they are a little heavy on the frames if you use a lot of them. And I can assure we use them a lot, as there are a couple of ten thousand line segments on the airfield.

So now I have been trying to upgrade the converter a little bit and produce these markings in another few. And until now that is working out quite nice. Instead of placing all the markings as narrow roads, I am now drawing them as polygons using the floating point commands (like the Fs2002 GMax gamepack output for ground polygons, but then created by my own tool). The main advantage of this is that it seems to be better for the performance and also it makes it slightly easier to connect all the lines nicely together without gaps. To generate these polygons from the line data I am using the OGR library I talked about before. With this library I am merging all line segments in a certain area together and I then give this total line a buffer (so a width around it). The advantage of this method is that as all lines are first put together, the connections between the different segments are almost perfect. And luckily the library does most of the hard work for me.

I will try to post a few comparison images in the coming days, when I continue to test my new tool.

GDAL

arno — Thu, 16 Feb 2006 20:10:00 GMT

About a week ago I wrote about OGR, but actually that library is part of a bigger one. That is the Geospatial Data Abstraction Library (GDAL). And this library offers some very nice features as well.

Let me give an example. It comes with a few tools for example and at work I used one of these to convert a scanned airport chart into a GeoTIFF image. I just told the program the latitude and longitude of a few pixel locations (threshold, airport reference point, etc). With this information the tools were able to convert my image into WGS84 projection and after that the image lined up with our runway data perfectly!

These are powerful features that can also be of use for FS scenery designers. The more I think about it, the more I think I should try to use features like this in a future tool...

OGR

arno — Wed, 08 Feb 2006 19:43:00 GMT

The last few days I have been using the OGR library at work. As the website says, it is a Simple Feature Library. It allows you to rather easy read, manipulate and write vector features in different file formats (for example DXF or SHP).

This can be very useful, as you do not have to create your own file format parser, as I did for my DXF2SCA tool. And the manipulation of the vector data is really nice. At work we for example use the union function to make one big polygon out of different taxiway segments. And the buffer function is used to calculate a grass area around all features of an airport. This way we can rather easy make some generic airports for our visual databases.

I think I will also give this library a try for my next (updated) version of DXF2SCA. That could certainly save me some coding and allows me to focus on the scenery output for FS, instead of reading a certain file format.

Datums - Who Needs 'Em Anyway?

arno — Tue, 07 Feb 2006 18:17:00 GMT

Some time ago I made a post about the importance of knowing what projection your data uses etc. Today I came across a nice article on the ESRI site discussing the importance of knowing your datum as well.

TerraVista

arno — Thu, 08 Dec 2005 21:26:00 GMT

At work we use a tool called TerraVista to create the visual databases. This tool has some very interesting features, that are certainly part of my imaginary ultimate-scenery-design tool.

TerraVista for example allows you to import a lot of different data types. DEM altitude data, aerial or satelite photos, vector data for roads, etc. You can import them all, from a lot of different formats. If TerraVista knows the projection used in the data, it is able to combine them all for the project you are working on.

You can also determine rather flexibile how you want to use your data. Based on some attributes of your vector file, you can for example determine if it should be used to draw a road, a river or maybe a line of light poles. After you have defined these processing passes, the compiler takes care of the rest of the work, while it generates the visual database.

One of the things it can for example do during the generation is cut out different polygons. In Flight Simulator we are used to layer polygons if we have different ones on top of each other, but the Image Generator we use at work prefers to have only one polygon at a certain position. So TerraVista makes sure that the overlapping parts are cut away.

This is of course a very short discription of what the tool can do. It is so flexible that it sometimes even becomes hard to get something done, because you get lost in all the parameters that you can set.

But what about the ultimate-scenery-design tool? I would love to have a tool that I could provide with some altitude data, some photos, data for roads and that would then create the proper scenery files for me. Without me having to wonder about the LOD of the terrain and complicated things like that.

In that perfect imaginary world we would only need two kind of tools. One to design our 3D objects (GMax for example) and another tool that takes care of all the mesh and XML parts of the scenery. Maybe somebody would create that perfect tool in the future?

I know there are tools that can do part of this, but wouldn't it be cool if we had all this in one tool? Especially for new designers it can be very frustrating to learn that they have to use yet another tool to get something done that sounds very simple at first.

As I design tools myself, I know how much work it would be to create a tool like this, that is powerfull on one hand and easy to use on the other hand. Maybe I can better keep dreaming.....

The projection hell

arno — Mon, 05 Dec 2005 22:49:00 GMT

The different projection systems used in geographical data might be one of the aspects of MSFS scenery design that most designers know hardly anything about. But it surely a very important thing.

Almost each country has its own projection system and although most of these systems will give you the position in either degrees or meters, that does not mean that it will all fit together. So when using data to design your scenery it is a good idea to take a look at which projection system they use.

In the terrain SDK it is written that the MSFS mesh terrain uses the WGS84 projection system. But an interesting question remains which projection is used for the scenery objects (so the coordinates in meters you define in the MDL object file). As objects are in general small, the influence of the projection system will be a small, but it would still be interesting to know.

While working on my Dutch sceneries I first met all the trouble that different projections can give. We got some data that used the Dutch Rijksdriehoekscoordinaten system (the position is defined in meters from a church tower near the center of the country). But compared to the WGS84 projection system, the Dutch RD system also has an additional rotation of a few degrees. So when you think you can just use the position in meters, you end up with your scenery in the wrong location. In the end we luckily found a set of good formulas to do all the transformations to WGS84 and back.

Another example where you will meet different projection systems is when you use a background image in a program like Ground2k4 or SceneGenX. The mesh scenery and the XML style scenery are all defined in WGS84, but a lot of maps are in meters. So in that case you would have to reproject the image to get a perfect match. SBuilder is a tool that has added this reprojection functionality. If you do not take notice of this, you might be positioning your scenery at the wrong position, especially when you get further from the reference point.

For those that have become really enthousiastic about projection systems after reading this, take a look at the Proj4 library. With this library you can easily convert between almost any projection system. At work I have also used this library to build some conversion tools.