L3DT for Spring, anyone?

[Forboding Angel]

Here is the error when I try to export as one large file (25600x25600).

CMapWrap::CombineMosaic error:
- Cannot initialise submap

CL3DTio_FIApp::SaveMapFile error
- FreeImage failed to allocate bitmap

CExtDLL::SaveMapFile error
- call to DLL function 'SaveMapFile' failed

CMapGroup::SaveMapFile3 error:
- save failed

[Madcowthomas]

Believe exporting a map that is 8192 and above you will receive that error. You need to use the " Use disk-drive paging (mosaic title).

[Aaron]

Hi Forboding,

To combine a 256k x 256k mosaic texture needs ~1.8Gb of contiguous RAM, which is more than Microsoft will give me (~1.2GB max). I think this thread gives a decent overview of the memory limitations as they apply to image exports. Basically, to overcome that limitation, I (or some other enterprising developer) would have to write some new plugins for the image file formats, using image libraries that don't require the maps to be in one contiguous memory block. It's easy to do with bitmaps (thus Grout), but it's harder with jpegs, pngs, etc. It can be done, but it's just a bit harder.

As for the compiler; I'm prepared to have a crack at it. SM3 looks pretty easy, and it should be a doddle to export all the blendmaps &c required. However, am I right in thinking you're after the old-school SMF/SMT/SMD/SM7 thingyhoozit? If so, can you point me in the direction of some documentation? I gather that the SM7 file is a 7-zip of an SMT, SMD and SMF. The SMD is a text file, which describes stuff like starting positions, skyboxes, etc. The SMT and SMF are created by MapConv from the metal map, height map, feature map and type map. Unfortunately, I can't seem to find any further documentation on these two (presumably binary?) formats. Do you know where I should be looking?

PS: Did I mention that you can now make feature maps, metal maps, etc in L3DT? In the latest dev build you can create custom map layers. Re-reading your tutorial I see that to position these features correctly, you will probably want at the texture and/or heightfield overlaid on the metal/feature/etc map display when painting those masks. That can currently be achieved using the 'view->image overlay' feature, but I'll be doing something better with this in a future release. Also, have you tried the new 3D heightfield editor?

Cheers,
Aaron.

[Madcowthomas]

Wish we had more tutorials on this wonderful product. So many switches to and knobs to turn that start to get lost. Myself I don't learn from reading a technical book, I learn by watching a talking and trying things out. Hope some are advanced users take the time someday and create some more tutorials.

[Forboding Angel]

Thanks for looking into that stuff Aaron. I wish I could personally hire you to make l3dt mapping a concrete part of spring (yet another pipe dream ).

The person to talk to would probably be Tobi. JCNossan would be probably the best person to talk to, but he has kind of washed his hands of spring in favor of Command Engine (recreation of spring engine with much more flexability etc).

I'll try to poke at Tobi and get him in contact with you.

Here is the short version of what I know about the compiler... It takes the texture file and breaks it into a series of dds tiles, copying tiles whenever and wherever it can. It's really sloppy, and quite inflexable. I've pushed it to it's limits which sadly, doesn't take a lot of pushing.

When spring renders the map it only renders the tiles that are onscreen, and that can be set with the viewradius. The map format is bulky, ineffecient, and damn near impossible for a newbie to use, but it does work.

Contrary to what someone in this thread posted, a small minority of spring users complain loud and hard about filesize, but according to past polls, 97% have broadband, and tbh, I could give a damn about the other 3%. For extreme quality, there is always a downside, and in this case, it's filesize.

The smf is some sort of binary instruction set, I imagine it's comperable to the MMF files l3dt uses (but I don't know for sure). THe smt is the file that contains the dds tiles. The smd is an instruction set (modified in notepad) that controls environment settings such as metal amounts, gravity, lighting (non dynamic), wind, etc.

About creating the metal/feature etc maps in l3dt... that is uber win. Only one things scares me, metal maps must be 1 pixel larger (as the heightmap must be). If they must be created in l3dt at normal size then resized, that could cause a problem, as with metal amounts that 1 pixel can throw your metal amount calculations into complete disarray.


One thing that strikes me as neat is... What if l3dt could be the official map making program for spring, literally. As in possibly intigrating l3dt for spring use to a T so that everything for spring SMF and SM3 mapping could be done in l3dt? Might possibly increase some $$$ for you, wile giving spring users an excellent product for creating beautiful maps.

Hell, I could whip up some great stock climates for it etc.

Meh, aren't pipe dreams fun?

[Forboding Angel]

Found an smf file breakdown that might prove useful...

http://spring.clan-sy.com/wiki/Map_File_Format

[Aaron]

Hi Forboding,

Just a quick update: I've finished the SMT part of the plugin with the help of Tobi's new documentation. I'll now move on to the SMF and SMD parts.

About creating the metal/feature etc maps in l3dt... that is uber win. Only one things scares me, metal maps must be 1 pixel larger (as the heightmap must be). If they must be created in l3dt at normal size then resized, that could cause a problem, as with metal amounts that 1 pixel can throw your metal amount calculations into complete disarray.

You can make metal/feature/etc maps at any size you like in L3DT using the new map layer feature, so no resizing is necessary.

Cheers,
Aaron.

[Aaron]

Okie dokie, more progress:

SMT plugin seems OK. SMF plugin needs more debugging (it crashes Spring). To help with debugging, I'm building a SMF/SMT decompiler in the plugin. So far it loads the height, type map, minimap and metal maps OK. Some time tomorrow I'll finish the texture tilemap importer, and if all goes well I'll look at the vegetation map too. I've no idea what I'm going to do with features, but I'll cross that bridge when I burn it down (or something like that...)

Update:

The decompiler is almost done. Here is a screenshot of SmallDivide running in Sapphire:



Cheerio,
Aaron.

[Forboding Angel]

OMFG you are made of win!!!!!!

THank you!!! Where do I mail this check for $1,000,000 to again?

[Aaron]

Hi FA,

Don't mention it. I've written-up a brief usage guide on the Spring forums here:

http://spring.clan-sy.com/phpbb/viewtopic.php?t=11029

Could you provide some expert feedback on what else is required?

Best regards,
Aaron.

[carldt]

This is what I was after all along...

Aaron, I can't find source code for the Spring plug in, can you please make this source code available so myself and others can work on the items in the TODO list.

Thanks

[Aaron]

Hi Carl,

I'll upload the source later today, after I've cleaned up, LGPL-ified everything, etc.

I should point out that the Spring importer/exporter is actually a collection of three plugins that call one another using extension functions. They break down like this:

L3DTio_SMT.zeo - Loads/saves SMTs, also compresses/decompresses minimap using DXT1 algorithm.

L3DTio_SMF.zeo - Loads/saves SMFs, calls L3DTio_SMT to load/save associated SMTs and compress/decompress minimap.

L3DTio_Spring.zeo - Mainly provides coordinated user-interface for SMF/SMT plugins. Also creates metal/type/etc maps, and calculates minimap from texture.

Still to be done are:

L3DTio_SMD.zeo - To load/save SMD files, and provide dialog interface for editing SMD data.

L3DTio_7ZIP.zeo - To load/save 7ZIP/ZIP/etc archives.

L3DTio_SD7.zeo - To load/save SD7 archives using SMF/SMD/SMT/7ZIP plugins. Once complete, L3DTio_Spring will be modified to import/export SD7 rather than SMF.

L3DTio_DXT.zeo - Separating out the DXT1 compression/decompression algorithm from L3DTio_SMT.

ImageKit.zeo - A plugin for manipulating/combining images. Will be used to 'bake' metal patch textures onto the ground texture, amongst other things.

The idea for having all these plugins is borrowed from the Unixian* way of doing things: each tool does just one thing, but does it well. In theory, this should allow each component to be developed/modified/improved independently, and also potentially re-used easily for other purposes (I have other purposes for ImageKit, L3DTio_7ZIP and L3DTio_DXT in mind).

Is there anything you think I've missed there, or should be done differently? Is there any particular chunk you'd like to have a go at first?

Cheers,
Aaron.

* Hah: did you mean Tunisian?

[carldt]

Hi Aaron,

I found the 7z SDK here, the license seems really permissive.

http://www.7-zip.org/sdk.html

I can have a crack at implementing this plugin.

Carl

[Aaron]

Hi Carl,

Here's the source code:

• L3DTio_Spring
• L3DTio_SMF
• L3DTio_SMT

I can have a crack at implementing this plugin.

Fantastic. In that case, I think my next task is the image-combiner plugin and the metal-patch stuff.

Cheerio,
Aaron.

[Aaron]

Hi All,

Some more progress: I've finished the image-blending plugin that will be used to automagically bake metal patches onto your texture. Here is a demo screenshot:



I still have to wire this up with the main Spring plugin, but that should be done some time this week.

Cheerio,
Aaron.