large bt file import error

[monks]

Hi Aaron, I've tried to load in a 20x20 K bt but I'm getting an error. I don't think this is the latest version I'm using (I've just tried all 3 instances on my desktop: 2.5 build 10, build 19 and the 2.5 pro non-dev).

I get an error dialog:

operating system could not allocate 1.490 GB of contiguous RAM. (I have 2 x 2 GB)

then-

I get the same log errors:

CMapWrap::InitMap error
- cannot allocate SubMap

CL3DTio_BTApp::LoadMapFile error
- cannot initialise map

CExtDLL::LoadMapFile error
- call to DLL function 'LoadMapFile' failed

CMapGroup::LoadMapFile2 error
- error loading file

CMapGroup::HF_ImportFromFile error:
- call to LoadMapFile failed



The bt is a slightly odd size: 20003 x 20002 I think- it's definitely not an exact 20K put it that way.
The bt opens ok in WM (64 bit enabled tho) so it's not corrupted.


//Edit ditto for the latest dev build.

monks

[Aaron]

Hi Monks,

I've uploaded an update to the BT plugin here:

http://www.bundysoft.com/wiki/doku.php? ... :l3dtio_bt

When the plugin finds it can't allocate the required memory (more on this below), it will initialise the map as a mosaic and load from the BT file into it. This will work for heightfields up to 128k x 128k pixels. If you want to get the map back out as one big BT at the end, you can simply export as a non-mosaic via the usual means.

In the current implementation, the plugin will still throw the same errors. However, if this trick works OK for you I'll modify L3DT to do this automatically and silently. In fact, it's been on my to-do list for a while to automatically split really large maps into mosaics, kind of like how Photoshop does it (i.e. the user never knows).

As for why the plugin can't allocate the 20k x 20k map; the 32-bit version of the Microsoft Foundation Classes (MFC), which I use, will not allocate more than 1.2GB of RAM in one contiguous block. One option is to update to the 64 bit version (which I'll do eventually), but that would only get us to the next ceiling, which is the amount of RAM in your box. If you did nothing else on your computer, this would get to you to a 32k x 32k heightfield (in reality, less). By splitting the map into a non-contiguous mosaic we go past that, to sixteen times the map area, and because it's a mosaic, you can also do other things without running out of RAM (e.g. make a texture, render in 3D, etc.)

Please let me know if there are any problems.

Cheerio,
Aaron.

[monks]

Thanks Aaron, I've just run the water wizard and I had a few more error reports but it has calced and displayed fine- just for the record:

Water flooding wizard
Clculating mipmaps

CSubMap::InitMap error- insufficient memory
the os couldnot allocate 4 MB of contigupous RAM
system memory state: RAM_CUR_ALLOC = 1.607 GB, RAM_AVAIL = 1.143 GB
L3DT memory state: MAP_RAM = 1.246GB, MAP_PAGED = 8.o82GB.



pressed OK, then got



CL3DTio_BTApp::LoadMapFile error
- cannot initialise map

CMosaicMemTable::AllocMap error
- could not initialise map!

CMosaicMap::SubMapAlloc error
- cannot init mem table

CMosaicMap::PrepareSubMap error
- cannot init sub map

CMosaicMap::SetPixel error
- cannot prepare sub map for pixel not in memory
- pixel 4096, 0 of 10001x10001
- tile 4,0 of 10x10
- name Mip2

CMapWrap::SetPixel general error

mxMip_GenMipsForTile error:
- cannot set tile value

CMapWrap::GenMipMaps error:
- call to GetMipsForTile failed

...but it has calculated and dsiplayed it fine.

monks

[Aaron]

Hi Monks,

Can you please send me the log file from that session? It's in the start menu, under 'Bundysoft->L3DT [version]->L3DT log file'.

In other news, I made a mistake with that plugin and have uploaded an updated version. I forgot to un-comment some code. This won't fix the errors you've seen, however.

Best regards,
Aaron.

[monks]

-the log's in the post.

monks

[Aaron]

Hi Monks,

Thanks for the log. This fault was an unintended consequence of a change to the mipmap generation system that I had made for the sake of faster megatexture rendering in Sapphire. I've more carefully targeted that optimisation so that it shouldn't trigger the general memory blow-out that seemed to have occurred within the mipmaps of your map. I've also made some other modifications to the mipmap system to moderate the memory usage. I'm testing it now, and should upload another release candidate today or tomorrow. Thanks again.

Cheerio,
Aaron.

[Aaron]

Hi Monks,

I've just uploaded RC2.1, which should fix the later bugs you found. It will still throw a few messages when you load the 20k x 20k BT, and the initial mipmap generation progress bar will appear to stall for a minute or two (long story...), but everything else should be OK. Please let me know if you find otherwise.

Best regards,
Aaron.