L3DT users' community
Large 3D terrain generator

Multi processor issue in dev 12.12 pro dev build 0

Please report software faults here.

Multi processor issue in dev 12.12 pro dev build 0

Postby demi » Thu Feb 21, 2013 11:35 pm

I was doing ok with 8K maps but I changed the size to 41K and when I do a attrib map now the cores seem to lock up. I investigated the problem in the log and found that it would hang or get out of sync after about 5 minutes. I though maybe my processor was going bad but a hard test of the cores shows it to be fine.

It doesn't have any problems with the map in the released version but in this dev version it appears that the cores are not syncing correctly.

I set it to use 2 of the 4 cores and forced sync which appears to have fixed the problem but you may want to check it.

All is well with 2 cores :)
demi
Oracle
 
Posts: 227
Joined: Thu Nov 24, 2005 4:56 am

Re: Multi processor issue in dev 12.12 pro dev build 0

Postby Aaron » Fri Feb 22, 2013 11:54 am

Hi Demi,

Thanks for the bug report. I'll see if I can reproduce the fault. Can you please send me your project's map definition file (*.def.xml)? Please send to aaron@bundysoft.com.

Best regards,
Aaron.
User avatar
Aaron
Site Admin
 
Posts: 3687
Joined: Sun Nov 20, 2005 2:41 pm
Location: Melbourne, Australia

Re: Multi processor issue in dev 12.12 pro dev build 0

Postby demi » Sat Feb 23, 2013 3:05 am

I sent the current def. Also included the current running debug and log. It has been running since like 8 or 9 PM yesterday and only done like one row of 40 blocks. I realize that the thing is doing 19 alphas but it never dogged down like this on a 8K or 32 k map before.

update:

ziped it
demi
Oracle
 
Posts: 227
Joined: Thu Nov 24, 2005 4:56 am

Re: Multi processor issue in dev 12.12 pro dev build 0

Postby Aaron » Sat Feb 23, 2013 11:12 am

Hi Demi,

Thank you for the definition file and the logs.

The log seems to suggest this is a tile cache problem. There appears to be a bit of cache thrashing going on, and it looks like the memory compactor has crunched the alpha map tile caches down to a level that is quite inefficient. I'll run some more test and let you know when a fix is available. It might take a while, given the size of the maps involved.

Thanks for the bug report, and sorry for the inconvenience.

Best regards,
Aaron.


FYI: the alpha map generator isn't multithreaded (other than the UI update thread), so it's less likely to be a multicore processing problem.
User avatar
Aaron
Site Admin
 
Posts: 3687
Joined: Sun Nov 20, 2005 2:41 pm
Location: Melbourne, Australia

Re: Multi processor issue in dev 12.12 pro dev build 0

Postby demi » Sat Feb 23, 2013 12:53 pm

Hi Aaron,

Well this is a interesting problem. I decided to do some investigating in the logs and looking at the first layer mosaic that were generated. I noticed a unique thing that there were multiple files on one block all 1k in size and the log was saying the cache was growing in a warning message.

I decided to make the alphas a size with the power of two and see what would happen since the map engine seems to be making the mosaic with a pixel size of such. I started the alpha map generations and to my surprise the map generated in under 3 hours!

conclusions:

It appears that the map engine does not like odd sized maps that are not divided up into power of two mosaics.


The reasons I chose this size is because the game engine has a set pole distance for 5 different map resolutions. I made the original map 8kK pixel at 4 meters per pole so it was sized at 328 100 meter blocks. This was the lowest setting of resolution so I could look at the map in the client and decide if I wanted to regenerate or go with it. This resolution only allows a fix set of textures and is called a simple terrain. The next size is 100/32, 100/64, 100/128 (recommended for performance) and 256/100 resolutions with unlimited textures. A 328 block map with recommended 100/128 (0.78125) resolution is 41984 pixels. I resized the original map but I used the default 1K pixel mosaic.


edit: instead of 1024 x 1024 whihc is 41 blocks I resized the alpha generation to 1312 x 1312 which is 32 blocks instead of 41. BTW that fixed the multicore problem too
D
demi
Oracle
 
Posts: 227
Joined: Thu Nov 24, 2005 4:56 am

Re: Multi processor issue in dev 12.12 pro dev build 0

Postby Aaron » Sat Mar 02, 2013 8:17 pm

Hi Demi,

I've just uploaded v13.03 build 0, which should properly fix the memory/cache and multicore problems, and run faster with tiles un-synchronised too.

Regarding tile sizes vs. performance: many of L3DT's algorithms will run significantly faster if the tile sizes of all maps are equal, and if they are powers of two. This eliminates the need for linear interpolation of pixel values, which is really slow. There are also other optimisations that kick in under these conditions to make the calculations faster.

Cheerio,
Aaron.
User avatar
Aaron
Site Admin
 
Posts: 3687
Joined: Sun Nov 20, 2005 2:41 pm
Location: Melbourne, Australia

Re: Multi processor issue in dev 12.12 pro dev build 0

Postby demi » Tue Mar 05, 2013 12:07 pm

Thanks.
demi
Oracle
 
Posts: 227
Joined: Thu Nov 24, 2005 4:56 am


Return to Bug reports

Who is online

Users browsing this forum: No registered users and 3 guests

cron