L3DT users' community
Large 3D terrain generator

Time remaining eratic

Please report software faults here.

Time remaining eratic

Postby metalliandy » Sat Nov 17, 2007 3:56 pm

Hey Aaron,

I started to render a 32x light map last night about 4am and left it over night to run, the Time remaining dialogue stated that it would take 3 hrs to run.

Upon waking this morning the dialogue stated 250 hrs and seems to jump around from anywhere from 1s to 1500hrs.
Currently it seems to be stuck on cast shadows with 689 hrs to go, though the green progress bar is just over half way.

I guess its hard to estimate how long it takes big maps to render :S

maybe its not a bug but i thought i would mention it :)
metalliandy
Doyen
 
Posts: 103
Joined: Tue Mar 20, 2007 11:28 am

Postby Aaron » Sun Nov 18, 2007 8:53 pm

Hi Andy,

Thanks for the bug report. Fixing the time estimate for light mapping is on the to-do list for v2.6.

Cheers,
Aaron.
User avatar
Aaron
Site Admin
 
Posts: 3696
Joined: Sun Nov 20, 2005 2:41 pm
Location: Melbourne, Australia

Postby metalliandy » Sun Nov 18, 2007 9:30 pm

Hey Aaron,

Thanks for the reply,

Looking forward to 2.6 :)

Any idea how long a 32x LM should take to render, mine is still going and its been running for 29 hrs and it still calculating shadows lol

Its rendering a 65536 pixel map

My PC spec is AMD athlon X2 5600+(2.8ghz) and 2 GB ram with XP home

Thanks for looking :)
metalliandy
Doyen
 
Posts: 103
Joined: Tue Mar 20, 2007 11:28 am

Postby Aaron » Mon Nov 19, 2007 11:26 am

Hi Andy,

Sorry, I can't make a reasonable guess based on system specs. Too many factors affect calculation speed, including the shape of your map, the services running in the background, etc. The calculation is fairly linear though; so if you've made, say, a 4096x4096 light map, then the time taken to make a 65536x65536 pixel map will be about 256 times longer, since there are 256 times as many pixels.

Cheerio,
Aaron.
User avatar
Aaron
Site Admin
 
Posts: 3696
Joined: Sun Nov 20, 2005 2:41 pm
Location: Melbourne, Australia

Postby metalliandy » Mon Nov 19, 2007 6:27 pm

Hey Aaron,

Ok no worries m8 :)

I woke this morning to a crashed PC...half way through rendering the textures :(

Is there anyway i can point L3DT to the LM.mmf to continue just rendering the textures...it was the calculations that took the most time the textures only would of took a few hrs.

What i mean is...are the calculations saved in the LM.mmf, or done on the fly.

At the moment im sitting with map15_LM_x127y63.bmp as the last texture done, with all the textures weighing in at a whopping 6gb

Just a bit gutted that i might of lost 2.5 days of rendering..lol
metalliandy
Doyen
 
Posts: 103
Joined: Tue Mar 20, 2007 11:28 am

Postby Aaron » Mon Nov 19, 2007 9:28 pm

Hi Andy,

I'm sorry to year you had a crash.

Is there anyway i can point L3DT to the LM.mmf to continue just rendering the textures...it was the calculations that took the most time the textures only would of took a few hrs.


Yes. The completed lightmap should be listed in the project file, so L3DT should still load the lightmap. If it doesn't, there are two ways to get it back. The quick way is to manually edit the project file using notepad. Under the 'maps' list, enter the following section:

Code: Select all
<varlist name="LM">
    <string name="Filename">LM\MyMapName_LM.mmf</string>
    <string name="MapType">24bit</string>
</varlist>


(where you change 'MyMapName' to the appropriate map name).

Alternatively, you can use the 'File->Import->Custom map layer' option. Here, select the MMF file, set the layer name to "LM" and the layer type to "BYTE[3] (colour map)". This option will be slower than the notepad path above, because it involves the copying of the map data.

Code: Select all
What i mean is...are the calculations saved in the LM.mmf, or done on the fly.


The calculation results are saved, but no other calculation state information is saved because it would make the process absurdly slow(er). Thus, a calculation can't be resumed if it stops part way through a map layer.

Code: Select all
At the moment im sitting with map15_LM_x127y63.bmp as the last texture done, with all the textures weighing in at a whopping 6gb


Wait a second...I thought you said the light map calculation was finished? If the light map tiles aren't all there, the lightmap calculation didn't complete. The lightmap calculation has two parts; the first generates a shadow map, and the second uses the shadow map, the normals, the water map, etc to generate the light map. The light map tiles are saved in the second stage. If that didn't complete, then I'm afraid the data is lost.

At present the shadow map is not stored for later use, although I think I'll now change that for the next release so that you don't lose it if the system crashes during the 2nd stage (now on the to-do list).

I've also popped the 'calculate light map area' option onto the top of the priority list, and I should make a start on it in the next few days if the winds are fair.

Best regards,
Aaron.
User avatar
Aaron
Site Admin
 
Posts: 3696
Joined: Sun Nov 20, 2005 2:41 pm
Location: Melbourne, Australia

Postby metalliandy » Mon Nov 19, 2007 11:06 pm

Hey Aaron,

Thanks for the reply :)


Im sorry if my post was a little confusing...i assumed that the calculations were done as separate entity's, thus maybe allowing the shadows etc to be rendered separately from the textures :S

I shall put this down to a learning experience..lol

Thanks for putting the stuff onto the to-do list, they will be a great help:)
metalliandy
Doyen
 
Posts: 103
Joined: Tue Mar 20, 2007 11:28 am

Postby Aaron » Thu Nov 22, 2007 11:04 am

Hi Andy,

...thus maybe allowing the shadows etc to be rendered separately from the textures :S


The shadow map is actually rendered separately to the texture map. However, the shadow map and the light map are rendered sequentially, and I think I was just confused by your terminology (I think you were using "texture map" for "light map"). Your crash was occurring after the shadow map, in the light map, but before the texture map.

Anyhoo, in the latest dev build I have added an option that once the shadow map has been generated, it is saved and can be re-used. It's still generated the same way (i.e. in-line with the light map), but once the shadow-map has been generated once, you then get the option when re-generating the light map to re-use the old shadow map or make a new one (a new check-box will magically appear in the light map wizard). However, note that if you change the light map resolution, L3DT will have to re-generate the shadow map regardless of your preference.

Also, as noted in LordThrash's thread, the crashing problem seems to be related to the multi-threaded calculation manager (well, I blame Microsoft, but what can you do? I should have read the Win32 specs more closely.). If you disable multithreading temporarily, you'll be able to make maps of any size without worrying about those crashes. I'll have a fix out as soo as I can, but it's quite tricky work.

Cheerio,
Aaron.

PS: Since shadow-mapping is not multithreaded, you don't have to worry about crashes in that part.
User avatar
Aaron
Site Admin
 
Posts: 3696
Joined: Sun Nov 20, 2005 2:41 pm
Location: Melbourne, Australia

Postby Aaron » Fri Nov 23, 2007 11:21 am

Hi All,

I've had a look into the erratic time-remaining problem for shadow-casting, and it turns out that the time remaining for this particular calculation cannot be reliably predicted in advance. It's just too non-linear, and dependent on the contents of the heightmap. So, instead of providing a wildly fluctuating time estimate, it now won't bother to estimate the time remaining for the shadow-map calculation.

On the up-side, I have re-built the progress map display for the shadow map calculation, so at least you'll know what it's up to:

Image

Cheerio,
Aaron.
User avatar
Aaron
Site Admin
 
Posts: 3696
Joined: Sun Nov 20, 2005 2:41 pm
Location: Melbourne, Australia

Postby metalliandy » Sat Nov 24, 2007 10:47 am

Hey Aaron,

Thanks for the reply and sorry for the lateness of my own.

The progress map improvements will be much better than the old version...Nice work m8 :)

I shall also try a 32x map with a single core :)

Thanks for the hard work :)
metalliandy
Doyen
 
Posts: 103
Joined: Tue Mar 20, 2007 11:28 am


Return to Bug reports

Who is online

Users browsing this forum: No registered users and 11 guests