New 3D renderer

[mauronen]

Hi Aaron.

If i remember well, in to-do-list there was a new 3D renderer that should substitute L3DTVi2. I've controlled just now and it's not present in lists (2.4c, 2.5, unscheduled)
I imagine that you want rely on an external 3D engine (like Torque).
If it's true, my opinion is that is a GREAT idea! L3DT is (for me) the best landscape generator of the world (and i, continously, make research through Internet to find better tools), and his job is to create landscapes.

If i can give you some tips, development plan of L3DT should focus on:
1) better improve heightfield generation (and it seem that it's present in your development plan)
2) externalize, through plugins, import/export processes (see above). It don't concerns export into 3D viewers or engines (see point 7 below).
3) externalize, through plugins, filters processes like erosions, ridges, etc. Actually it seems to me that an imported heightfield could be modified hardly (only in DM). Some ideas could be found taking a look at Leveller (that you just knows) and GeoControl (great tool concerning of filters, but poor about the rest).
4) externalize, through plugins, other modeling aspects like making rivers, waterfalls, roads that concerns non-vegetation aspects
5) manage vegetation coverage mapping (it's in your "wish list") that allows to place standard 3D objects "references" (created with 3DStudio, Maya or others well-known foliage tools like XFrog, SpeedTree, etc.). Import plugin could be the welcome in this case
6) foresee a step that manages skyies and clouds through import of a sky dome. It could be, again, realized with a plugin (very hard to develop, i suppose) or relying on images realized through an external tools like SkyPaint or Terragen
7) foresee a final step (in "Calculate" workflow) that allows export to a 3D environment relying on an external 3D engine that could have an internal 3D viewer, could export itself to an external 3D viewer or could build an application that package all stuffs in an executable app. Better if 3D engine/environment is open-source (Ogre, Irrlich, etc.), but low-costs engines/environments like Torque could be a valid alternatives

As you see the word "plugin" recurs many times. Therefore a L3DT plugin repository that contains free or purchasable plugins developed by willing developers could be set to host them.

In few words, L3DT could become a "plugin assembler" that have a well known pluggable interface. L3DT would become only an heightfield modeler engine that rely on external components. But L3DT would become too a "plugin", or an external tools for a 3D engine/environment.

If this scenario seems to you realistic, could announce a competion (like IOTM) for the best "plugin developer of the month, or the year", at the moment that L3DT SDK is ready. In this way L3DT would be a L3DT community's creature, and not only Aaron's creature.

I hope to be been clear in my exposition.

Cheers.

Mauro.

[Aaron]

Hi Mauro,

Don't worry, I'm not ignoring you. I'll post a (lengthy) reply when I get a spare moment, probably tomorrow or Monday.

Cheers,
Aaron.

PS: I totally agree with everything you said.

[mauronen]

PS: I totally agree with everything you said.

I think that you're totally agree particularly about:

...L3DT is (for me) the best landscape generator of the world...

Cheers.

Mauro.

[Aaron]

Hi Mauro,

Okie dokie, lots to answer here.

If i remember well, in to-do-list there was a new 3D renderer that should substitute L3DTVi2. I've controlled just now and it's not present in lists (2.4c, 2.5, unscheduled) I imagine that you want rely on an external 3D engine (like Torque).

You are correct, I do (or did) have that somewhere on the to-do list. I haven’t started work on it yet, but at the moment I’m thinking of including the 3D renderer as a plugin so that L3DT does all the heavy-lifting of loading/handling maps, and I don’t have to duplicate such code in another program. However, before I start on the renderer I want to clear some other items off the to-do list, such as manual editing of the heightfield, which is already long overdue.

1) better improve heightfield generation (and it seem that it's present in your development plan)

Yes indeed. I’ll be adding some modifications to the Design/Inflate routine to give more control from the design map (lower HF/DM ratios.) There will also be some mouse tools for manual editing, and the plugin system will allow for any manner of filter to be applied. My Spherical distortion plugin is an example of this.

2) externalize, through plugins, import/export processes (see above). It don't concerns export into 3D viewers or engines (see point 7 below).

This is pretty-much done. File I/O is now handled with plugins, and since plugins can now add menu options that call back to functions in those plugins, they can do whatever they like, including new import/export options. To make this truly universal I’ll have to add some extra functions to allow plugins to set/get settings from the project, but that’s only required if you’re doing something fancy.

3) externalize, through plugins, filters processes like erosions, ridges, etc.

I haven’t put these functions into plugins, but there’s nothing stopping anyone from doing so. L3DT’s erosion routines will probably stay within the application, but they will be callable by plugins in the near-future.

Actually it seems to me that an imported heightfield could be modified hardly (only in DM).

Agreed. The next release (v2.4c) should correct this.

4) externalize, through plugins, other modeling aspects like making rivers, waterfalls, roads that concerns non-vegetation aspects

This can be done also. It’s just a matter of someone writing the plugins.

5) manage vegetation coverage mapping (it's in your "wish list") that allows to place standard 3D objects "references" (created with 3DStudio, Maya or others well-known foliage tools like XFrog, SpeedTree, etc.). Import plugin could be the welcome in this case

I don’t think I’ll do vegetation in L3DT itself, but I might mess around with a vegetation plugin or two. The reason I’m cautious about this is because lots of users will expect different things here. For instance, you're talking about placing individual instances of vegetation on the map, whereas most game developers would use something like a density map (greyscale image) for placing vegetation, or use both approaches. Adding vegetation mapping by plugins makes it easier for these methods to be added piecemeal, without risking further delays to the L3DT release cycle (i.e. if development of a plugin doesn’t work out right, it doesn’t affect L3DT at all).

6) foresee a step that manages skyies and clouds through import of a sky dome. It could be, again, realized with a plugin (very hard to develop, i suppose) or relying on images realized through an external tools like SkyPaint or Terragen

Plugins again, I would say. I expect the skybox itself would be generated in another application (Bryce is also popular), as to program all the atmospheric modelling in a plugin for L3DT would be a bit insane. There’s no reason you couldn’t, of course, but it would be a lot of duplicated effort, given that you could just use the free version of Terragen to get really nice results. Anyway, a plugin for compositing the skyboxes would have to be designed specifically for the targetted 3D renderer, as each will typically have their own requirements for formats, sizes, etc.

7) foresee a final step (in "Calculate" workflow) that allows export to a 3D environment relying on an external 3D engine that could have an internal 3D viewer, could export itself to an external 3D viewer or could build an application that package all stuffs in an executable app. Better if 3D engine/environment is open-source (Ogre, Irrlich, etc.), but low-costs engines/environments like Torque could be a valid alternatives

On the first part, adding pages to wizards is a bit hairy and won’t be supported in plugins for quite some time, if ever. Though, as I say above, there’s no reason a plugin couldn’t add a menu option to export to a 3D renderer. As an example of this I’m going to make some plugins for ‘Export to L3DTVi2’ and ‘Export to VTP Enviro’ to replace the regular menu options, and to remove some fairly yucky code from the core of L3DT (particularly, native support for MGF). I’ll provide the source code as well, so other developers can see how it’s done.

One the second, bold, part; I'll see if I can get a hacky 3D renderer plugin going, and then maybe look at a standalone renderer. However, there are plenty of L3DT users who are far, far more capable at 3D graphics programming than myself, so if something like this is going to happen soon, it is most likely it will be done by a community member.

As you see the word "plugin" recurs many times. Therefore a L3DT plugin repository that contains free or purchasable plugins developed by willing developers could be set to host them.

I propose this page in the wiki be the plugin repository. Users can add their own plugin pages, can link to plugins hosted on their sites, and are of course free to charge whatever money/goods/services they see fit for their plugins. I will also offer to host plugins on bundysoft.com provided they are open-source, so that we can verify they’re safe.

In few words, L3DT could become a "plugin assembler" that have a well known pluggable interface. L3DT would become only an heightfield modeler engine that rely on external components.

Bingo! That’s been my goal for around about a year now, and I actually wrote most of the plugin system in Oct/Nov of last year, but didn’t quite have enough mojo to finish and debug it. Now that I know what I’m doing (or at least think I do), I’m pushing ahead with extending the plugin system so that developers can do pretty much whatever they want with it. L3DT then will be just an application that comes with a bunch of file format support, automatic map tiling/disk paging, some nice calculations, and an API that can be used to do whatever you want.

But L3DT would become too a "plugin", or an external tools for a 3D engine/environment.

Well, that’s a lot more complex. I would basically need to build L3DT (or significant parts of it) again as a DLL. This was actually what I was doing late last year, but the sheer size of the task was a bit much back then. I might try this again in the future, but I make no promises.

If this scenario seems to you realistic, could announce a competion (like IOTM) for the best "plugin developer of the month, or the year", at the moment that L3DT SDK is ready. In this way L3DT would be a L3DT community's creature, and not only Aaron's creature.

Well, the SDK is available, along with a bunch of examples, so I guess that’s the point we’re at now.

I certainly like the idea of a plugin development competition. Would there be any takers for a “Plugin developer of the Season” competition? (every three months, obviously.) I figure a month is too short, and a year is too long, but three months is just long enough for people to do some really cool things. At least initially, it might take some time for developers to get their head around the API, given that it’s still very new, growing by the day, only available in C++, not particularly well documented, and probably quite foreign at first glance (”What the hell is a ZVAR?”). I am, however, writing the documentation, and once the feature set has stabilised a little bit more, I’ll write some bindings for C, Pascal/Delphi, and maybe even FORTRAN if I’m feeling insane. Unfortunately I don’t think there’s really an option for using interpreted languages like C#, Java, or VB.NET, unless some helpful Guru knows how to call native functions from raw FARPROCs in those languages. I’m not all that familiar with them, but I’m pretty sure such hijinks are expressly verboten.

I hope to be been clear in my exposition.

Exquisitely so. Thank-you for both a very thoughtful, and a very timely post, and I hope that all that you ask will come to pass soon.

Cheers,
Aaron.

[jydog]

My. That was a lengthy reply. This is where it would be nice to have a print thread option to print out and read off line.

But as I said before L3DT takes precedence, cutesie things when you get bored working on earth shaking projects.

[mauronen]

Hi Aaron.

Now i'm busy (work and personal troubles), but this weekend i'l try to take some time to reply.

Cheers.

Mauro.

[Q-dad]

Hi Aaron,

Sounds great! Especially looking forward to Delphi support for plugins...

Not sure if this should rather be posted in the plugins forum, but would it be possible to include the following features, either directly inside the new 3D renderer engine, or as plugins to it...:

- Panorama export (360 degrees horizontally, but maybe even vertically as well), i.e. the user selects a camera point in 3d before pressing an export button, maybe have a separate preview window popping up and letting the user make adjustments to the position of the camera

- Video export of movement along a user definable path (waypoints) in 3d, including the possibility for the user to define several parameters at each waypoint, like for instance angle of camera, speed, etc.

[DeathTwister]

Cool,

New toys for Terragen thanks brother /smiles. That program works real good with L3DT, good combo untill the new viewer/exporter gets to the beta stage of Aaron's. Maybe still even after then, as TG does have stuff that is a help and Aaron can't get them all, he is after all just one man in a world of many. One and 2 man teams are good in some ways, suck in others. /winks........ROTFL

DT

[mauronen]

Hi Aaron.

Some replies:

5) manage vegetation coverage mapping (it's in your "wish list") that allows to place standard 3D objects "references" (created with 3DStudio, Maya or others well-known foliage tools like XFrog, SpeedTree, etc.). Import plugin could be the welcome in this case

I don’t think I’ll do vegetation in L3DT itself, but I might mess around with a vegetation plugin or two. The reason I’m cautious about this is because lots of users will expect different things here. For instance, you're talking about placing individual instances of vegetation on the map, whereas most game developers would use something like a density map (greyscale image) for placing vegetation, or use both approaches. Adding vegetation mapping by plugins makes it easier for these methods to be added piecemeal, without risking further delays to the L3DT release cycle (i.e. if development of a plugin doesn’t work out right, it doesn’t affect L3DT at all).

I didn't want to talk about placing individual instances, but about vegetation coverage map. I think that it would be insane to place tons of trees and bushes individually

3) externalize, through plugins, filters processes like erosions, ridges, etc.

I haven’t put these functions into plugins, but there’s nothing stopping anyone from doing so. L3DT’s erosion routines will probably stay within the application, but they will be callable by plugins in the near-future.

If those routines that concerns about heightfield modification (erosion, cliffs, peaks, slopes, ridges, canyons, dunes, hills, etc.) rest into L3DT there's a tight coupling between them and main application and limits developer to use those routines rather than use another. My hint is to decouple an abstraction from its implementation so that the two can vary independently (GoF Bridge Pattern). Thus allows developers to write your own filter modifier that manages erosions, cliffs and so on. A good approach IMHO is into Leveller and GeoControl.

Shortly i'll open new thread that explains my point of view about how could be L3DT. Naturally, you can absolutely ignore that.

Cheers.

Mauro.