ZeoGraph requests

[David Walters]

Hi, I've been using ZeoGraph a bit tonight to create a map that I can use to model vegetation density. I've got a few suggestions for things that would make life easier.

* Can you add 'specular map' support to the GetProjectMap node. I'd like to use it as an array of scalar values - in this case to raise the vegetation density slightly on the 'sunny' side of a terrain.

* A Gradient node would be great for reducing density on steep slopes.

* A 'clamp' node would be nice, it would let me remove min/max pairs I've added to do the same job.

* I've used a multiply (-1) and add (+1) pair to create an inverse function for data I've put into the (0,1) range. A node to create an 'inverse' intellegently using the maximum of the given map would be much clearer.

* A 'constant scalar' node would help to make self-documenting graphs. Sometimes you might want to make a key value quite prominent and this would help. e.g. naming the node "Tree Line Altitude".

* In a similar vein, having a 'random scalar' node would be great for Perlin maps to use a new seed each time. Inputs would be min and max values, and it'd have a dual purpose int/float output pin (if poss).

* It'd be nice to be able to access the ZeoScript function browser from the ZeoGraph window too, as a convenience when writing custom nodes.


Finally, I just wanted to say thanks for ZeoGraph. At first I was wondering if I'd ever really use it for anything practical - but I'm really glad you added it now! It's been fun working out how to make the result I want.

Here's a link to the graph I made in case some of my request need a bit of context: http://pastebin.com/m5184a3a7


Oh and I think I found a bug, if you attach the output of a 'Get water map' to a SetRange nodes. It gives the error (in 2.8.0.6): "calc_HF_ClampMap error:
- GetPixel returned false". The expected behaviour was to return a map of water altitudes for use like a height field.

[Aaron]

Hi David,

Thanks for the requests and the bug report. I'll include these changes in the next developmental build, and post back here when done.

Best regards,
Aaron.

[Aaron]

Hi David,

* A 'clamp' node would be nice, it would let me remove min/max pairs I've added to do the same job.

That's implemented in the 'Heightfield->SetRange' filter.

* I've used a multiply (-1) and add (+1) pair to create an inverse function for data I've put into the (0,1) range. A node to create an 'inverse' intellegently using the maximum of the given map would be much clearer.

I'll add an invert filter, but what you want can also be implemented using a 'SetRange' filter with the Max set to 0 and the Min set to 1 (if min>max, the map is inverted).

* It'd be nice to be able to access the ZeoScript function browser from the ZeoGraph window too, as a convenience when writing custom nodes.

Just now I've made the ZeoScript filter use the L3DT script editor window, so you'll have access to the function browser, file open/save, help links, etc. This will be included in the next developmental build.

Oh and I think I found a bug, if you attach the output of a 'Get water map' to a SetRange nodes. It gives the error (in 2.8.0.6): "calc_HF_ClampMap error:
- GetPixel returned false". The expected behaviour was to return a map of water altitudes for use like a height field.

The 'Heightfield.SetRange' filter only works on heightfields. To use it with the water map, you will have to explicitly create a heightfield from the water map. This can be done using the 'atConvertWaterToHeightfield' plugin, which is included with the default installation of L3DT. I'll add a ZeoGraph filter for this calculation.

Cheerio,
Aaron.

[Aaron]

Hi David,

As discussed above, I've now added a filter for converting the water map to a heightmap. The filter will appear as 'WaterMap.ConvertToHF'.

* Can you add 'specular map' support to the GetProjectMap node. I'd like to use it as an array of scalar values - in this case to raise the vegetation density slightly on the 'sunny' side of a terrain.

Will do. For the moment, select the 'Other' option in the map name list, and enter 'SpecLM' as the map name.

Cheerio,
Aaron.

[David Walters]

* A 'clamp' node would be nice, it would let me remove min/max pairs I've added to do the same job.

That's implemented in the 'Heightfield->SetRange' filter.

Oh, right, I thought that SetRange was a multiply/add not a chop. (Perhaps my request wasn't clear though, sorry about that).

Will do. For the moment, select the 'Other' option in the map name list, and enter 'SpecLM' as the map name.

Ah, okay thanks -- I tried other but didn't know the right code name for that map.

I'll add an invert filter, but what you want can also be implemented using a 'SetRange' filter with the Max set to 0 and the Min set to 1 (if min>max, the map is inverted).

Oh dear, I wish I'd tried that!

The 'Heightfield.SetRange' filter only works on heightfields. To use it with the water map, you will have to explicitly create a heightfield from the water map. This can be done using the 'atConvertWaterToHeightfield' plugin, which is included with the default installation of L3DT. I'll add a ZeoGraph filter for this calculation.

Great, thanks, I didn't realise there was a difference between the water heights and land heights. It's a bit confusing why this is, although I'm sure it's got a logical reasoning behind the scenes - and I'm not too familair with the complexities of water in L3DT either.

From a user's perspective though would it be possible to have automatic conversion be done? or perhaps having more specific hmap pins ( hmap<water> , or something)?

Cheerio,
Aaron.

Thanks for the in-depth response to all my little requests. This is great customer service and is much appreciated!

[Aaron]

Hi David,

Oh, right, I thought that SetRange was a multiply/add not a chop. (Perhaps my request wasn't clear though, sorry about that).

Oops, right you are. SetRange is a multiply/add, not a chop. I'll pop a 'chop' filter on the to-do list.

Great, thanks, I didn't realise there was a difference between the water heights and land heights. It's a bit confusing why this is, although I'm sure it's got a logical reasoning behind the scenes - and I'm not too familair with the complexities of water in L3DT either.

The pixel type of a water map is a struct consisting of a float for the water height, an unsigned short for the water body ID, and a byte for the water type enumeration. The extra data is used to optimise the flooding/water table/salinity routines, the attributes map calculation, and miscellaneous other features. In hindsight I probably could have split the water map type into a heightfield and one or more metadata layers, but that decision to pack it all into one map cannot be easily reversed.

From a user's perspective though would it be possible to have automatic conversion be done?

Potentially, but it would require a rewrite of all filters that take map arguments, and/or the creation of a shim to automatically convert GetPixel/SetPixel requests*. I'll think about it.

or perhaps having more specific hmap pins ( hmap<water> , or something)?

Variable handles don't specify a map type**, as this may change at runtime (e.g. the map handle of the output pin of the 'Project.GetMap' filter cannot be determined until the filter is rendered). I could change this with a new handle variable type that stores the referenced type, but all filters would need to be rewritten, and I'd have to add some extra conversion functions throughout L3DT and various plugins. This is probably my preferred option, as it allows for comprehensive build-time type checking. However, it's quite a bit of work, so I'll have to think carefully about it first.

On another topic, I've just added a set of filters for creating and editing variables (bool, byte, int, string, float, etc...), as per your request. This too will be included in the next build. Now for the random number generators...

Thank you also for the excellent feedback. It helps immensely to have another's considered opinion of one's design.

Cheers,
Aaron.

* I remember looking at this a few years ago. I recall there was some unpleasantness involved.

** nor even a variable type! You could pass in a handle to a boolean just as easily as a handle to a map (though the types are checked when used).

[David Walters]

The pixel type of a water map is a struct consisting of a float for the water height, an unsigned short for the water body ID, and a byte for the water type enumeration. The extra data is used to optimise the flooding/water table/salinity routines, the attributes map calculation, and miscellaneous other features. In hindsight I probably could have split the water map type into a heightfield and one or more metadata layers, but that decision to pack it all into one map cannot be easily reversed.

Nah, that's fine. As long as there's a cast operation it can be made to work with a bit of experimentation (i.e. seeing the filter will imply that it's probably worth a look). The thing that got me was not knowing about the extra data in the water table - in hindsight that explains why the 'getpixel' failed, thanks for clearing that up.

Variable handles don't specify a map type**, as this may change at runtime (e.g. the map handle of the output pin of the 'Project.GetMap' filter cannot be determined until the filter is rendered). I could change this with a new handle variable type that stores the referenced type, but all filters would need to be rewritten, and I'd have to add some extra conversion functions throughout L3DT and various plugins. This is probably my preferred option, as it allows for comprehensive build-time type checking. However, it's quite a bit of work, so I'll have to think carefully about it first.

I agree this sounds like the best direction, but it does sound like a big job! hehe.

On another topic, I've just added a set of filters for creating and editing variables (bool, byte, int, string, float, etc...), as per your request. This too will be included in the next build. Now for the random number generators...

Thanks, that'll help when sharing graphs (or returning to ones you made a while ago )

[Aaron]

Hi David,

I apologise for the delay in releasing the changes you requested. I'm currently working on a method to define filters using scripts, which should make the process of creating new filters much easier. In the current release, filters can only be created by plugins, and this is unnecessarily complex for most filter tasks. I'll ley you know when it's all ready for download.

Cheers,
Aaron.

[David Walters]

Hi aaron, thanks for the update. Don't worry about the delay though - for me this is just part of a hobby project and I tend to flick between various things anyway I'm looking forward to trying out the new build, but I'm more than happy to wait while you add some more neat stuff like this filter definition work!

[Aaron]

Hi David,

On the downloads page now you'll find L3DT Pro 2.8 build 8, which includes the following improvements to ZeoGraph:

• The 'Advanced.ZeoScript' plugin invokes L3DT's script editor window, allowing access to file load/save, function explorer, etc.
• When connecting two pins of different data types, ZeoGraph will automatically insert a casting filter to convert from one type to another (if conversion is possible).
• Filters may now be defined from text script files, rather than binary plugins, and examples are provided in the 'Filters' resource folder. No documentation yet, though.
• Added 'General.Select' filter, for forwarding one of several inputs depending on an integer switch input.
• Added some math filters ('Math.Modulo', 'Math.Power', 'Math.SquareRoot'), using script files.
• Added some string filters ('String.Concatenate', 'String.Length', 'String.Compare'), also from script files.
• Added some boolean logic filters ('Logic.Or', 'Logic.And', 'Logic.Not', 'Logic.Xor', 'Logic.Nor' and 'Logic.Nand').

Please let me know if you find any problems with the new build of ZeoGraph. I did make some drastic changes to the underlying mechanics, so whilst I didn't find any bugs in testing, I would not bet the farm that there aren't any.

I've still to do a number of your requests (e.g. random number generator), but with the above changes out of the way, they shouldn't take too long.

Cheers,
Aaron.

[Aaron]

Hi David,

I've just added a 'Math.RandomFloat' filter for generating random numbers. The filter is included with L3DT v2.8 build 10.

The gradient and chop filters are still on the to-do list. I'll post back here when they're available.

Cheerio,
Aaron.

[David Walters]

Hi Aaron,

I've been working on terrain again and so I've continued my experiments with ZeoGraph. First off thanks for the work you've done so far - build 11 is really strong and it's really becoming a very versatile and powerful system. I liked the auto-casting when I tried to connect a float random genertor to the Perlin seed - very cool! I have to admit though I've been a little scared of the advanced scripting nodes

If you're interested - a few more feature requests have emerged from my work today.

1. My original test graph used the salinity map to remove vegetation - when I tried to run it on a new project I got an error in GetProjectMap because as it turns out there was no salinity map present.

Unfortuantely I had to be quite destructive to my graph to remove this error, so can I make a general suggestion for GetProjectMap - that if a map does not exist it returns a zero version instead? I'd be okay with getting a warning if you feel that some user notification is required (I'm not sure it's necessary) -- but the benefit of being able to make a unified graph that works across a whole set of maps with varying properties would be very helpful.

As far as I can tell - all map types have a well defined 'zero' state (except perhaps the normal map? 'up' I suppose, or attributes? hmmm - maybe just error on that one!) - so I hope this isn't too hard to add.


2. For some reason I keep double-clicking empty space on the graph window to summon the 'add filter' window. It seems like maybe a nice UI sweetener to have this, so just throwing it out there.


3. I tried to use the specular map 'SpecLM' as you suggested but I didn't have much success (multiplying by 5 created a mostly blue RGB(0,0,128) map!). I think this must be because it's represented as an RGB map internally, and I would like to use it as a scalar value?

Can I suggest, as a generic filter, something like RGBMap > "DotProduct3" that would operate on the texture map, specular map, light map and normal map; take an additional triplet of floats and perform a DOT3 to result in a float based map / heightfield.

I can see this being very useful - not only for my specular map usage, but for the normal-map as you could then conceivably do some basic lighting calculations with it. For visual images (or, in future, coloured specular) then I'd use standard luminance ratios to get a grey-scale luminosity


That's all for now, keep up the good work!

[Aaron]

Hi David,

1) GetProjectMap error behaviour

I suspect having 'GetProjectMap' quietly fail and return an empty map would only defer the error; a subsequent filter in the graph will probably fail, and the cause of the failure would be less obvious to the user. For instance, 'GetProjectMap' won't know what type or size of empty map to create, and there's no way to guess the expected values, since 'GetProjectMap' knows nothing of the internals of the downstream filters (and nor should it). This is a problem because some of the downstream filters may require a particular map type or dimension, such as the 'CopyMap' filter, which requires the map types to the the same, and has certain map size requirements that are determined by the other input pins. Thus, even a 'safe' default value for the map type / size is will cause 'CopyMap' to fail most of the time.

My preferred approach would be to have 'GetProjectMap' throw a warning (continue/abort), and have it return a NULL map handle. Now, NULL map handles will still cause downstream filters to throw errors, but at least it will be a testable return value. Consequently, flow control filters (coming soon, I hope) will allow you to switch on and off different filter chains depending on whether the map handle is NULL or not.

Furthermore, getting at the root of the problem, I think we need some mechanism to easily enable/disable different sections of the graph without having to disconnect it all. I see no problem with adding a 'Disable filter' option. This would use similar mechanics as the flow control filters, which will also have to selectively disable/enable output peer filters.

Another option I have half-implemented is to wrap up entire graphs into custom 'macro filters', which can be used as filters in other graphs. This way, you could put all your salinity map processing into one macro filter, and then disconnect/disable that filter when the salinity map isn't initialised.

2) Double-click for add filter window

I'll try opening the 'add filter' window by double-clicking on the background. There might be a problem for the 'inexact clickers' amongst us, who might double-click to edit a filter/pin but miss by a pixel or two, and end up with the 'add filter' window when they expected to be editing values. Of course, I could fix that problem by making the hit-testing more intelligent. I'll have a play with the option to see how it pans out. Thanks for the suggestion.

3) Dot3 filter and working with RGB maps

I'll add a bunch of RGB image manipulation filters, including the Dot3 function as requested (probably dot4, so as to work with RGBA images too). I guess there should also be a bunch of map type converters (heightfield<->RGB, etc.) I'll put this on the to-do list.

I tried to use the specular map 'SpecLM' as you suggested but I didn't have much success (multiplying by 5 created a mostly blue RGB(0,0,128) map!).

Oops, that's a bug. The heightfield filters should have checked that the map input type was a heightfield, and rejected the RGB map. I have now modified the relevant functions accordingly.

Thanks again for your thoughtful feedback and suggestions. I'll post back here when the changes are ready for testing.

Cheerio,
Aaron.

Update: I've managed to bash together a 'gate' filter that conditionally renders/disables its downstream filters depending on its input. I've used this to create a demonstration graph that attempts to get a project map, if successful reports the map size, or if unsuccessful reports that the map is not initialised. I'll include the demo graph in the next build, but here's a preview:



It's not at all pretty, but I'll clean some of that up by replacing the two 'gate' filters, the 'not' filter and one of the 'Tee' filters with a single 'IfElse' filter (to be implemented). Stay tuned...

[Aaron]

Hi David,

In the latest dev build of L3DT Pro I've made the following changes to ZeoGraph:

• Double-click on background to open 'Add filter' window.
• Modified behaviour of 'GetProjectMap' to return NULL map handle when map is not present, and throw warning.
• Added the 'General.Gate' filter, which conditionally runs/suspends its downstream peer filters.

Regarding the 'General.Gate' filter, I've provided an example graph called 'GateTest.zgraph' (pictured in prev. post) that demonstrates how you might use the Gate filter to handle the return value of the GetProjectMap filter. The graph has two execution paths (using two gates) that will either echo to the event log the size of the map, or if the map is not initialised, report the error. If you don't care for the error reporting, you could easily cut this graph down by half.

Best regards,
Aaron.

[Aaron]

Hi David,

There's yet another update to ZeoGraph in the latest developmental build of L3DT Pro. This once includes a 'disable filter' menu option and toolbar button (hotkey = spacebar). Disabled filters, which are displayed as being greyed out, will not render, and will block rendering of all filters connected downstream.

To disable a section of a graph, I recommend you disable the first filter in the section, and then disconnect any output pins from that section to the rest of the graph (so as not to disable the whole graph).

Also, there's now a 'General.IfElse' filter, which will run one of its two output pins, depending on a boolean input. I've added an 'IfElseTest' graph file with the filter to demonstrate how this can be used to handle cases where 'GetProjectMap' returns a NULL map handle. It's much simpler than using 'General.Gate'.

Finally, I've alpha-sorted the filters in the 'add filter' window.

Cheerio,
Aaron.