Python plugin

[carldt]

I think we should be able to write a TGEA or other format export in Python. Perhaps some open source code to do this already exists out there, and could be adapted to work from L3DT.

As for integrating L3DT and Blender to work together through Python, this could be possible with some kind of inter-app communication - keep in mind both cdPython and Blender will have their own Python interpreters, so effectively they would be running two separate Python programs. I'm guessing here, but I think you would have to write two Python scripts and get them to communicate through pipes or sockets or something, you could probably serialize and pass objects between two Python programs using the "pickle" facility. But I'm not sure there's any point to what I have just suggested, because L3DT generates static data, and the exercise would ultimately deliver little benefit.

Exporting to a file that Blender can use is a great subject for a Python script. If someone can provide file formats, then let's see them and have a go!

[Aaron]

Hi Carl,

The TGEA Atlas terrain format is proprietary and can only be encoded/decoded by code provided as part of TGEA by GarageGames. They are currently debugging a binary plugin for L3DT to write that format, which was built using the Zeolite API and the TGEA code. The plugin works pretty well, but has a few bugs in need of squishing before release.

Regarding using sockets/etc to integrate L3DT with Blender; I agree, the benefits aren't entirely obvious. I can however see that the difficulties in implementing, testing and on-going support would be considerable. I suppose I do have something sort of like this on the cards insofar as I'll be implementing master/slave network calculation plugin at some time down the track (to allow L3DT to run on render farms), but that's just between L3DT and itself, not a different app. Compatibility, testing and support are therefore much simpler. Anyway, I think users would need to mount a stronger case for cross-application session integration for it to be considered seriously.

Finally, on the subject of Blender formats, I've not done anything here (yet) in part because there format spec isn't published anywhere I could find, and also because there is a TER2Blend python script for Blender that coverts Terragen TER files to BLEND files. TER is natively supported by L3DT, hence there already appears to be an export route. That could be made neater by an exporter plugin for L3DT, but in my assessment it's not such a critical gap that I need to stop what I'm doing and write yet another format plugin.

Cheers,
Aaron.

[DeathTwister]

Hay guys,

Getting back to you all, I very busy atm like the rest of us /smiles....

I apologise for snapping at you there. I did some more reading, and I see that Blender has a python API too, so your question was not as far off topic as I initially thought.

Hehe dat OK Boss, and thanks for the apology my brother, excepted, /bows, sorry I didn't make what I was trying to say clearer for you, I do that sometimes

In answer to your supplementary question; this plugin allows users to run python scripts in L3DT (python is a fancy and newish programming/scripting language). These scripts can do simple things like load/save files, more complex things like read/apply settings, fancy things like custom calculations, and probably much fancier things still. The possibilities are indeed very broad.

Ah Ha, so it looks like it can go both ways once everyone gets used to it?, very interesting. Way above my little head, but I am kinda getting to understand it now, thanks. So to kinda answer my own questain then, if I hang in there a bit, someone will probably make it for us/me. /winks.... I will follow this thread, but keep mouth closed more on this one until I understand it more.

The TGEA Atlas terrain format is proprietary and can only be encoded/decoded by code provided as part of TGEA by GarageGames. They are currently debugging a binary plugin for L3DT to write that format, which was built using the Zeolite API and the TGEA code. The plugin works pretty well, but has a few bugs in need of squishing before release.

Cool, but I think I will wait until they finish with it and it has been squished well and many times, as I have been so frustrated with that end of stuff lately, the Work flow is killing me, meaning I was at a standstill for a while, err still am with TGEA, but making progress now with Ogre in many ways.

Finally, on the subject of Blender formats, I've not done anything here (yet) in part because there format spec isn't published anywhere I could find, and also because there is a TER2Blend python script for Blender that coverts Terragen TER files to BLEND files. TER is natively supported by L3DT, hence there already appears to be an export route. That could be made neater by an exporter plugin for L3DT, but in my assessment it's not such a critical gap that I need to stop what I'm doing and write yet another format plugin.

Cool Cool Aaron, WTG my Man, I needed that link, thankies very much, yes yes that helps me with the terrain export/import problems I have been having getting them into Ogre easy, now I have a pipline workflow /dance a jig with happyness.

I have like 50-60 VR worlds ready to go, and am stuck with workflow troubles /hangs head in sadness, but looks like the dawn is near.

KK I am done, hope all was not to far off-topic, and I keep quiet now /winks.
DT

[carldt]

It's interesting to read DeathTwister's comments about workflow and getting stuck here and there when it comes to getting from one format to another. These are the kind of problems cdPython could help solve, however it still requires someone to write Python programs to do these export/import tasks... So I guess I should have better explained that cdPython doesn't actually do anything on it's own... cdPython is really just a tool for people who want to develop their own plugins, and who don't know C or have a C compiler.

It was observant of DT to make the connection between cdPython and Blender's Python support. There's potential to get some Python code reuse.

The TGEA Atlas terrain format is proprietary and can only be encoded/decoded by code provided as part of TGEA by GarageGames. They are currently debugging a binary plugin for L3DT to write that format, which was built using the Zeolite API and the TGEA code. The plugin works pretty well, but has a few bugs in need of squishing before release.

Ugh. I'm no fan of proprietary, restricted formats. But hey, with L3DT I have options other than TGEA.

Finally, on the subject of Blender formats, I've not done anything here (yet) in part because there format spec isn't published anywhere I could find, and also because there is a TER2Blend python script for Blender that coverts Terragen TER files to BLEND files. TER is natively supported by L3DT, hence there already appears to be an export route. That could be made neater by an exporter plugin for L3DT, but in my assessment it's not such a critical gap that I need to stop what I'm doing and write yet another format plugin.

Thanks for that link, it's probably trivial to get TER2Blend working from cdPython. In fact, from Python we can launch other applications, so possibly could fire up Blender after the export finishes...

[demi]

Hi carldt,

I don't know if you would want to use TER2blend myself. I found that using directx export from L3DT and importing it in Blender gives a better results because TER2blend does not export in triangles (polys), of course that depends on what you are attempting to do.

my 2 copper,

Demi

[Aaron]

Hmmm...this is a reminder for me to make the X export binary (currently it's ASCII, and huge).

Edit: If I'm fixing-up X, I may as well write 3DS too.

[3DUser]

Demi,
I was unable to get the Direct X file from L3DT to import into Blender's 2.43 version. A check on the forum indicated others had problems with the import as well. It may just be a version change. Aaron, I hope the 3DS or .obj exporter work goes well.

3DUser

[DeathTwister]

Hay der,

Hummm,

Hmmm...this is a reminder for me to make the X export binary (currently it's ASCII, and huge).
Edit: If I'm fixing-up X, I may as well write 3DS too.

Yes, yes , please, please /smiles....

I was unable to get the Direct X file from L3DT to import into Blender's 2.43 version. A check on the forum indicated others had problems with the import as well. It may just be a version change. Aaron, I hope the 3DS or .obj exporter work goes well.

Oh, I really could use .3ds and .obj exporter, yum yum.

DT

[demi]

Demi,
I was unable to get the Direct X file from L3DT to import into Blender's 2.43 version. A check on the forum indicated others had problems with the import as well. It may just be a version change. Aaron, I hope the 3DS or .obj exporter work goes well.

3DUser

Yeah, 2.43 is a problem. I reverted to 2.37 when I was using directx for my land masses. Now I use Multiverse which takes a PNG directly from L3DT. Current world is 64K x 32K pixels.

Demi

[Aaron]

Hi Carl,

I've updated the API slightly (v2.3.1). One addition that may be of interest is the 'script_Execute' function:

Code: Select all

script_Execute(const char *lpLanguage, const char* lpScriptStr, ZVAR hRval)


This function allows plugin programmers to run Python or ZeoScript scripts through the API. If the lpLanguage parameter is "py", the script is passed through to the RunScript function of cdPython, and if it's "zs" it goes through to ZeoScript. Note that the last parameter is a handle to a variable that takes the return value of the script. The idea here is that developers will be able to use scripts to do things more efficiently than they could with the C++ API. This is probably not the most elaborate example, but the following code calls the 'file.SaveDlg' extension function to open a file-saving dialog and get the filename returned:

Code: Select all

CzStr FileName;
theAPI.script_Execute("zs", "file.SaveDlg bmp NULL \"Bitmap image (*.bmp)|*.bmp|\"", FileName.GetZVAR());


This approach is used 'for real' in the Sapphire 3D renderer, where the heightfield editing brush is run via ZeoScript calls to the 'calc.HF.GenPeakMap' and 'calc.HF.ApplyOverlay' extension functions. Using 'script_Execute', it takes about three lines of code to do the heightfield editing, whereas using CzFuncs took about twenty.

Anyway, can you think of a way to get a script return value from cdPython?

Oh, and another thought: what would you say to auto-inserting "import zeolite" to the start of python scripts run through the cdPython::RunScript function, so that users don't have to type it each time? I'm guessing that we can probably assume that python scripts run through cdPython will always use the zeolite module.

Cheers,
Aaron.

[carldt]

Anyway, can you think of a way to get a script return value from cdPython?

Scripts are like programs, I believe that the return value is an integer, where a non-zero integer indicates a failure. I think you what you want is to be able to call a Python method from C code, and pass the return value from the Python method back to C code. I think the only useful return type is a ZVAR; I guess I can add this facility, I don't feel really inspired right now can you give me a funky example to work with?

Can you create a zeolite named variable in Python, and then attach to that named variable in C? That would be the work around in the meantime.

Oh, and another thought: what would you say to auto-inserting "import zeolite" to the start of python scripts run through the cdPython::RunScript function, so that users don't have to type it each time? I'm guessing that we can probably assume that python scripts run through cdPython will always use the zeolite module.

Auto-inserting "import zeolite" would be a pain because a file handle is passed to the Python interpreter, so we'd have to create a temporary file that was a copy of the chosen Python file, tack import zeolite at the top, then pass this temporary file to the Python interpreter. For the sake of typing 15 or so characters I think this is a complete waste of effort. Don't take it personally but I'm going to disregard this suggestion, it seems way too trivial to me, especially when comparing the effort of the change to the benefit. I figure most people would take an existing script and copy/paste/modify it how they want anyway.

[Aaron]

Hi Carl,

I agree that including 'import zeolite' is hardly a huge inconvenience for users. However, I don't agree that appending 'import zeolite' to the start of the script is quite as difficult as you make out. A duplicate file is unnecessary because the script can be read from the file, have the 'import zeolite' added to the beginning, and then run through python (see the RunScript function in cdPython for running scripts from strings in memory). In any case, it's just an idea, and I don't mind at all if it's disregarded.

I think you what you want is to be able to call a Python method from C code, and pass the return value from the Python method back to C code.

Yep. That's what the above example does; it calls the 'file.SaveDlg' zeofunc through a zs script and puts the return value into a CzStr.

I think the only useful return type is a ZVAR;

Agreed, since ZVARS wrap other data types (int, float, etc).

I guess I can add this facility, I don't feel really inspired right now can you give me a funky example to work with?

Sorry, I don't have anything funky right now. This is really just some infrastructure planning on my part: allowing developers to call py/zs scripts from C as if they were any other function call seemed like it might be useful. I have implemented the return value handling for scripts run through ZeoScript, and I do find that useful. However, it's too soon to say whether other developers will also find the script_Execute function useful, so I guess this can go to the backburner until there is a need.

Cheers,
Aaron.

[carldt]

A duplicate file is unnecessary because the script can be read from the file, have the 'import zeolite' added to the beginning, and then run through python (see the RunScript function in cdPython for running scripts from strings in memory).

PyRun_SimpleString takes a char *. This means having to either realloc the script string that is passed to RunScript, or allocate a new one before passing the new string to PyRun_SimpleString. I prefer to let the caller of RunScript do memory allocation because the caller knows how the memory should be allocated, and how much they can allocate.

The other point where I become nervous is reading a script from file into memory. How big will the file be? Will it fit into RAM? PyRun_SimpleFile takes a file descriptor so it can load and parse chunks of the file, this way it doesn't have to load the entire file into memory.

We can probably assume that most Python scripts will be fairly small, however we can't safely provide a maximum script size limit. Ultimately, the benefit is so tiny, even if this is a quick hack, I wouldn't want to support this code as part of any long term infrastructure. I'm more open to the idea if RunScript takes a function pointer to a memory allocation (and probably a pointer to a free function too) so that cdPython can wash its hands of memory allocation responsibility.

I think you what you want is to be able to call a Python method from C code, and pass the return value from the Python method back to C code.

Yep. That's what the above example does; it calls the 'file.SaveDlg' zeofunc through a zs script and puts the return value into a CzStr.

I think the only useful return type is a ZVAR;

Agreed, since ZVARS wrap other data types (int, float, etc).

Looks like we should define a zeolite friendly interface for calling script methods like:

Code: Select all

   ZVAR  myCallableMethod (ZLIST params);


This is really just some infrastructure planning on my part: allowing developers to call py/zs scripts from C as if they were any other function call seemed like it might be useful. I have implemented the return value handling for scripts run through ZeoScript, and I do find that useful. However, it's too soon to say whether other developers will also find the script_Execute function useful, so I guess this can go to the backburner until there is a need.

In reality I think mixed language extensions will be difficult to maintain and use. Since cdPython is not a core module, there will configuration issues as well. It's just my opinion this morning though... I reserve the right to change my mind

[Aaron]

Hi Carl,

Putting aside the cudgels for a moment; I have been making progress on your request for custom map layers. The project class and the file I/O code now support an arbitrary number of maps in the project. The challenge now is to upgrade the (rather scary) display class to handle arbitrary maps...oh, and add a brush tool for your masks. Anyway, I'll let you know when I've got something ready to test.

Cheers,
Aaron.

[carldt]

Thanks for the custom map layers, that's great. L3DT will now be more than a terrain generator, it will be a true "level editor" for games with this feature.

Thanks again.

Carl