Python plugin

[carldt]

Hi,

I have developed a plugin that will run Python scripts. The plugin has a builtin interface to the zeolite API so using the generated zeolite.py module you can write extensions in Python.

I have put this project here for now:

http://code.google.com/p/l3dtpython/

Hope this is useful to someone.

You can find the L3DT wiki page here:

http://www.bundysoft.com/wiki/doku.php?id=plugins:general:cdpython

In theory, now you can develop extensions in Python, which is free, so you don't need to know C++ and have Visual Studio...

Notes for DIY people

If you want to get the latest source code from SVN and build it for yourself with Visual Studio, I can recommend using tortoise SVN for windows, that is what I use: http://tortoisesvn.tigris.org/

You will also need SWIG to create the Python interface to the zeolite C++ code: http://www.swig.org

Why Python?

I want to use L3DT to create maps for TA Spring. I thought it would be cool to be able to automate the map creation process, run mapconv etc all using scripts that I and others could easily modify.

Python is a popular dynamic object-oriented programming language that can be learned really quickly.

Tips

If you have Python correctly installed, when you click "Run Python script" from the cdPython menu you get a open file dialog, if you right click on a python script you can choose Edit with IDLE, which will open that script in a IDE window that has syntax highlighting etc

[Aaron]

Hi Carl,

It looks fantastic. I'm just getting my head around SVN, python syntax, and the whole cdPython way of doing things, but hopefully I'll be able to contribute in a meaningful way soon.

I did notice when testing cdPython that the view_ShowMap Zeolite function is broken. I've fixed it for the next release. Also, the problem with the HF_to_BMP script might be in the way the format is retrieved. Calling map_GetFormat will only work if the map has already been saved. A safer bet might be to use format_GetByExt("HF", 0, "bmp").

Best regards,
Aaron.

[Aaron]

Hi Carl,

I'm trying to compile but I get a linking error:

Code: Select all

cdPython.obj : error LNK2019: unresolved external symbol _init_zeolite referenced in function _DllMain@12
Release/cdPython70.dll : fatal error LNK1120: 1 unresolved externals


Any ideas? How do I link to the SWIG-generated code, which presumably contains _init_zeolite?

Edit: D'oh! Sorry, I didn't read the above post properly. I'm off to download SWIG...

Cheers,
Aaron.

[carldt]

If you try to build using the current project file, all the include and library paths will be wrong for you, unless you happen to use the same drive and folder structure as me....

Essentially you need to know where Python.h and Python.lib are, and also adjust the custom build rule, to point to the swig.rule included in the cdPython source in the Tools folder.

The swig rule i created takes as its input a file like zeolite.i and outputs zeolite.i.cpp, so after the first time you run swig, you will need to add zeolite.i.cpp to the project so that it gets compiled into the DLL. Then you will have an extern reference to init_zeolite...

Hope that helps.

Note: I could commit zeolite.i.cpp to svn, but it always gets overwritten locally by swig. I should probably remove my paths from the project file, and instead use the global visual studio path settings. Edit: I have removed my paths from the proj file and added zeolite.i.cpp to svn

[Aaron]

Hi Carl,

Thanks for the help. I forgot to include the swig custom build rule.

Since I'm still using VC7.0, I had to build my own project file (7.0 can't load 8.0 projects). I've got it set up with the project file having a different name (cdPython70) and in another directory, but including the cdPython files. Hopefully this means I can checkout/in changes without messing up anyones project files.

Note: I could commit zeolite.i.cpp to svn, but it always gets overwritten locally by swig.

It probably won't be necessary unless I can't work out how to include SWIG rules in VC7.

Cheers,
Aaron.

[carldt]

Good idea to supply a build that uses the older C runtime.

It's unlikely that the code in cdPython will need to change much at all, if you look at the guts of it in cdPython.cpp, you will see it's rather simple.

The critical point for cdPython is that it is bound to the version of the zeolite API.

So, everytime zeolite changes, we must rebuild cdPython, so that SWIG can generate a new python wrapper to take into account changes to zeolite.

I think it would be more user friendly if for future releases of L3DT, you could rebuild cdPython against the correct version of zeolite, and then include that cdPython build in the L3DT installer.

Also you may notice I commented out the following in ExtAPI.h:

Code: Select all

//#ifdef _MSC_VER
//   #include "afx.h"
//#else
//   #include "windows.h"
//#endif


and in ExtAPI_defines.h I added :

Code: Select all

#include <windows.h>


This is because even though my DLL is not an MFC one, the _MSC_VER is still defined by the compiler and so would include afx.h, thus expecting my project to be an MFC one linking to afx.lib, which it does not.

[carldt]

If the Python25.dll links to a different version of the C runtime to cdPython.dll, then the console output will not work correctly. This is due to an incompatibility between versions of microsofts C runtime.

Basically this means you won't see errors in your python script in the console...

There's two solutions:

1) Get aaron or someone else to build cdPython with Visual Studio 2003
2) Get a Visual Studio 2005 build of Python

[Aaron]

Hi Carl,

I'm making progress, albeit very slowly. Now I get:

Code: Select all

Performing Custom Build Step
Unable to find file '[inputs]'.
Project : error PRJ0019: A tool returned an error code: "Performing Custom Build Step"


I've not worked out how to use the 'swig.rules' file as a custom build step for 'zeolite.i' in VS7.0 (I think this is what I have to do?), so I've entered the build command line instead:

Code: Select all

C:\Progra~1\SWIG\swigwin-1.3.31\swig.exe -c++ -python -Fmicrosoft -o [inputs].cpp [inputs]


However, it doesn't recognise [inputs], so I guess I have to specify a list of files or something. Anyway, it's half past twelve, so I'm going to bed.

Cheers,
Aaron.

[carldt]

Hi Aaron,

I guess the custom build rules work differently between vs2003 and vs2005.

If you open up the swig.rule file, it is XML, you will see the command line. In vs2005, [inputs] is substituted with the name of the *.i file that the rule applies to. So a swig cmd line like:

Code: Select all

swig.exe -c++ -python -Fmicrosoft -v -o [inputs].cpp [inputs]


Becomes:

Code: Select all

swig.exe -c++ -python -Fmicrosoft -v -o zeolite.i.cpp zeolite.i


The reason I created the swig.rules custom build step is so this step is automated for any SWIG interfaces (*.i). I guess in your case the zeolite.i can be hard coded, however I assume the vs2003 uses something similar to [inputs] possible something like $(InputPath), so you should be able to get a custom build rule for SWIG interfaces working under vs2003. I guess you'll have to dig around in MSDN.

Hope that helps.

[carldt]

Wow the CRT incompatibility problem is really chronic... plugin developers who aren't using the same tools as Aaron (VS2003) beware!

http://blog.kowalczyk.info/archives/2006/08/07/the-missing-msvcr80dll-story/

http://forums.microsoft.com/MSDN/ShowPost.aspx?PostID=72337&SiteID=1

http://www.grimes.demon.co.uk/workshops/fusWSThirteen.htm

http://msdn2.microsoft.com/en-us/library/ms235285.aspx

http://support.microsoft.com/kb/326922

I remember GNU/Linux had a libc incompatibilty problem for a while way back when, but after the transition it's been a non-issue. Microsoft's CRT incompatibility seems to be an ongoing problem...

So another good thing about developing extension scripts using cdPython (at least cdPython linked to the same CRT as Python), you don't have to care about CRT compatibility issues...

Note: I have built Python 25 using VS2005, and put the Python25.dll that links to msvcrt80.dll on the google project downloads page

[carldt]

I have found the Visual C++ 2003 Toolkit compiler here:

http://xona.com/2004/06/29.html

I will do the necessary things to get the cdPython code building with this compiler.

Need to download .Net 1.1 as well to get msvcr71.lib

And don't forget:

For advanced users, here is a tip to generate the correct lib file matching the msvcp71.dll included in the distribution :



The command “dumpbin /exports /OUT:msvcprt.def msvcp71.dll” generates the file msvcprt.def. This file is not usable as is, please take a look at the provided mscvprt.def file to see how it should look like.

Then the command “lib /def:msvcprt.def” generate a usable msvcprt.lib.

http://root.cern.ch/root/Procedure/Procedure%20to%20install%20the%20free%20Microsoft%20Visual%20C.htm

[carldt]

I have built cdPython with the 2003 Toolkit compiler, so it will now work properly out of the box with the Python 2.5 Windows installer.

Here's the steps:

1) Get Python here:

http://www.python.org/ftp/python/2.5/python-2.5.msi

2) Then get cdPython here:

http://l3dtpython.googlecode.com/files/cdPython-0.10.zip

3) Go to D:\Install\Bundysoft\L3DT Standard 2.5 RC3 (or wherever L3DT.exe is installed) extract the cdPython.zip to here.

4) Get a script to try out, like HFtoBMP:

http://l3dtpython.googlecode.com/files/HFtoBMP.py

5) Create a Terrain.

6) Start L3DT, go to cdPython->Run a Python script, browse to HFtoBMP.py, press OK, enter a filename in the text box (without the BMP extension), press OK

6) Go to the folder where the HFtoBMP.py script is and there should be a BMP file with the filename you gave.

[Aaron]

Bravo! You beat me to it by fifteen minutes. I just got cdPython compiling in MSVC7.0, and now the tinkering begins...

Edit:

I've uploaded 'v0.1b', which statically links to the runtimes (maybe avoid DLL hell?), and also has some test options like a menu item for an editor, wiki link, etc. If it's of any interest, I can probably work out how to commit changes to SVN (eventually...)

Cheers,
Aaron.

[carldt]

I'm not sure that statically linking is the solution, because regardless of how the crt is linked, the versions could still be different and cause strange problems...

Regarding an editor, Python comes with IDLE, see the tip far above:

If you have Python correctly installed, when you click "Run Python script" from the cdPython menu you get a open file dialog, if you right click on a python script you can choose Edit with IDLE, which will open that script in a IDE window that has syntax highlighting etc

A wiki link would be useful, and also a link to the API! Because it is the API documentation that will be REALLY come in useful to Python script authors...

[DeathTwister]

Haha funny I was getting ready to reinstall this all back as well and this could help me very much in the long hall, I will keep this one on the watch list as we go, Wow thanks allot /smiles
DT