Thursday, August 21, 2008

scipy.weave doesn't work - compiler troubles

The scientific python package SciPy contains a very powerful library called weave. Weave enables one to relatively easily embed C/C++ code into Python code. Thus, if you happen to have a computationally intensive code you can gain a lot of speed by rewriting it in C/C++. There are a lot of ways of how to accomplish this but weave makes this feat relatively easy even for an amateur like me.
There is however one caveate for Windows users. I commonly install all software into the 'C:\Program files\' directory. If you do this with Python and a weave requirement, MinGW compiler, weave will not work. The trouble is the space within directory name. Weave scripts and possibly also gcc are not written to deal with spaces within paths and will not find required files like gcc.exe. You will end up with error messages like:
error: command 'gcc' failed: No such file or directory
ValueError: The 'mingw32' compiler was not found.
At first I have made a hack in the first affected script, the 'platform_info.py', but this only resulted in another error:
Could not locate executable g++
Executable g++ does not exist
Traceback (most recent call last):
File "G:\Temp\tmpfgm8bn\_ev-uq.py", line 8, in
s,o = exec_command(cmd, _with_python=0, **{})
File "C:\Program Files\Python25\lib\site-packages\numpy\distutils\exec_command.py", line 253, in exec_command
use_tee=use_tee,**env)
File "C:\Program Files\Python25\lib\site-packages\numpy\distutils\exec_command.py", line 402, in _exec_command
so_dup = os.dup(so_fileno)
OSError: [Errno 9] Bad file descriptor
So, to make things short, just install Python and MinGW into C:\. This will save you from a lot of trouble.
UPDATE 08/10/22: It turned out today that your user name also may not have a space within it. I've just spent an hour trying to get weave working on a colleague's computer where the username is actually a "user name". No, renaming the user does not do the trick. I 'solved' it by creating a new user with 'valid' username.

2 comments:

Thorney said...

I had a similar issue on windows.
I found the error was actually in distutils not in weave tho. Bug report at python.org: Issue 4508

I still think the inline C/C++ code in weave is pretty sweet! Or swig for the heavy duty stuff.

R.L. said...

Thank you, Thorney, for posting this up as a bug. I'm not bold enough to do this nor am I skilled enough to supply a patch. Hopefully this issue will get fixed some time.
I really liked the simplicity of inline C/C++ but somehow the Pyrex/Cython option suits my mind better. It is also easier to distribute the result (no need to install mingw everywhere). It is of course more cumbersome and the development is definitely slower.