From NESTFE Wiki

Contents 1 Installation 1.1 Win32 1.2 Cygwin 1.3 Debian Linux 3.1 2 Post-install 2.1 all platforms 2.2 win32 and cygwin only 3 Testing the installation 3.1 Python 3.2 Jpype 3.3 Guis 4 Notes 4.1 installing JPype for win32 from CVS 4.2 Undoing a JPype CVS install 4.3 Math in python 4.4 Matlab to Python Interface (experimental)

Installation

To use pytos, you must install the python environment using one of the three sets of instructions below: windows, cygwin, or linux. Then, for all platforms, follow the post-install directions.

Notes:

1. If you want a basic setup, you only need python2.4 and jpype.
2. If you want a full setup (required for python GUIs), you will also need Tkinter, numeric, and matplotlib
3. If you are installing on a windows machine, we recommend the full win32 installation and/or the basic cygwin setup, since the cygwin libraries may have GUI problems.

Win32

• Python: Use python-2.4.1 win32 installer [here]
• JPype: Download and use the windows instaler and Copy msvcp71.dll to your python root directory (C:\python24 by default)
• Numpy: Get Win32 installer (Version 23.8 for python 2.4)
• Matplotlib: Use the win32 installer (Version 0.83.1 for Python 2.4)

Cygwin

BASIC INSTALL

• Use cygwin package manager to get python-2.4

• Jpype
  • Get/checkout the CVS version
  • In setup.py, make "cygwin" a recognized sys.platform
    • JAVA_HOME = "/path/to/jdk"; JDK_INCLUDE = "win32"; LIBRARIES = []; MACROS = [ ("WIN32",1) ]
  • Some python header want sys/select.h to exist (it doesn't otherwise use it), so do this in the JPype-0.5 directory:
    • mkdir src/native/common/include/sys && touch src/native/common/include/sys/select.h
  • To compile: python setup.py build --compiler=mingw32
    • tips (http://dkc.jhu.edu/gamera/html/install.html#cygwin-mingw32)
  • To install: python setup.py install
  • Note: The above instructions used to work, but recently the Cygwin install of pytos seems to segfault when the "startJVM" command is run and it is using sun's j2sdk1.4.2_11. If you get it to work (perhaps with another version of java), please update here (or let kamin know).
  • Todo: Somebody needs to debug the jpype segfault in cygwin with sun's j2sdk by turning on tracing before compilation, which should print out trace of jpype calls so we can see where the segfault is happening, ie.: uncomment line 23 in file src/native/common/include/jpype.h and do the following :
    • python setup.py build -f
    • python setup.py install -f

OPTIONAL

• numpy (aka numeric)
  • Get the tarball (version 24.0b2) and extract it.
  • install the Numeric tarball: python setup.py install

• matplotlib (Status: installs, but GUI seems problemic)
  • get the following cygwin packages (all related & devel versions): python, freetype, libpng, zlib
  • get matplotlib-0.83.1
  • add some library links
    • ln -s /lib/libtk84.a /lib/libtk8.4.a
    • ln -s /lib/libtcl84.a /lib/libtcl8.4.a
  • edit the setupext.py file:
    • add 'cygwin' to basedir dict by adding this line: 'cygwin' : ['/usr/local', '/usr',],
  • python setup.py build
  • python setup.py install

Debian Linux 3.1

Download and unzip the following file to a web server directory, and make the resulting directories/files world readable: http://www.cs.berkeley.edu/~kamin/pytosPackages.tgz

Then, add the following lines to your /etc/apt/sources.list file:

deb http://www.yourservernamehere.edu/ pytosPackages/

Install the following packages (and their dependencies) using synaptic or apt-get:

python2.4
python2.4-tk
python2.4-dev
python2.4-numeric
python2.4-numeric-ext
python2.4-matplotlib
libxml-simple-perl
(optionally, for java)
j2re1.4
j2sdk1.4
java-common

(alternatively, of course, you could have just used the .deb files in the .tgz file and installed them with dpkg. If somebody ends up putting these packages on a long-lived web server, please update the link above to allow others to use it)

Make sure that "which java" returns the executable in /usr/lib/j2se/1.4/bin (or whichever tree your libjvm.so file is), not the symbolic link in /usr/bin. Now, download jpype from cvs into /usr/local/src and install:

 sudo rm /usr/bin/python 
 sudo ln -s /usr/bin/python2.4 /usr/bin/python (to install to python2.4 instead of python2.3)
 cd /usr/local/src
 cvs -d:pserver:anonymous@cvs.sourceforge.net:/cvsroot/jpype login
 cvs -z3 -d:pserver:anonymous@cvs.sourceforge.net:/cvsroot/jpype co -P jpype
 cd jpype
 edited ./setup.py so that line 26 of jpype/setup.py reads "JAVA_HOME = '/usr/lib/j2se/1.4'"
 python setup.py build
 sudo python setup.py install
 sudo find /usr/lib/python2.4/site-packages -type d | grep jpype | xargs -i chmod a+rx {}
 sudo find /usr/lib/python2.4/site-packages -type f | grep jpype | xargs -i chmod a+r {}

Post-install

all platforms

• verify that your tree is up to date (or if not working from CVS, that you have tinyos 1.1.15 or later), and recompile all java tools.
• verify that you have perl installed, along with the XML::Simple module. If you don't from the command line run:

cpan -i XML::Simple

• verify that you are using nesc version 1.2 or later. It is best to download and compile the cvs version.
• create a ~/.matplotlibrc file with the following line (to use Tk instead of GTK as the default rendering engine):

 backend      : TkAgg    # the default backend

• Your ~/.profile must do the following:

Set TOSROOT (eg, export TOSROOT=$HOME/tinyos-1.x)
export MAKERULES=$TOSROOT/tools/make/Makerules
export CLASSPATH="`$TOSROOT/tools/java/javapath`" 
The last line of your .profile should source the pytos profile (eg, source $TOSROOT/tools/python/util/pytosProfile)

• If you are not receiving any Drain packets, make sure that your TOSBase is programmed with the correct ID and that your tools know what the ID is. In general, this should be 0xfffe. if you are using contrib/nestfe/apps/TOSBase, you must install your TOSBase with the correct address "make install.0xfffe". With tinyos-1.x/apps/TOSBase, this is not necessary. you can comm unicate the address to your pc-side tools in two ways. You can export MOTEID=65534 in your .bashrc file or you can edit the tools/python/pytos/util/RoutingMessages.py file (line 158).

win32 and cygwin only

• set PYTHONROOTWIN to your win32 python directory using win32 path form (eg, export PYTHONROOTWIN="C:\\python24" )
• verify that PATH includes $JDKROOT/bin (where,eg, JDKROOT is something like /cygdrive/c/apps/java/j2sdk1.4.2_08)
• verify that CLASSPATH is entirely in win32 form
• verify that PYTHONPATH is empty or entirely in unix form
• verify that PATH is entirely in unix form
• verify that cygpath.exe is in your PATH
• To start a win32 python session that will work with pytos, first call windowsPythonEnvironment.sh from a cygwin shell. This will spawn a new DOS prompt with it's environment setup to support pytos. To start python, type python from this DOS prompt. To start a python app, type the name of the app

Testing the installation

Python

Try running

$ python

from a command prompt (either a bash, cygwin, or dos prompt (using pytosCmdPrompt.sh)), depending on which installation you did). You should get a python command shell. If that works, try importing pytos (if you have trouble with this step on win32, you may need to put, for example, export JAVA_HOME=c:\tinyos\j2sdk1.4.2_08 in your .profile and call pytosCmdPrompt.sh again from cygwin):

>>> import pytos
>>>

If you can do that, python, jpype and all of your environment variables are setup correctly. Now, try importing a pytos class:

>>> import pytos.Comm

If you get a "no class found" type of error, then tools/python is not in your PYTHONPATH (if you get another kind of error, it might be a jpype problem; see below). To check what is in your PYTHONPATH, from the python prompt run

>>> import sys
>>> print sys.path

If tools/python is not in your sys.path, you can either add it from within python or add it to your PYTHONPATH environment variable before starting python.

If these don't work, check these troubleshooting tips:

• Make sure you do not have the environment variable $JAVA_HOME defined (if it is in cygwin path format, it will confuse one of the __init.py__() functions )
• If you have weird path issues, you might want to remove all paths with spaces from your PATH variable. For example, if you have 'Program Files' in your paths, you could use something like (the example doesn't match the last path in $PATH, but whatever):

   export PATH=`echo $PATH | sed 's/\/cygdrive\/c\/Program\ Files[^:]*\://g' -`

Jpype

From a python command prompt, run

>>> import jpype

If that does not give any errors, you have jpype installed. Now, run

>>> jpype.startJVM("path/to/my/javaLib")

where javaLib is either "jvm.dll" in Windows or "libjvm.so" in Linux. For both windows and cygwin python, the path should be a Windows path (ie "C:\...", not "/usr/..."), otherwise you could get a segfault.

If that works but you still get a segfault when importing something like NescApp, check the pytos/__init__.py file to make sure it is passing the right location of your javaLibrary.

If that works, try importing a java object:

>>> from jpype import jimport
>>> i = jimport.java.lang.Integer(100)
>>> i.intValue()
100

If that doesn't work, you probably don't have your CLASSPATH setup correctly. Make sure that tools/python/pytos is in your CLASSPATH, as well as the tinyos java tools. In both windows and cygwin python, you need to have your CLASSPATH in windows path format (ie "C:\...", not "/usr/...". Make sure paths are separated by semicolons, not colons.)

Guis

To verfiy that matplotlib is working alone, try plotting a straight line:

>>>from pylab import *
>>>plot([1,2,3,4])
>>>show()

To verify that Tkinter, Numberic, and Matplotlib are working together, try running Oscope.py

cd ~/tinyos-1.x/apps/Oscope
make telosb install 
Oscope.py telosb serial@COM1:telos

You could also try running the RFS demo GUI:

cd ~/tinyos/contrib/nestfe/nesc/apps/TestDetectionEvent
make telosb install fileset,ignore
./RfsFieldGui.py telosb serial@COM1:telos

If you are using cygwin and this doesn't work, check these troubleshooting tips:

• if you get the "fork_parent and child died waiting" error
  • get the rebaseall package from cygwin
  • close _all_ cygwin apps (shells, rxvt, etc)
  • open a bash shell only (no rxvt or xterm)
  • run rebaseall
  • Rebaseall may mess up emacs -nw (in Cygwin 1.5.17-1)
    • See this hack for how someone fixed it.
• you may need some X11 headers from cygwin too

Notes

installing JPype for win32 from CVS

• you don't need these, but in case you want to build jpype from CVS, the instructions are here
• Get the CVS version
• Follow the instructions and my modifications (below) to build msvcprt.lib
  • put dumpbin.exe in your path by adding the following to the windows PATH: C:\Program Files\Microsoft Visual Studio\VC98\Bin;C:\Program Files\Microsoft Visual Studio\VB98
  • from a cmd prompt, do
    • cd C:\Program Files\Microsoft Visual C++ Toolkit 2003\bin
    • wget http://libsh.org/win32/data/msvcprt/exports.sed
    • echo LIBRARY msvcp71.dll > msvcprt.def
    • echo EXPORTS >> msvcprt.def
    • dumpbin /exports msvcp71.dll | sed -nf exports.sed >> msvcprt.def
    • lib /def:msvcprt.def /machine:x86
    • cp msvcprt.lib ..\lib\
• Follow the instructions and my modifications(below) to build jpype
  • Patching msvccompiler.py
    • I patched it by hand because I am running python 2.4.1 (not 2.4 final)
    • I used a different reg-key: freeSDK = r"SOFTWARE\Microsoft\MicrosoftSDK\InstalledSDKs\8F9E5EF3-A9A5-491B-A889-C58EFFECE8B3"
  • In the vc7.bat file
    • I needed to add "Platform" to the SDK directories
    • For jpype, I needed to add "SET JAVA_HOME=C:\apps\java\j2sdk1.4.2_07"
    • my vc7.bat file looks like this
      • Set PATH=C:\Program Files\Microsoft Visual C++ Toolkit 2003\bin;C:\Program Files\Microsoft Platform SDK\Bin;C:\apps\Python24;%PATH%
      • Set INCLUDE=C:\Program Files\Microsoft Visual C++ Toolkit 2003\include;C:\Program Files\Microsoft Platform SDK\Include;C:\apps\Python24\include;%INCLUDE%
      • Set LIB=C:\Program Files\Microsoft Visual Studio .NET 2003\Vc7\lib;C:\Program Files\Microsoft Visual C++ Toolkit 2003\lib;C:\Program Files\Microsoft Platform SDK\Lib;C:\apps\Python24\libs;%LIB%
      • SET JAVA_HOME=C:\apps\java\j2sdk1.4.2_07
  • building it
    • from a cmd, do
    • "C:\Program Files\Microsoft Visual C++ Toolkit 2003\bin\vc7.bat"
    • python setup.py bdist_wininst -d C:\home

Undoing a JPype CVS install

• To undo the actions of 'python setup.py build' (perhaps so that you can run it again with a modified setup.py)
  • remove both the lib.<plat> directory and the temp.<plat> directory
  • running 'python setup.py clean' only remove the temp.<plat> directory
• To undo the actions of 'python setup.py install', remove the files that were copied into your 'site-packages' directory
  • This might be for instance 'c:/ProgramFiles/Python24/Lib/site-packages' or '/usr/lib/python2.4/site-packages'
  • 'python setup.py install' basically copies files from build/lib.<plat> to their appropriate places
• for more info, look at: installing Python modules

Math in python

• Numarray is a replacement (and competitor) for Numeric; it has a few missing pieces,drawbacks compared to numeric
• Numeric (aka Numerical Python, aka Numerical Python Extensions, aka NumPy) is being replaced by SciPy, but SciPy is not there yet
• Hence, probably stick with the matplotlib default of Numeric

Matlab to Python Interface (experimental)

• Two packages available, though not well documented and little development activity
  • pymat
    • I have been unsuccessful at getting it to compile properly in cygwin
  • mlabwrap
    • Can compile and get mlabraw.eval() to work, but not mlabraw.get(). Thus can call functions in matlab with string inputs, but cannot get values back into python.
    • Because of issuew with mlabraw, cannot get mlabwrap (higher level wrapper) to work

• Installation Instructions for partially working mlabraw (modify directories to your installation)
• Modified my setup.py to the proper directories on my computer:

   MATLAB_VERSION = 6.5
   MATLAB_DIR='c:/ProgramFiles/MATLAB6p5'
   EXTRA_COMPILE_ARGS=['-mno-cygwin']
   #EXTRA_COMPILE_ARGS=None
   PLATFORM_DIR='win32/microsoft/msvc60'
   # hopefully these 3 won't need modification
   MATLAB_LIBRARIES=None
   PYTHON_INCLUDE_DIR='/opt/Numeric-24.0b2/Include/Numeric'

• Changed line setup.py line 47 to 'cygwin' instead of 'win':

   if sys.platform.startswith('cygwin'):

• Added at line 51 of setup.py (under 'WINDOWS' section):

   CPP_LIBRARIES = ['stdc++']

• Added an extra include_dirs (for why, see step 5):

   include_dirs=MATLAB_INCLUDE_DIRS + ["/opt/mlabwrap-0.9.1/include"] + (PYTHON_INCLUDE_DIR and [PYTHON_INCLUDE_DIR] or []),

• Added an empty file to make the mingw32 compiler happy:

   mkdir -p /opt/mlabwrap-0.9.1/include/sys && touch /opt/mlabwrap-0.9.1/include/sys/select.h

• Set an environment variable for the compiler (not sure if makes a difference):

   export LD_LIBRARY_PATH=c:/ProgramFiles/MATLAB6p5/extern/lib/win32 

• At the command line:

   python setup.py build --compiler=mingw32
   python setup.py install 

• Verified Functionality of mlabraw

  phoebusc@phoebus /opt
  $ python
  Python 2.4.1 (#1, May 27 2005, 18:02:40)
  [GCC 3.3.3 (cygwin special)] on cygwin
  Type "help", "copyright", "credits" or "license" for more information.
  >>> import mlabraw
  >>> a = mlabraw.open()
  >>> a.get("a")
  Traceback (most recent call last):
    File "<stdin>", line 1, in ?
  AttributeError: 'int' object has no attribute 'get'
  >>> a
  422257344
  >>> mlabraw.get(a,'hi')
  Traceback (most recent call last):
    File "<stdin>", line 1, in ?
  mlabraw.error: Unable to get matrix from MATLAB(TM) workspace
  >>> mlabraw.get(a,abc)
  Traceback (most recent call last):
    File "<stdin>", line 1, in ?
  NameError: name 'abc' is not defined
  >>> help(mlabraw)
  
  >>> mlabraw.eval(a,"a=123")
  '\na =\n\n   123\n\n'
  >>> mlabraw.get(a,"a")
  Traceback (most recent call last):
    File "<stdin>", line 1, in ?
  
  Segmentation fault (core dumped)
  
  phoebusc@phoebus /opt
  $ python
  Python 2.4.1 (#1, May 27 2005, 18:02:40)
  [GCC 3.3.3 (cygwin special)] on cygwin
  Type "help", "copyright", "credits" or "license" for more information.
  >>> import mlabraw
  >>> mlabraw.eval(a,'b=int2str(123)')
  '\nb =\n\n123\n\n'
  >>> mlabraw.get(a,'b')
  '123'
  >>> import sys
  >>> import Numeric
  >>> dir()
  ['Numeric', '__builtins__', '__doc__', '__name__', 'a', 'mlabraw', 'sys']
  >>> mlabraw.eval(a,'c=123')
  '\nc =\n\n   123\n\n'
  >>> mlabraw.get(a,'c')
  Traceback (most recent call last):
    File "<stdin>", line 1, in ?
  Segmentation fault (core dumped)
  
  phoebusc@phoebus /opt
  $ python
  Python 2.4.1 (#1, May 27 2005, 18:02:40)
  [GCC 3.3.3 (cygwin special)] on cygwin
  Type "help", "copyright", "credits" or "license" for more information.
  >>> import mlabraw
  >>> a = mlabraw.open()
  >>> mlabraw.eval(a,'c(1,1,1) = 0')
  '\nc =\n\n     0\n\n'
  >>> mlabraw.get(a,'c')
  Traceback (most recent call last):
    File "<stdin>", line 1, in ?
  Segmentation fault (core dumped)
  
  phoebusc@phoebus /opt
  $