User:Luigi.scarso/luatex lunatic

From ConTeXt wiki

Introduction

After eurotex meeting 2009, I'm going to fix some typos here and there and maybe expand some examples too. I estimated that around 20 September article and site will be in synch.



!! W A R N I N G !!


!! THIS CODE IS HIGHLY EXPERIMENTAL !!


!! THIS CODE WORKS ONLY UNDER LINUX !!


!! THIS CODE WORKS ONLY WITH luatex 0.42 !!


!! THIS CODE WORKS ONLY WITH ConTeXt version: 2009.07.17 17:23



luatex_lunatic is a modification of lua side of luatex to host a python interpreter inside lua (see lunatic-python) .

I have made a set of patches because :

  • by design, lua in luatex doesn't permit dynamic loading ("Dynamic loading of .so and .dll files is disabled on all platforms." see "LUA changes" in manual/luatexref-t.pdf of src and/or svn dist.)
  • to prevent symbols collisions between luatex and an arbitrary library

A python interpreter hosted in luatex can make easier to use existing python's bindings; also the ctype modules (included in python releases at from 2.5 version) permits a binding to a .so without using SWIG or similar (almost like loadlib of lua) .

As general rule, I want the smallest set of patches ; so, for example, I have choose to not use a system libpng, even if it's easy to modify build.sh. to do so.

Also, I'm not concerned about portability: I will talk only about Linux .

Please note that luatex_lunatic was born to answer to (my) question "Can I apply lunatic to luatex ?" and nothing else.

Every others meanings can be interestring, but are not mine , and I will not partecipate to any discussion about that.

How to

I prefear to put a bash script that can help to compile a luatex-lunatic binary. It's clear that some skills are needed, so I hope in this way to avoid useless questions . Of course, I will try to correct all errors .

The trick is hide all, then unhide what is needed, then make only what is needed and in the end re-compile luatex .

The changes are :

  • in build.sh.linux add
28a29,36
> export CONFIG_SHELL=/bin/bash
> CFLAGS="-g -O2 -Wno-write-strings -fvisibility=hidden"
> CXXFLAGS="$CFLAGS -fvisibility-inlines-hidden"
> export CFLAGS
> export CXXFLAGS
> 
> 
> 
  • in source/texk/web2c/luatexdir/lua51/loadlib.c
69c69
<   void *lib = dlopen(path, RTLD_NOW);
---
>   void *lib = dlopen(path, RTLD_NOW|RTLD_GLOBAL);
  • in source/texk/web2c/Makefile.in:
98c98
< @MINGW32_FALSE@am__append_14 = -DLUA_USE_POSIX
---
> @MINGW32_FALSE@am__append_14 = -DLUA_USE_LINUX
1674c1674
<       $(CXXLINK) $(luatex_OBJECTS) $(luatex_LDADD) $(LIBS)
---
>       $(CXXLINK) $(luatex_OBJECTS) $(luatex_LDADD) $(LIBS) -Wl,-E -uluaL_openlibs -fvisibility=hidden -fvisibility-inlines-hidden -ldl -lreadline -lhistory -lncurses

So, first compile as usual

$>build.sh

Then use trick.zip" to unhide symbols from liblua51.a and recompile luatex (put it at the same level of build.sh) .

Python packages

These are python packages that are not in standard libraries ; "✔" means that I have made only small lua wrapper .

  • numpy,scipy matplot ✔ here
  • odfpy ✔
  • PIL, python imaging library ✔ here

For these, I still have to decide what todo.

  • TO FIX ; pygegl (I like its syntax, but need too much and it's easy only in python)
  • TODO: binding of libtiff with ctypes (or use gdal ?)
  • TODO :gle (for Massimiliano "Max" Dominici, GUIT)
  • TODO : using the binding of VIPS

Bindings

These are shared lib that come with native python binding, or eventually I have made a wrapper using ctypes .

  • ghostscript 8.64 ✔ here
  • graphviz 2.24.0 ✔ here
  • ImageMagick-6.4.9 with pythonmagickwand ✔ here
  • fontforge 20090224 ✔ here Useful to check symbols collision, and if one want to play with the last fontforge, eg to draw the outline of a glyph .
  • R-2.8.1 with rpy2-2.0.3 (For Maurizio "Mau" Himmelman , GUIT) ✔ here (someone says also considering-r-as-python-supplement ) .
  • quantlib 0.9.7 ✔ (need an example with output in pdf)
  • dbxml-2.4.16 (and sqlite) ✔ here

Dedicated systems

When we have a huge and coordinate set of *so with python binding, it' better to dedicate a specific luatex-lunatic . As example I choose sagemath and ROOT .


SageMath

From sagemath :
Sage is a free open-source mathematics software system licensed under the GPL. It combines the power of many existing open-source packages into a common Python-based interface.
Given that sagemath is rooted on a CPtyhon , we can try to use sagemath as "plugin" for luatex instead of use latex as external process of sagemath.

sagemath comes with own python, and we must use it; in this case I put all stuffs under a separated lunatic-python-SAGEMATH .

HOME_LUN=/opt/luatex/luatex-lunatic
cd $HOME_LUN
cp -r lunatic-python lunatic-python-SAGEMATH
cd lunatic-python-SAGEMATH
##
## We prepare new bridge
##
rm -r build
$HOME_LUN/sage/local/bin/python setup.py build
cd $HOME_LUN
mkdir tests-SAGEMATH && cd tests-SAGEMATH 
##
## I have already installed prev. python.so, I don't want mess things
##
ln -s ../lunatic-python-SAGEMATH/build/lib.linux-i686-2.5/python.so
ln -s  ../luatex/build/texk/web2c/luatex
##
## now we need to setup sagemath
##
cd $HOME_LUN/sage/local/bin
. sage-env
cd $HOME_LUN

We need also a first run of sagemath:

cd sage
./sage
##
## now exit from sage
##

That is .

Now we need a stub, because usually sage is used as an interactive shell:

## sagestub.py
from sage.all_cmdline import *

OK let's start with an example with mixed code (or, better, 'messed code'):

\startluacode
function test_ode(graphout)
  require("python")
  pg = python.globals()
  SAGESTUB = python.import("sagestub")
  sage = SAGESTUB.sage 
  python.execute([[
def f_1(t,y,params):
   return[y[1],-y[0]-params[0]*y[1]*(y[0]**2-1)]
]])
python.execute([[
def j_1(t,y,params):
    return [ [0,1.0],[-2.0*params[0]*y[0]*y[1]-1.0,-params[0]*(y[0]*y[0]-1.0)], [0,0] ]
]])

  T=sage.gsl.ode.ode_solver()
  T.algorithm="rk8pd"

  f_1 = pg.f_1  
  j_1 = pg.j_1
  pg.T=T

  python.execute("T.function=f_1")
  T.jacobian=j_1
  python.execute("T.ode_solve(y_0=[1,0],t_span=[0,100],params=[10],num_points=1000)")
  python.execute(string.format("T.plot_solution(filename='%s')",graphout ))
end
\stopluacode

\def\TestODE#1{%
\ctxlua{test_ode("#1")}%
\startcombination[2*1]
{%foo
\vbox{\hsize=8cm
Consider solving the Van der pol oscillator 

$x''(t) +ux'(t)(x(t)^2-1)+x(t)=0 $

between $t=0$ and $t= 100$.

As a first order system it is 

$x'=y$

$y'=-x+uy(1-x^2)$
Let us take $u=10$ and use initial conditions $(x,y)=(1,0)$ and use the
\emphsl{\hbox{runga-kutta} \hbox{prince-dormand}}  algorithm.
}%
}{\ss \ }
{\externalfigure[#1][width=9cm]}{\ss Result for 1000 points}
\stopcombination
}

\starttext
\startTEXpage
\TestODE{ode1.pdf}
\stopTEXpage
\stoptext

Test-ode.png

ROOT (CERN)

ROOT -- an Oject-Oriented Data Analysis Framework -- is explained in Wikipedia:
ROOT is an object-oriented program and library developed by CERN. It was originally designed for particle physics data analysis and contains several features specific to this field, but it is also commonly used in other applications such as astronomy and data mining.

For more infos, see here (here for python stuffs).

Under Linux installation is not difficult at all, but in this I choose to not create a luatex-lunatic apart, as done above for sagemath.

This example shot how to literally embed original python source code .

\startluacode
function test_ROOT(filename)
  require("python")
  pg = python.globals()

  python.execute([[
def run(filename):
    from ROOT import TCanvas, TGraph
    from ROOT import gROOT
    from math import sin
    from array import array


    gROOT.Reset()

    c1 = TCanvas( 'c1', 'A Simple Graph Example', 200, 10, 700, 500 )

    c1.SetFillColor( 42 )
    c1.SetGrid()

    n = 20
    x, y = array( 'd' ), array( 'd' )

    for i in range( n ):
          x.append( 0.1*i )
          y.append( 10*sin( x[i]+0.2 ) )

    gr = TGraph( n, x, y )
    gr.SetLineColor( 2 )
    gr.SetLineWidth( 4 )
    gr.SetMarkerColor( 4 )
    gr.SetMarkerStyle( 21 )
    gr.SetTitle( 'a simple graph' )
    gr.GetXaxis().SetTitle( 'X title' )
    gr.GetYaxis().SetTitle( 'Y title' )
    gr.Draw( 'ACP' )
    c1.Update()
    c1.Print(filename)
]])
  run = pg.run
  run(filename)
end
\stopluacode

\starttext
\startTEXpage
\ctxlua{test_ROOT("testsin.pdf")}
\rotate[rotation=90]{\externalfigure[testsin.pdf][width=5cm]}
\stopTEXpage
\stoptext
Testsin.jpg

We can do a bit better: separate python code from lua code .
Save this in test-ROOT1.py (so it's also easy to test) :

from ROOT import TCanvas, TGraph ,TGraphErrors,TMultiGraph
from ROOT import gROOT
from math import sin
from array import array

def run(filename):
    c1 = TCanvas("c1","multigraph",200,10,700,500)
    c1.SetGrid()

    # draw a frame to define the range
    mg = TMultiGraph()
    #   create first graph
    n = 24;
    x = array('d',range(24))
    data = file('data').readlines()
    for line in data:
        line = line.strip()
        y  = array('d',[float(d) for d in line.split()])
        gr =  TGraph(n,x,y)
        gr.Fit("pol6","q")
        mg.Add(gr)

    mg.Draw("ap")

    #force drawing of canvas to generate the fit TPaveStats
    c1.Update()
    c1.Print(filename)

Here file 'data' is a 110 lines file with 24 floats values space separated, ie
20.6000 19.4000 19.4000 18.3000 17.8000 16.1000 16.7000 21.1000 23.3000 26.1000 26.1000 27.2000 27.8000 28.3000 28.3000 27.2000 25.6000 22.8000 21.7000 21.7000 21.7000 21.7000 21.7000 21.7000 .
Now a tex file, with a simple layer in lua as interface for python:

\startluacode
function test_ROOT(filename)
  require("python")
  test = python.import('test-ROOT1')
  test.run(filename)
end
\stopluacode

\starttext
\startTEXpage
\ctxlua{test_ROOT("data.pdf")}
\rotate[rotation=90]{\externalfigure[data.pdf]}
\stopTEXpage
\stoptext
Test-ROOT1.jpg

ConTeXt mkIV examples

Here I will collect some tex snippets, just to show some ideas.

Scipy

Watch how python code z = x*np.exp(-x**2-y**2) is translated in lua code z = x.__mul__( np.exp( (x.__pow__(2).__add__(y.__pow__(2))).__neg__() ) )


\startluacode
function testSCIPY(figname,dpi)
  require("python")
  pg = python.globals()
  python.apply = python.eval('apply') or {}
  np = python.import("numpy")
  mlab =  python.import("matplotlib.mlab")
  griddata = mlab.griddata 
  plt = python.import("matplotlib.pyplot")
  ma = np.ma
  random =   python.import("numpy.random")
  uniform = random.uniform

  -- make up some randomly distributed data 
  npts = 200
  x = uniform(-2,2,npts)
  y = uniform(-2,2,npts)
  -- z = x*np.exp(-x**2-y**2)
  z = x.__mul__( np.exp( (x.__pow__(2).__add__(y.__pow__(2))).__neg__() ) )
  --  define grid. 
  xi = np.linspace(-2.1,2.1,100)
  yi = np.linspace(-2.1,2.1,100)
  -- grid the data.
  zi = griddata(x,y,z,xi,yi)
  --  contour the gridded data, plotting dots 
  --  at the randomly spaced data points. 
  --  we put this in python globals space  
  -- CS = plt.contour(xi,yi,zi,15,linewidths=0.5,colors='k')
  pg.xi = xi ; pg.yi = yi ; pg.zi = zi
  args = python.eval("[xi,yi,zi,15]")
  kv = python.eval("{'linewidth': 0.5 ,'colors' :'k'}")
  CS = python.apply(plt.contour, args,kv)
  --
  pg.jet = plt.cm.jet
  args = python.eval("[xi,yi,zi,15]")
  kv = python.eval("{'cmap': jet}")
  CS = python.apply(plt.contourf, args,kv)
  --  draw colorbar 
  plt.colorbar() 
  -- plot data points. 
  pg.x = x; pg.y = y 
  args = python.eval("[x,y]")
  kv = python.eval("{'marker': 'o', 'c':'b','s':5}")
  CS = python.apply(plt.scatter, args,kv)
  plt.xlim(-2,2)
  plt.ylim(-2,2)
  plt.title(string.format('griddata test (%i points)',npts))
  --plt.savefig(figname, dpi, 'white')
  --
  pg.figname = figname ; pg.dpi = dpi
  args = python.eval("[figname]")
  kv = python.eval("{'dpi': dpi ,'facecolor' :'white'}")
  CS = python.apply(plt.savefig, args,kv)
end
\stopluacode


\def\testSCIPY[#1]{%
\getparameters[scipy][#1]%
\ctxlua{testSCIPY("\csname scipyfigname\endcsname",
                  "\csname scipydpi\endcsname")}%
\externalfigure[\csname scipyfigname\endcsname]%
}

\starttext
\startTEXpage
\testSCIPY[figname={test-scipy-1.pdf},dpi={150}]
\stopTEXpage
\stoptext
Test-scipy.png

Python Imaging Library (PIL)

\startluacode
function testPIL(imageorig,imagesepia)
  require("python")
  PIL_Image = python.import("PIL.Image") 
  PIL_ImageOps = python.import("PIL.ImageOps") 
  python.execute([[
def make_linear_ramp(white):
    ramp = []
    r, g, b = white
    for i in range(255):
        ramp.extend((r*i/255, g*i/255, b*i/255))
    return ramp
]])
        
    -- make sepia ramp (tweak color as necessary)
    sepia = python.eval("make_linear_ramp((255, 240, 192))") 
    im = PIL_Image.open(imageorig)
    
    -- convert to grayscale
    if not(im.mode == "L")
     then 
        im = im.convert("L")
    end
    
    --  optional: apply contrast enhancement here, e.g.
    im = PIL_ImageOps.autocontrast(im)

    --  apply sepia palette
    im.putpalette(sepia)

    --  convert back to RGB so we can save it as JPEG
    --  (alternatively, save it in PNG or similar)
    im = im.convert("RGB")

    im.save(imagesepia)
end
\stopluacode

\def\SepiaImage#1#2{%
\ctxlua{testPIL("#1","#2")}%
\startcombination[2*1]
{\externalfigure[#1]}{\ss Orig.}
{\externalfigure[#2]}{\ss Sepia}
\stopcombination
}


\starttext
\startTEXpage
\SepiaImage{lena.jpg}{lena-sepia.jpg}
\stopTEXpage
\stoptext
Test-PIL.png


ImageMagick

ImageMagick® (here) is a software suite to create, edit, and compose bitmap images. It can read, convert and write images in a variety of formats (over 100) including DPX, EXR, GIF, JPEG, JPEG-2000, PDF, PhotoCD, PNG, Postscript, SVG, and TIFF. Use ImageMagick to translate, flip, mirror, rotate, scale, shear and transform images, adjust image colors, apply various special effects, or draw text, lines, polygons, ellipses and Bézier curves. There are at least two python bindings, and this time I consider PythonMagickWand which is a binding "ala" ctypes way .

Code is simple

\usetypescriptfile[type-gentium]
\usetypescript[gentium]
\setupbodyfont[gentium,10pt]
\setuppapersize[A5][A5]
\setuplayout[height=middle,topspace=1cm,header={2\lineheight},footer=0pt,backspace=1cm,margin=1cm, width=middle]


\startluacode
function testimagemagick(box,t)
  local w
  local h
  local d
  local f       
  local res = 118.11023622047244094488 -- 300 dpi
  local opacity = 25
  local sigma = 15
  local x  = 10
  local y  = 10 

  require("python")
  pg = python.globals()
  PythonMagickWand = python.import("PythonMagickWand")
  w =  math.floor((tex.wd[box] / 65536 ) / 72.27 * 2.54 * res )  
  h =  math.floor(((tex.ht[box] / 65536) + (tex.dp[box] / 65536)) / 72.27 *2.54 *res ) 
  f = string.format("%s.png",t)

  wand = PythonMagickWand.NewMagickWand()
  background = PythonMagickWand.NewPixelWand(0)
  -- PythonMagickWand.MagickNewImage(wand,w,h,background)
  PythonMagickWand.MagickNewImage(wand,w,h,background)

  PythonMagickWand.MagickSetImageResolution(wand,res,res)
  PythonMagickWand.MagickSetImageUnits(wand,PythonMagickWand.PixelsPerCentimeterResolution)
  PythonMagickWand.MagickShadowImage(wand,opacity,sigma,x,y) 
  PythonMagickWand.MagickWriteImage(wand ,f) 

  print(w,h,f)
end
\stopluacode




\def\testimagemagick[#1]{%
\getparameters[imagemagick][#1]%
\ctxlua{testimagemagick(\csname imagemagickbox\endcsname,"\csname imagemagickfilename\endcsname")}%
}

\newcount\shdw
\long\def\startShadowtext#1\stopShadowtext{%
\bgroup%
\setbox0=\vbox{#1}%
\testimagemagick[box=0,filename={shd-\the\shdw}]%
%%
\defineoverlay[backg][{\externalfigure[shd-\the\shdw.png]}]%
\framed[background=backg,frame=off,offset=4pt]{\box0}%
%%\framed{\box0}
\global\advance\shdw by 1%
\egroup%
}

\starttext
\startTEXpage%
\startShadowtext%
\input tufte
\stopShadowtext%
\stopTEXpage
\stoptext

And here is the result:

Test-imagemagick.png

Fontforge

In this example, we will use Metapost to draw a bezier curve of a glyph (Note: starting from Metapost 1.200 it is now possible to get the actual path drawing routines from a font glyph, so this example is only to show how to translate a path in metapost).
We will use 3-layer approach:

  1. a python layer that export a class,
  2. a lua layer to manage objects of this class
  3. a (con)TeX(t) layer that exports macros, because tex works very well with macros .

Let's start with python code, test-fontforge.py:

import sys
import fontforge


class simpledraw(object):

    def __init__(self,font_file):
        self.font = fontforge.open(font_file)

    def getcurve(self,letter):
        self.glname = letter
        res_Array =  []
        res = dict()
        try :
            #glyph_letter = [ g  for g in self.font.glyphs() if g.glyphname == self.glname][0]
            g = self.font[letter]
        except Exception ,e :
            res['err'] = str(e)
            res_Array.append(res)
            return res_Array
        layer_idx = 0;
        for layer_name in g.layers:
            layer = g.layers[layer_name]
            for contour_idx in range(len(layer)):
                res = dict()
                contour = layer[contour_idx]
                contour_name = contour.name
                res['name'] = contour.name
                res['is_quadratic'] = contour.is_quadratic
                res['closed'] = contour.closed
                res['points'] = [(p.x,p.y,"%i" %p.on_curve) for p in contour ]
                res['design_size'] = self.font.design_size
                res['em'] = self.font.em
                res_Array.append(res)    
        return res_Array


    def drawmpostpath(self,letter):
        res_Array = self.getcurve(letter) 
        state = 0
        paths = ''
        for res in res_Array:
            temp = '' 
            for p in res['points'] :
                if p[2]=='1' : 
                    if state == 1 :
                        temp = temp + '-- (%s,%s)' %(p[0] ,p[1]) ; state = 1; continue
                    else:
                        temp = temp + '.. (%s,%s)' %(p[0] ,p[1]) ; state = 1; continue
                if state == 1 : temp = temp + ' .. controls (%s,%s)' %(p[0],p[1]) ; state =2; continue
                if state == 2 : temp = temp + ' and  (%s,%s) ' %(p[0],p[1]) ; state =0; continue 
            if res['closed']  :  
                if state == 1 :
                    temp = 'draw ' + temp[2:] + " -- cycle;\n"
                else:
                    temp = 'draw ' + temp[2:] + " .. cycle;\n"
            else:
                temp = 'draw ' + temp[2:] + ";\n"
            paths  = paths + temp
        return paths



    def drawmpostpoints(self,letter):
        res_Array = self.getcurve(letter) 
        dots = ''
        for res in res_Array:
            temp = '\n'.join( ["drawdot %s;" %str((p[0],p[1])) for p in  res['points'] if p[2] == '1'] ) + "\n"
            dots = dots + temp
        return dots



if __name__ == '__main__':
    s = simpledraw("lmmono10-regular.otf")
    #res = s.getmpostpointsSugar('C')
    #print res
    #print s.getmpostoutline('C')
    print s.getcurve('e')
    print s.drawmpostpath('e')
    print s.drawmpostpoints('e')

Note the '__main__' check, so we can test this class from python.
Next lua layer, which in this case is embed in a tex file:


\setupcolors[state=start]


\startluacode
function testFontforge(fontfile,letter)
   require("python")    
   testoutlines = python.import("test-fontforge")
   s = testoutlines.simpledraw(fontfile)
   g = s.drawmpostpath(letter)
   p = s.drawmpostpoints(letter)
   --print( string.format("\%s = \%s ==", letter,g ))
  tex.sprint(tex.ctxcatcodes,"\\startMPcode")
  tex.sprint(tex.ctxcatcodes,"pickup pencircle scaled 1pt;")
  tex.sprint(tex.ctxcatcodes,string.format("\%s",g) )
  tex.sprint(tex.ctxcatcodes,"pickup pencircle scaled 8pt;")
  tex.sprint(tex.ctxcatcodes,string.format("\%s",p) )
  tex.sprint(tex.ctxcatcodes,"\\stopMPcode")
end
\stopluacode


\def\Outline[#1]{%
\getparameters[test][#1]%
\ctxlua{testFontforge("\testfontfile", "\testletter")}%
}

\starttext
\startTEXpage
\Outline[letter={C}, fontfile={lmmono10-regular.otf}]%
\Outline[letter={o}, fontfile={lmmono10-regular.otf}]%
\Outline[letter={n}, fontfile={lmmono10-regular.otf}]%
\Outline[letter={T}, fontfile={lmmono10-regular.otf}]%
\Outline[letter={e}, fontfile={lmmono10-regular.otf}]%
\Outline[letter={X}, fontfile={lmmono10-regular.otf}]%
\Outline[letter={t}, fontfile={lmmono10-regular.otf}]%
\stopTEXpage
\stoptext

Here we use tex.sprint(tex.ctxcatcodes,"\\stopMPcode") to inject tex code (actually Metapost code) into TeX parser .
\Outline is the TeX layer: of course one can write \Outline and testFontforge in a different manner to avoid use of tex.sprint(..)
And this is the result:
Test-fontforge.png

Ghostscript

There are essentially 2 kind of use of ghostscript :

  • convert an existing eps / ps file in pdf ;
  • use a program in postscript that take an input, do something and make a ps output ( e.g. a barcode/label generator ).

For the first case, we consider an implementation of eps2pdf, being ps2pdf virtually the same .
Actually there is not a python binding of ghostscript, so we build a simple wrapper using ctypes module testgs.py :

import ctypes
import sys


class gs(object):

   def __init__(self):

      self.ierrors = dict()
      self.ierrors['e_unknownerror'] = -1       
      self.ierrors['e_dictfull'] = -2
      self.ierrors['e_dictstackoverflow'] = -3
      self.ierrors['e_dictstackunderflow'] = -4
      self.ierrors['e_execstackoverflow'] = -5
      self.ierrors['e_interrupt'] = -6
      self.ierrors['e_invalidaccess'] = -7
      self.ierrors['e_invalidexit'] = -8
      self.ierrors['e_invalidfileaccess'] = -9
      self.ierrors['e_invalidfont'] = -10
      self.ierrors['e_invalidrestore'] = -11
      self.ierrors['e_ioerror'] = -12
      self.ierrors['e_limitcheck'] = -13
      self.ierrors['e_nocurrentpoint'] = -14
      self.ierrors['e_rangecheck'] = -15
      self.ierrors['e_stackoverflow'] = -16
      self.ierrors['e_stackunderflow'] = -17
      self.ierrors['e_syntaxerror'] = -18
      self.ierrors['e_timeout'] = -19
      self.ierrors['e_typecheck'] = -20
      self.ierrors['e_undefined'] = -21
      self.ierrors['e_undefinedfilename'] = -22
      self.ierrors['e_undefinedresult'] = -23
      self.ierrors['e_unmatchedmark'] = -24
      self.ierrors['e_VMerror'] = -25
      self.ierrors['e_configurationerror'] = -26
      self.ierrors['e_undefinedresource'] = -27
      self.ierrors['e_unregistered'] = -28
      self.ierrors['e_invalidcontext'] = -29
      self.ierrors['e_invalidid'] = -30
      self.ierrors['e_Fatal'] = -100
      self.ierrors['e_Quit'] = -101
      self.ierrors['e_InterpreterExit'] = -102
      self.ierrors['e_RemapColor'] = -103
      self.ierrors['e_ExecStackUnderflow'] = -104
      self.ierrors['e_VMreclaim'] = -105
      self.ierrors['e_NeedInput'] = -106
      self.ierrors['e_Info'] = -110

      self.libgspath = '/opt/luatex/luatex-lunatic/lib/libgs.so'

      self.OutFile = ''
      self.InFile = ''

      self.args = []

      
  
   def appendargs(self,arg):
     if arg.find('-sOutputFile')>= 0: return
     if arg.find('-c quit')>= 0: return
     self.args.append(arg)


   def rawappendargs(self,arg):
     self.args.append(arg)



   def run(self):
      libgs = ctypes.CDLL(self.libgspath)

      exit_status   = ctypes.c_int()
      code          = ctypes.c_int(1)
      code1         = ctypes.c_int()
      instance      = ctypes.c_void_p(None)
      exit_code     = ctypes.c_int()


      code.value = libgs.gsapi_new_instance(ctypes.byref(instance), None)
      if code.value  == 0 :
              libgs.gsapi_set_stdio(instance, None, None, None)
              self.args.insert(0,'')
              #
              if len(self.OutFile) > 0:
                 self.args.append('-sOutputFile=%s' %self.OutFile)
              if len(self.InFile) > 0:
                 self.args.append("%s" %self.InFile)
              self.args.append('-c quit')
              arguments = self.args
              #
              argc = ctypes.c_int(len(arguments))
              argv = (ctypes.c_char_p * argc.value)(*arguments)
              code.value = libgs.gsapi_init_with_args(instance, argc, argv)

              code1.value = libgs.gsapi_exit(instance)

              if code.value == 0 or code.value == self.ierrors['e_Quit']:
                  code.value  = code1.value

              if code.value == self.ierrors['e_Quit']:
                  code.value = 0 

              libgs.gsapi_delete_instance(instance)

      exit_status.value = 0

if __name__ == '__main__':
   pyctyps = gs()
   pyctyps.appendargs('-q')
   pyctyps.appendargs('-dNOPAUSE')
   pyctyps.appendargs('-dEPSCrop')
   pyctyps.appendargs('-sDEVICE=pdfwrite')
   pyctyps.InFile = 'test.eps'
   pyctyps.OutFile = 'test.pdf'
   pyctyps.run()

The tex code

\startluacode
function testgs(epsin,pdfout)
  require("python")
  gsmodule = python.import("testgs") 
  ghost = gsmodule.gs()
  ghost.appendargs('-q')
  ghost.appendargs('-dNOPAUSE')
  ghost.appendargs('-dEPSCrop')
  ghost.appendargs('-sDEVICE=pdfwrite')
  ghost.InFile = epsin
  ghost.OutFile = pdfout
  ghost.run()
end
\stopluacode

\def\epstopdf#1#2{\ctxlua{testgs("#1","#2")}}
\def\EPSfigure[#1]{%lazy way to load eps
\epstopdf{#1.eps}{#1.pdf}%
\externalfigure[#1.pdf]%
}

\starttext
\startTEXpage
\startcombination[2*1]
{\EPSfigure[tiger]}{\ss tiger.eps}
{\EPSfigure[golfer]}{\ss golfer.eps}
\stopcombination
\stopTEXpage
\stoptext

and the result :
Testgs.png

Another example:
here we use a library to generate barcodes (see here) .

\startluacode
function epstopdf(epsin,pdfout)
  require("python")
  gsmodule = python.import("testgs") 
  ghost = gsmodule.gs()
  ghost.appendargs('-q')
  ghost.appendargs('-dNOPAUSE')
  ghost.appendargs('-dEPSCrop')
  ghost.appendargs('-sDEVICE=pdfwrite')
  ghost.InFile = epsin
  ghost.OutFile = pdfout
  ghost.run()
end

function barcode(text,type,options,savefile)
  require("python")
  gsmodule = python.import("testgs") 

  barcode_string = string.format("%%!\n100 100 moveto (%s) (%s) %s barcode showpage" ,text,options,type)

  psfile = string.format("%s.ps",savefile)
  epsfile = string.format("%s.eps",savefile)
  pdffile = string.format("%s.pdf",savefile)

  temp = io.open(psfile,'w')
  temp:write(tostring(barcode_string),"\n")
  temp:flush()
  io.close(temp)
  ghost = gsmodule.gs()
  ghost.rawappendargs('-q')
  ghost.rawappendargs('-dNOPAUSE')
  ghost.rawappendargs('-sDEVICE=epswrite')
  ghost.rawappendargs(string.format('-sOutputFile=%s',epsfile))
  ghost.rawappendargs('barcode.ps')
  ghost.InFile= psfile
  ghost.run()
end
\stopluacode

\def\epstopdf#1#2{\ctxlua{epstopdf("#1","#2")}}
\def\EPSfigure[#1]{%lazy way to load eps
\epstopdf{#1.eps}{#1.pdf}%
\externalfigure[#1.pdf]%
}

\def\PutBarcode[#1]{%
\getparameters[bc][#1]%
\ctxlua{barcode("\csname bctext\endcsname","\csname bctype\endcsname","\csname bcoptions\endcsname","\csname bcsavefile\endcsname" )}%
\expanded{\EPSfigure[\csname bcsavefile\endcsname]}%
}

\starttext
\startTEXpage
\startcombination[2*2]
{\PutBarcode[text={CODE 39},type={code39},options={includecheck includetext},savefile={TEMP1}]}{\ss code39}
{\PutBarcode[text={CONTEXT},type={code93},options={includecheck includetext},savefile={TEMP2}]}{\ss code93}
{\PutBarcode[text={977147396801},type={ean13},options={includetext},savefile={TEMP3}]}{\ss ean13}
{\PutBarcode[text={0123456789},type={interleaved2of5},options={includecheck includetext},savefile={TEMP4}]}{\ss interleaved2of5}
\stopcombination
\stopTEXpage
\stoptext

Test-ghostscript-barcode.png

Graphviz

Graphviz is a Graph Visualization Software .
Standard distribution comes with several binding (lua and python among others) so it's not difficult to integrate in luatex lunatic .
In this example, we draw a graph of the nodes of

\def\StudyBox#1{%
\startluacode
require "python"
gv = python.import("gv")
g = gv.digraph("G")
gv.setv(g,'rankdir','LR')

nodes = nodes or {}

local nd = {};
local kd = {};

local k = 0;
local function nodesprint(head,n)
  while head do
    local id = head.id
    local oldn = n
    ndlbl = string.format("nd_\%03d",k)
    texio.write_nl(string.format("id=\%s, ndlbl=\%s",id,ndlbl))

    if id == node.id("vlist") then
     k = k + 1
     nd[n] = gv.node(g,ndlbl)
     res = gv.setv(nd[n],"shape","record")
     res = gv.setv(nd[n],"label",tostring(k) .. " " .. 
                          "vlist" .. "\|id:" .. tostring(head.id) .. 
                          "\|subtype:" .. tostring(head.subtype) .. 
                          "\|attr:" .. string.gsub(tostring(head.attr),"([><])","\\\%1") .. 
                          "\|width:" .. tostring(head.width) .. 
                          "\|depth:" .. tostring(head.depth) .. "\|height:" .. tostring(head.height) .. 
                          "\|dir:" .. tostring(head.dir) .. "\|shift:" .. tostring(head.shift) .. 
                          "\|glue_order:" .. tostring(head.glue_order) .. 
                          "\|glue_sign:" .. tostring(head.glue_sign) .. 
                          "\|glue_set:" .. tostring(head.glue_set) .. 
                          "\|list:" .. string.gsub(tostring(head.list),"([><])","\\\%1") .. 
                          "\|prev:" .. string.gsub(tostring(head.prev),"([><])","\\\%1") .. 
                          "\|next:" .. string.gsub(tostring(head.next),"([><])","\\\%1"))
    end

    if id == node.id("rule") then
     k = k + 1
     nd[n] = gv.node(g,ndlbl)
     res = gv.setv(nd[n],"shape","record")
     res = gv.setv(nd[n],"label",tostring(k) .. " " .. "rule" .. 
             "\|id:" .. tostring(head.id) .. 
             "\|subtype:" .. tostring(head.subtype) .. 
             "\|attr:" .. string.gsub(tostring(head.attr),"([><])","\\\%1") .. 
             "\|width:" .. tostring(head.width) .. 
             "\|depth:" .. tostring(head.depth) .. 
             "\|height:" .. tostring(head.height) .. 
             "\|dir:" .. tostring(head.dir) .. 
             "\|next:" .. string.gsub(tostring(head.next),"([><])","\\\%1") .. 
             "\|prev:" .. string.gsub(tostring(head.prev),"([><])","\\\%1"))
    end
    if id == node.id("ins") then
     k = k + 1
     nd[n] = gv.node(g,ndlbl)
     res = gv.setv(nd[n],"shape","record")
     res = gv.setv(nd[n],"label",tostring(k) .. " " .. "ins" .. 
             "\|id:" .. tostring(head.id) .. 
             "\|subtype:" .. tostring(head.subtype) .. 
             "\|attr:" .. string.gsub(tostring(head.attr),"([><])","\\\%1") .. 
             "\|cost:" .. tostring(head.cost) .. 
             "\|depth:" .. tostring(head.depth) .. 
             "\|height:" .. tostring(head.height) .. 
             "\|spec:" .. string.gsub(tostring(head.spec),"([><])","\\\%1") .. 
             "\|list:" .. string.gsub(tostring(head.list),"([><])","\\\%1") .. 
             "\|next:" .. string.gsub(tostring(head.next),"([><])","\\\%1") .. 
             "\|prev:" .. string.gsub(tostring(head.prev),"([><])","\\\%1"))
    end
    if id == node.id("mark") then
     k = k + 1
     nd[n] = gv.node(g,ndlbl)
     res = gv.setv(nd[n],"shape","record")
     res = gv.setv(nd[n],"label",tostring(k) .. " " .. "mark" .. 
             "\|id:" .. tostring(head.id) .. 
             "\|subtype:" .. tostring(head.subtype) .. 
             "\|attr:" .. string.gsub(tostring(head.attr),"([><])","\\\%1") .. 
             "\|class:" .. tostring(head.class) .. 
             "\|mark:" .. tostring(head.mark) .. 
             "\|next:" .. string.gsub(tostring(head.next),"([><])","\\\%1") .. 
             "\|prev:" .. string.gsub(tostring(head.prev),"([><])","\\\%1"))
    end
    if id == node.id("adjust") then
     k = k + 1
     nd[n] = gv.node(g,ndlbl)     res = gv.setv(nd[n],"shape","record")
     res = gv.setv(nd[n],"label",tostring(k) .. " " .. "adjust" .. 
             "\|id:" .. tostring(head.id) .. 
             "\|subtype:" .. tostring(head.subtype) .. 
             "\|attr:" .. string.gsub(tostring(head.attr),"([><])","\\\%1") .. 
             "\|list:" .. string.gsub(tostring(head.list),"([><])","\\\%1") .. 
             "\|prev:" .. string.gsub(tostring(head.prev),"([><])","\\\%1") .. 
             "\|next:" .. string.gsub(tostring(head.next),"([><])","\\\%1"))
    end
    if id == node.id("disc") then
     k = k + 1
     nd[n] = gv.node(g,ndlbl)     res = gv.setv(nd[n],"shape","record")
     res = gv.setv(nd[n],"label",tostring(k) .. " " .. "disc" .. 
             "\|id:" .. tostring(head.id) .. 
             "\|subtype:" .. tostring(head.subtype) .. 
             "\|attr:" .. string.gsub(tostring(head.attr),"([><])","\\\%1") .. 
             "\|pre:" .. string.gsub(tostring(head.pre),"([><])","\\\%1") .. 
             "\|post:" .. string.gsub(tostring(head.post),"([><])","\\\%1") .. 
             "\|replace:" .. string.gsub(tostring(head.replace),"([><])","\\\%1") .. 
             "\|next:" .. string.gsub(tostring(head.next),"([><])","\\\%1") .. 
             "\|prev:" .. string.gsub(tostring(head.prev),"([><])","\\\%1"))
    end
    if id == node.id("whatsit") then
     k = k + 1
     nd[n] = gv.node(g,ndlbl)     res = gv.setv(nd[n],"shape","record")
     if head.subtype == node.subtype("write") then
       res = gv.setv(nd[n],"label",tostring(k) .. " " .."whatsit:write" .. 
             "\|id:" .. tostring(head.id) .. 
             "\|subtype:" .. tostring(head.subtype) .. 
             "\|attr:" .. string.gsub(tostring(head.attr),"([><])","\\\%1") .. 
             "\|stream:" .. tostring(head.stream) .. 
             "\|data:" .. tostring(head.data) .. 
             "\|next:" .. string.gsub(tostring(head.next),"([><])","\\\%1") .. 
             "\|prev:" .. string.gsub(tostring(head.prev),"([><])","\\\%1"))
     end
     if head.subtype == node.subtype("close") then
       res = gv.setv(nd[n],"label",tostring(k) .. " " .."whatsit:close" .. 
             "\|id:" .. tostring(head.id) .. 
             "\|subtype:" .. tostring(head.subtype) .. 
             "\|attr:" .. string.gsub(tostring(head.attr),"([><])","\\\%1") .. 
             "\|stream:" .. tostring(head.stream) .. 
             "\|prev:" .. string.gsub(tostring(head.prev),"([><])","\\\%1") .. 
             "\|next:" .. string.gsub(tostring(head.next),"([><])","\\\%1"))
     end
     if head.subtype == node.subtype("special") then
       res = gv.setv(nd[n],"label",tostring(k) .. " " .."whatsit:special" .. 
             "\|id:" .. tostring(head.id) .. 
             "\|subtype:" .. tostring(head.subtype) .. 
             "\|attr:" .. string.gsub(tostring(head.attr),"([><])","\\\%1") .. 
             "\|data:" .. tostring(head.data) .. 
             "\|prev:" .. string.gsub(tostring(head.prev),"([><])","\\\%1") .. 
             "\|next:" .. string.gsub(tostring(head.next),"([><])","\\\%1"))
     end
     if head.subtype == node.subtype("local_par") then
       res = gv.setv(nd[n],"label",tostring(k) .. " " .."whatsit:local_par" .. 
             "\|id:" .. tostring(head.id) .. 
             "\|subtype:" .. tostring(head.subtype) .. 
             "\|attr:" .. string.gsub(tostring(head.attr),"([><])","\\\%1") .. 
             "\|pen_inter:" .. tostring(head.pen_inter) .. 
             "\|pen_broken:" .. tostring(head.pen_broken) .. 
             "\|dir:" .. tostring(head.dir) .. 
             "\|box_left:" .. tostring(head.box_left) .. 
             "\|box_left_width:" .. tostring(head.box_left_width) .. 
             "\|box_right:" .. tostring(head.box_right) .. 
             "\|box_right_width:" .. tostring(head.box_right_width) .. 
             "\|prev:" .. string.gsub(tostring(head.prev),"([><])","\\\%1") .. 
             "\|next:" .. string.gsub(tostring(head.next),"([><])","\\\%1"))
     end
     if head.subtype == node.subtype("dir") then
       res = gv.setv(nd[n],"label",tostring(k) .. " " .."whatsit:dir" .. 
             "\|id:" .. tostring(head.id) .. 
             "\|subtype:" .. tostring(head.subtype) .. 
             "\|attr:" .. string.gsub(tostring(head.attr),"([><])","\\\%1") .. 
             "\|dir:" .. tostring(head.dir) .. 
             "\|level:" .. tostring(head.level) .. 
             "\|dvi_ptr:" .. tostring(head.dvi_ptr) .. 
             "\|dvi_h:" .. tostring(head.dvi_h) .. 
             "\|next:" .. string.gsub(tostring(head.next),"([><])","\\\%1") .. 
             "\|prev:" .. string.gsub(tostring(head.prev),"([><])","\\\%1"))
     end
     if head.subtype == node.subtype("pdf_literal") then
       res = gv.setv(nd[n],"label",tostring(k) .. " " .."whatsit:pdf_literal" .. 
             "\|id:" .. tostring(head.id) .. 
             "\|subtype:" .. tostring(head.subtype) .. 
             "\|attr:" .. string.gsub(tostring(head.attr),"([><])","\\\%1") .. 
             "\|mode:" .. tostring(head.mode) .. 
             "\|data:" .. tostring(head.data) .. 
             "\|next:" .. string.gsub(tostring(head.next),"([><])","\\\%1") .. 
             "\|prev:" .. string.gsub(tostring(head.prev),"([><])","\\\%1"))
     end
     if head.subtype == node.subtype("pdf_refobj") then
       res = gv.setv(nd[n],"label",tostring(k) .. " " .."whatsit:pdf_refobj" .. 
             "\|id:" .. tostring(head.id) .. 
             "\|subtype:" .. tostring(head.subtype) .. 
             "\|attr:" .. string.gsub(tostring(head.attr),"([><])","\\\%1") .. 
             "\|objnum:" .. tostring(head.objnum) .. 
             "\|prev:" .. string.gsub(tostring(head.prev),"([><])","\\\%1") .. 
             "\|next:" .. string.gsub(tostring(head.next),"([><])","\\\%1"))
     end
     if head.subtype == node.subtype("pdf_refxform") then
       res = gv.setv(nd[n],"label",tostring(k) .. " " .."whatsit:pdf_refxform" .. 
             "\|id:" .. tostring(head.id) .. 
             "\|subtype:" .. tostring(head.subtype) .. 
             "\|attr:" .. string.gsub(tostring(head.attr),"([><])","\\\%1") .. 
             "\|width:" .. tostring(head.width) .. 
             "\|height:" .. tostring(head.height) .. 
             "\|depth:" .. tostring(head.depth) .. 
             "\|objnum:" .. tostring(head.objnum) .. 
             "\|next:" .. string.gsub(tostring(head.next),"([><])","\\\%1") .. 
             "\|prev:" .. string.gsub(tostring(head.prev),"([><])","\\\%1"))
     end
     if head.subtype == node.subtype("pdf_refximage") then
       res = gv.setv(nd[n],"label",tostring(k) .. " " .."whatsit:pdf_refximage" .. 
             "\|id:" .. tostring(head.id) .. 
             "\|subtype:" .. tostring(head.subtype) .. 
             "\|attr:" .. string.gsub(tostring(head.attr),"([><])","\\\%1") .. 
             "\|width:" .. tostring(head.width) .. 
             "\|height:" .. tostring(head.height) .. 
             "\|depth:" .. tostring(head.depth) .. 
             "\|objnum:" .. tostring(head.objnum) .. 
             "\|next:" .. string.gsub(tostring(head.next),"([><])","\\\%1") .. 
             "\|prev:" .. string.gsub(tostring(head.prev),"([><])","\\\%1"))
     end
     if head.subtype == node.subtype("pdf_annot") then
       res = gv.setv(nd[n],"label",tostring(k) .. " " .."whatsit:pdf_annot" .. 
             "\|id:" .. tostring(head.id) .. 
             "\|subtype:" .. tostring(head.subtype) .. 
             "\|attr:" .. string.gsub(tostring(head.attr),"([><])","\\\%1") .. 
             "\|width:" .. tostring(head.width) .. 
             "\|height:" .. tostring(head.height) .. 
             "\|depth:" .. tostring(head.depth) .. 
             "\|objnum:" .. tostring(head.objnum) .. 
             "\|data:" .. tostring(head.data) .. 
             "\|next:" .. string.gsub(tostring(head.next),"([><])","\\\%1") .. 
             "\|prev:" .. string.gsub(tostring(head.prev),"([><])","\\\%1"))
     end
     if head.subtype == node.subtype("pdf_start_link") then
       res = gv.setv(nd[n],"label",tostring(k) .. " " .."whatsit:pdf_start_link" .. 
             "\|id:" .. tostring(head.id) .. 
             "\|subtype:" .. tostring(head.subtype) .. 
             "\|attr:" .. string.gsub(tostring(head.attr),"([><])","\\\%1") .. 
             "\|width:" .. tostring(head.width) .. 
             "\|height:" .. tostring(head.height) .. 
             "\|depth:" .. tostring(head.depth) .. 
             "\|objnum:" .. tostring(head.objnum) .. 
             "\|link_attr:" .. tostring(head.link_attr) .. 
             "\|action:" .. tostring(head.action) .. 
             "\|prev:" .. string.gsub(tostring(head.prev),"([><])","\\\%1") .. 
             "\|next:" .. string.gsub(tostring(head.next),"([><])","\\\%1"))
     end
     if head.subtype == node.subtype("pdf_end_link") then
       res = gv.setv(nd[n],"label",tostring(k) .. " " .."whatsit:pdf_end_link" .. 
             "\|id:" .. tostring(head.id) .. 
             "\|subtype:" .. tostring(head.subtype) .. 
             "\|attr:" .. string.gsub(tostring(head.attr),"([><])","\\\%1") .. 
             "\|prev:" .. string.gsub(tostring(head.prev),"([><])","\\\%1") .. 
             "\|next:" .. string.gsub(tostring(head.next),"([><])","\\\%1"))
     end
     if head.subtype == node.subtype("pdf_dest") then
       res = gv.setv(nd[n],"label",tostring(k) .. " " .."whatsit:pdf_dest" .. 
             "\|id:" .. tostring(head.id) .. 
             "\|subtype:" .. tostring(head.subtype) .. 
             "\|attr:" .. string.gsub(tostring(head.attr),"([><])","\\\%1") .. 
             "\|width:" .. tostring(head.width) .. 
             "\|height:" .. tostring(head.height) .. 
             "\|depth:" .. tostring(head.depth) .. 
             "\|named_id:" .. tostring(head.named_id) .. 
             "\|dest_id:" .. tostring(head.dest_id) .. 
             "\|dest_type:" .. tostring(head.dest_type) .. 
             "\|xyz_zoom:" .. tostring(head.xyz_zoom) .. 
             "\|objnum:" .. tostring(head.objnum) .. 
             "\|prev:" .. string.gsub(tostring(head.prev),"([><])","\\\%1") .. 
             "\|next:" .. string.gsub(tostring(head.next),"([><])","\\\%1"))
     end
     if head.subtype == node.subtype("pdf_thread") then
       res = gv.setv(nd[n],"label",tostring(k) .. " " .."whatsit:pdf_thread" .. 
             "\|id:" .. tostring(head.id) .. 
             "\|subtype:" .. tostring(head.subtype) .. 
             "\|attr:" .. string.gsub(tostring(head.attr),"([><])","\\\%1") .. 
             "\|width:" .. tostring(head.width) .. 
             "\|height:" .. tostring(head.height) .. 
             "\|depth:" .. tostring(head.depth) .. 
             "\|named_id:" .. tostring(head.named_id) .. 
             "\|thread_id:" .. tostring(head.thread_id) .. 
             "\|thread_attr:" .. tostring(head.thread_attr) .. 
             "\|prev:" .. string.gsub(tostring(head.prev),"([><])","\\\%1") .. 
             "\|next:" .. string.gsub(tostring(head.next),"([><])","\\\%1"))
     end
     if head.subtype == node.subtype("pdf_start_thread") then
       res = gv.setv(nd[n],"label",tostring(k) .. " " .."whatsit:pdf_start_thread" .. 
             "\|id:" .. tostring(head.id) .. 
             "\|subtype:" .. tostring(head.subtype) .. 
             "\|attr:" .. string.gsub(tostring(head.attr),"([><])","\\\%1") .. 
             "\|width:" .. tostring(head.width) .. 
             "\|height:" .. tostring(head.height) .. 
             "\|depth:" .. tostring(head.depth) .. 
             "\|named_id:" .. tostring(head.named_id) .. 
             "\|thread_id:" .. tostring(head.thread_id) .. 
             "\|thread_attr:" .. tostring(head.thread_attr) .. 
             "\|prev:" .. string.gsub(tostring(head.prev),"([><])","\\\%1") .. 
             "\|next:" .. string.gsub(tostring(head.next),"([><])","\\\%1"))
     end
     if head.subtype == node.subtype("pdf_end_thread") then
       res = gv.setv(nd[n],"label",tostring(k) .. " " .."whatsit:pdf_end_thread" .. 
             "\|id:" .. tostring(head.id) .. 
             "\|subtype:" .. tostring(head.subtype) .. 
             "\|attr:" .. string.gsub(tostring(head.attr),"([><])","\\\%1") .. 
             "\|prev:" .. string.gsub(tostring(head.prev),"([><])","\\\%1") .. 
             "\|next:" .. string.gsub(tostring(head.next),"([><])","\\\%1"))
     end
     if head.subtype == node.subtype("pdf_save_pos") then
       res = gv.setv(nd[n],"label",tostring(k) .. " " .."whatsit:pdf_save_pos" .. 
             "\|id:" .. tostring(head.id) .. 
             "\|subtype:" .. tostring(head.subtype) .. 
             "\|attr:" .. string.gsub(tostring(head.attr),"([><])","\\\%1") .. 
             "\|prev:" .. string.gsub(tostring(head.prev),"([><])","\\\%1") .. 
             "\|next:" .. string.gsub(tostring(head.next),"([><])","\\\%1"))
     end
     if head.subtype == node.subtype("pdf_thread_data") then
       res = gv.setv(nd[n],"label",tostring(k) .. " " .."whatsit:pdf_thread_data" .. 
             "\|id:" .. tostring(head.id) .. 
             "\|subtype:" .. tostring(head.subtype) .. 
             "\|next:" .. string.gsub(tostring(head.next),"([><])","\\\%1"))
     end
     if head.subtype == node.subtype("pdf_link_data") then
       res = gv.setv(nd[n],"label",tostring(k) .. " " .."whatsit:pdf_link_data" .. 
             "\|id:" .. tostring(head.id) .. 
             "\|subtype:" .. tostring(head.subtype) .. 
             "\|next:" .. string.gsub(tostring(head.next),"([><])","\\\%1"))
     end
     if head.subtype == node.subtype("open") then
       res = gv.setv(nd[n],"label",tostring(k) .. " " .."whatsit:open" .. 
             "\|id:" .. tostring(head.id) .. 
             "\|subtype:" .. tostring(head.subtype) .. 
             "\|attr:" .. string.gsub(tostring(head.attr),"([><])","\\\%1") .. 
             "\|stream:" .. tostring(head.stream) .. 
             "\|name:" .. tostring(head.name) .. 
             "\|area:" .. tostring(head.area) .. 
             "\|ext:" .. tostring(head.ext) .. 
             "\|next:" .. string.gsub(tostring(head.next),"([><])","\\\%1") .. 
             "\|prev:" .. string.gsub(tostring(head.prev),"([><])","\\\%1"))
     end
     if head.subtype == node.subtype("late_lua") then
       res = gv.setv(nd[n],"label",tostring(k) .. " " .."whatsit:late_lua" .. 
             "\|id:" .. tostring(head.id) .. 
             "\|subtype:" .. tostring(head.subtype) .. 
             "\|attr:" .. string.gsub(tostring(head.attr),"([><])","\\\%1") .. 
             "\|reg:" .. tostring(head.reg) .. 
             "\|data:" .. tostring(head.data) .. 
             "\|name:" .. tostring(head.name) .. 
             "\|next:" .. string.gsub(tostring(head.next),"([><])","\\\%1") .. 
             "\|prev:" .. string.gsub(tostring(head.prev),"([><])","\\\%1"))
     end
     if head.subtype == node.subtype("fake") then
       res = gv.setv(nd[n],"label",tostring(k) .. " " .."whatsit:fake" .. 
             "\|id:" .. tostring(head.id) .. 
             "\|subtype:" .. tostring(head.subtype) .. 
             "\|next:" .. string.gsub(tostring(head.next),"([><])","\\\%1"))
     end
     if head.subtype == node.subtype("pdf_colorstack") then
       res = gv.setv(nd[n],"label",tostring(k) .. " " .."whatsit:pdf_colorstack" .. 
             "\|id:" .. tostring(head.id) .. 
             "\|subtype:" .. tostring(head.subtype) .. 
             "\|attr:" .. string.gsub(tostring(head.attr),"([><])","\\\%1") .. 
             "\|stack:" .. tostring(head.stack) .. 
             "\|cmd:" .. tostring(head.cmd) .. 
             "\|data:" .. tostring(head.data) .. 
             "\|next:" .. string.gsub(tostring(head.next),"([><])","\\\%1") .. 
             "\|prev:" .. string.gsub(tostring(head.prev),"([><])","\\\%1"))
     end
     if head.subtype == node.subtype("pdf_save") then
       res = gv.setv(nd[n],"label",tostring(k) .. " " .."whatsit:pdf_save" .. 
             "\|id:" .. tostring(head.id) .. 
             "\|subtype:" .. tostring(head.subtype) .. 
             "\|attr:" .. string.gsub(tostring(head.attr),"([><])","\\\%1") .. 
             "\|prev:" .. string.gsub(tostring(head.prev),"([><])","\\\%1") .. 
             "\|next:" .. string.gsub(tostring(head.next),"([><])","\\\%1"))
     end
     if head.subtype == node.subtype("cancel_boundary") then
       res = gv.setv(nd[n],"label",tostring(k) .. " " .."whatsit:cancel_boundary" .. 
             "\|id:" .. tostring(head.id) .. 
             "\|subtype:" .. tostring(head.subtype) .. 
             "\|attr:" .. string.gsub(tostring(head.attr),"([><])","\\\%1") .. 
             "\|prev:" .. string.gsub(tostring(head.prev),"([><])","\\\%1") .. 
             "\|next:" .. string.gsub(tostring(head.next),"([><])","\\\%1"))
     end
     if head.subtype == node.subtype("close_lua") then
       res = gv.setv(nd[n],"label",tostring(k) .. " " .."whatsit:close_lua" .. 
             "\|id:" .. tostring(head.id) .. 
             "\|subtype:" .. tostring(head.subtype) .. 
             "\|attr:" .. string.gsub(tostring(head.attr),"([><])","\\\%1") .. 
             "\|reg:" .. tostring(head.reg) .. 
             "\|prev:" .. string.gsub(tostring(head.prev),"([><])","\\\%1") .. 
             "\|next:" .. string.gsub(tostring(head.next),"([><])","\\\%1"))
     end
     if head.subtype == node.subtype("pdf_setmatrix") then
       res = gv.setv(nd[n],"label",tostring(k) .. " " .."whatsit:pdf_setmatrix" .. 
             "\|id:" .. tostring(head.id) .. 
             "\|subtype:" .. tostring(head.subtype) .. 
             "\|attr:" .. string.gsub(tostring(head.attr),"([><])","\\\%1") .. 
             "\|data:" .. tostring(head.data) .. 
             "\|prev:" .. string.gsub(tostring(head.prev),"([><])","\\\%1") .. 
             "\|next:" .. string.gsub(tostring(head.next),"([><])","\\\%1"))
     end
     if head.subtype == node.subtype("pdf_restore") then
       res = gv.setv(nd[n],"label",tostring(k) .. " " .."whatsit:pdf_restore" .. 
             "\|id:" .. tostring(head.id) .. 
             "\|subtype:" .. tostring(head.subtype) .. 
             "\|attr:" .. string.gsub(tostring(head.attr),"([><])","\\\%1") .. 
             "\|prev:" .. string.gsub(tostring(head.prev),"([><])","\\\%1") .. 
             "\|next:" .. string.gsub(tostring(head.next),"([><])","\\\%1"))
     end
     if head.subtype == node.subtype("user_defined") then
       res = gv.setv(nd[n],"label",tostring(k) .. " " .."whatsit:user_defined" .. 
             "\|id:" .. tostring(head.id) .. 
             "\|subtype:" .. tostring(head.subtype) .. 
             "\|attr:" .. string.gsub(tostring(head.attr),"([><])","\\\%1") .. 
             "\|user_id:" .. tostring(head.user_id) .. 
             "\|type:" .. tostring(head.type) .. 
             "\|value:" .. tostring(head.value) .. 
             "\|next:" .. string.gsub(tostring(head.next),"([><])","\\\%1") .. 
             "\|prev:" .. string.gsub(tostring(head.prev),"([><])","\\\%1"))
     end
    end
    if id == node.id("math") then
     k = k + 1
     nd[n] = gv.node(g,ndlbl)     res = gv.setv(nd[n],"shape","record")
     res = gv.setv(nd[n],"label",tostring(k) .. " " .. "math" .. 
             "\|id:" .. tostring(head.id) .. 
             "\|subtype:" .. tostring(head.subtype) .. 
             "\|attr:" .. string.gsub(tostring(head.attr),"([><])","\\\%1") .. 
             "\|surround:" .. tostring(head.surround) .. 
             "\|prev:" .. string.gsub(tostring(head.prev),"([><])","\\\%1") .. 
             "\|next:" .. string.gsub(tostring(head.next),"([><])","\\\%1"))
    end
    if id == node.id("glue") then
     k = k + 1
     nd[n] = gv.node(g,ndlbl)     res = gv.setv(nd[n],"shape","record")
     res = gv.setv(nd[n],"label",tostring(k) .. " " .. "glue" .. 
             "\|id:" .. tostring(head.id) .. 
             "\|subtype:" .. tostring(head.subtype) .. 
             "\|attr:" .. string.gsub(tostring(head.attr),"([><])","\\\%1") .. 
             "\|spec:" .. string.gsub(tostring(head.spec),"([><])","\\\%1") .. 
             "\|leader:" .. tostring(head.leader) .. 
             "\|next:" .. string.gsub(tostring(head.next),"([><])","\\\%1") .. 
             "\|prev:" .. string.gsub(tostring(head.prev),"([><])","\\\%1"))
    end
    if id == node.id("kern") then
     k = k + 1
     nd[n] = gv.node(g,ndlbl)     res = gv.setv(nd[n],"shape","record")
     res = gv.setv(nd[n],"label",tostring(k) .. " " .. "kern" .. 
             "\|id:" .. tostring(head.id) .. 
             "\|subtype:" .. tostring(head.subtype) .. 
             "\|attr:" .. string.gsub(tostring(head.attr),"([><])","\\\%1") .. 
             "\|kern:" .. tostring(head.kern) .. 
             "\|prev:" .. string.gsub(tostring(head.prev),"([><])","\\\%1") .. 
             "\|next:" .. string.gsub(tostring(head.next),"([><])","\\\%1"))
    end
    if id == node.id("penalty") then
     k = k + 1
     nd[n] = gv.node(g,ndlbl)     res = gv.setv(nd[n],"shape","record")
     res = gv.setv(nd[n],"label",tostring(k) .. " " .. "penalty" .. 
             "\|id:" .. tostring(head.id) .. 
             "\|subtype:" .. tostring(head.subtype) .. 
             "\|attr:" .. string.gsub(tostring(head.attr),"([><])","\\\%1") .. 
             "\|penalty:" .. tostring(head.penalty) .. 
             "\|prev:" .. string.gsub(tostring(head.prev),"([><])","\\\%1") .. 
             "\|next:" .. string.gsub(tostring(head.next),"([><])","\\\%1"))
    end
    if id == node.id("unset") then
     k = k + 1
     nd[n] = gv.node(g,ndlbl)     res = gv.setv(nd[n],"shape","record")
     res = gv.setv(nd[n],"label",tostring(k) .. " " .. "unset" .. 
             "\|id:" .. tostring(head.id) .. 
             "\|subtype:" .. tostring(head.subtype) .. 
             "\|attr:" .. string.gsub(tostring(head.attr),"([><])","\\\%1") .. 
             "\|width:" .. tostring(head.width) .. 
             "\|depth:" .. tostring(head.depth) .. 
             "\|height:" .. tostring(head.height) .. 
             "\|dir:" .. tostring(head.dir) .. 
             "\|shrink:" .. tostring(head.shrink) .. 
             "\|glue_order:" .. tostring(head.glue_order) .. 
             "\|glue_sign:" .. tostring(head.glue_sign) .. 
             "\|stretch:" .. tostring(head.stretch) .. 
             "\|span:" .. tostring(head.span) .. 
             "\|list:" .. string.gsub(tostring(head.list),"([><])","\\\%1") .. 
             "\|prev:" .. string.gsub(tostring(head.prev),"([><])","\\\%1") .. 
             "\|next:" .. string.gsub(tostring(head.next),"([><])","\\\%1"))
    end
    if id == node.id("style") then
     k = k + 1
     nd[n] = gv.node(g,ndlbl)     res = gv.setv(nd[n],"shape","record")
     res = gv.setv(nd[n],"label",tostring(k) .. " " .. "style" .. 
             "\|id:" .. tostring(head.id) .. 
             "\|subtype:" .. tostring(head.subtype) .. 
             "\|attr:" .. string.gsub(tostring(head.attr),"([><])","\\\%1") .. 
             "\|style:" .. tostring(head.style) .. 
             "\|prev:" .. string.gsub(tostring(head.prev),"([><])","\\\%1") .. 
             "\|next:" .. string.gsub(tostring(head.next),"([><])","\\\%1"))
    end
    if id == node.id("choice") then
     k = k + 1
     nd[n] = gv.node(g,ndlbl)     res = gv.setv(nd[n],"shape","record")
     res = gv.setv(nd[n],"label",tostring(k) .. " " .. "choice" .. 
             "\|id:" .. tostring(head.id) .. 
             "\|subtype:" .. tostring(head.subtype) .. 
             "\|attr:" .. string.gsub(tostring(head.attr),"([><])","\\\%1") .. 
             "\|display:" .. tostring(head.display) .. 
             "\|text:" .. tostring(head.text) .. 
             "\|script:" .. tostring(head.script) .. 
             "\|scriptscript:" .. tostring(head.scriptscript) .. 
             "\|next:" .. string.gsub(tostring(head.next),"([><])","\\\%1") .. 
             "\|prev:" .. string.gsub(tostring(head.prev),"([><])","\\\%1"))
    end
    if id == node.id("noad") then
     k = k + 1
     nd[n] = gv.node(g,ndlbl)     res = gv.setv(nd[n],"shape","record")
     res = gv.setv(nd[n],"label",tostring(k) .. " " .. "noad" .. 
             "\|id:" .. tostring(head.id) .. 
             "\|subtype:" .. tostring(head.subtype) .. 
             "\|attr:" .. string.gsub(tostring(head.attr),"([><])","\\\%1") .. 
             "\|nucleus:" .. tostring(head.nucleus) .. 
             "\|sub:" .. tostring(head.sub) .. 
             "\|sup:" .. tostring(head.sup) .. 
             "\|next:" .. string.gsub(tostring(head.next),"([><])","\\\%1") .. 
             "\|prev:" .. string.gsub(tostring(head.prev),"([><])","\\\%1"))
    end
    if id == node.id("op") then
     k = k + 1
     nd[n] = gv.node(g,ndlbl)     res = gv.setv(nd[n],"shape","record")
     res = gv.setv(nd[n],"label",tostring(k) .. " " .. "op" .. 
             "\|id:" .. tostring(head.id) .. 
             "\|subtype:" .. tostring(head.subtype) .. 
             "\|attr:" .. string.gsub(tostring(head.attr),"([><])","\\\%1") .. 
             "\|nucleus:" .. tostring(head.nucleus) .. 
             "\|sub:" .. tostring(head.sub) .. 
             "\|sup:" .. tostring(head.sup) .. 
             "\|next:" .. string.gsub(tostring(head.next),"([><])","\\\%1") .. 
             "\|prev:" .. string.gsub(tostring(head.prev),"([><])","\\\%1"))
    end
    if id == node.id("bin") then
     k = k + 1
     nd[n] = gv.node(g,ndlbl)     res = gv.setv(nd[n],"shape","record")
     res = gv.setv(nd[n],"label",tostring(k) .. " " .. "bin" .. 
             "\|id:" .. tostring(head.id) .. 
             "\|subtype:" .. tostring(head.subtype) .. 
             "\|attr:" .. string.gsub(tostring(head.attr),"([><])","\\\%1") .. 
             "\|nucleus:" .. tostring(head.nucleus) .. 
             "\|sub:" .. tostring(head.sub) .. 
             "\|sup:" .. tostring(head.sup) .. 
             "\|next:" .. string.gsub(tostring(head.next),"([><])","\\\%1") .. 
             "\|prev:" .. string.gsub(tostring(head.prev),"([><])","\\\%1"))
    end
    if id == node.id("rel") then
     k = k + 1
     nd[n] = gv.node(g,ndlbl)     res = gv.setv(nd[n],"shape","record")
     res = gv.setv(nd[n],"label",tostring(k) .. " " .. "rel" .. 
             "\|id:" .. tostring(head.id) .. 
             "\|subtype:" .. tostring(head.subtype) .. 
             "\|attr:" .. string.gsub(tostring(head.attr),"([><])","\\\%1") .. 
             "\|nucleus:" .. tostring(head.nucleus) .. 
             "\|sub:" .. tostring(head.sub) .. 
             "\|sup:" .. tostring(head.sup) .. 
             "\|next:" .. string.gsub(tostring(head.next),"([><])","\\\%1") .. 
             "\|prev:" .. string.gsub(tostring(head.prev),"([><])","\\\%1"))
    end
    if id == node.id("open") then
     k = k + 1
     nd[n] = gv.node(g,ndlbl)     res = gv.setv(nd[n],"shape","record")
     res = gv.setv(nd[n],"label",tostring(k) .. " " .. "open" .. 
             "\|id:" .. tostring(head.id) .. 
             "\|subtype:" .. tostring(head.subtype) .. 
             "\|attr:" .. string.gsub(tostring(head.attr),"([><])","\\\%1") .. 
             "\|nucleus:" .. tostring(head.nucleus) .. 
             "\|sub:" .. tostring(head.sub) .. 
             "\|sup:" .. tostring(head.sup) .. 
             "\|next:" .. string.gsub(tostring(head.next),"([><])","\\\%1") .. 
             "\|prev:" .. string.gsub(tostring(head.prev),"([><])","\\\%1"))
    end
    if id == node.id("close") then
     k = k + 1
     nd[n] = gv.node(g,ndlbl)     res = gv.setv(nd[n],"shape","record")
     res = gv.setv(nd[n],"label",tostring(k) .. " " .. "close" .. 
             "\|id:" .. tostring(head.id) .. 
             "\|subtype:" .. tostring(head.subtype) .. 
             "\|attr:" .. string.gsub(tostring(head.attr),"([><])","\\\%1") .. 
             "\|nucleus:" .. tostring(head.nucleus) .. 
             "\|sub:" .. tostring(head.sub) .. 
             "\|sup:" .. tostring(head.sup) .. 
             "\|next:" .. string.gsub(tostring(head.next),"([><])","\\\%1") .. 
             "\|prev:" .. string.gsub(tostring(head.prev),"([><])","\\\%1"))
    end
    if id == node.id("punct") then
     k = k + 1
     nd[n] = gv.node(g,ndlbl)     res = gv.setv(nd[n],"shape","record")
     res = gv.setv(nd[n],"label",tostring(k) .. " " .. "punct" .. 
             "\|id:" .. tostring(head.id) .. 
             "\|subtype:" .. tostring(head.subtype) .. 
             "\|attr:" .. string.gsub(tostring(head.attr),"([><])","\\\%1") .. 
             "\|nucleus:" .. tostring(head.nucleus) .. 
             "\|sub:" .. tostring(head.sub) .. 
             "\|sup:" .. tostring(head.sup) .. 
             "\|next:" .. string.gsub(tostring(head.next),"([><])","\\\%1") .. 
             "\|prev:" .. string.gsub(tostring(head.prev),"([><])","\\\%1"))
    end
    if id == node.id("inner") then
     k = k + 1
     nd[n] = gv.node(g,ndlbl)     res = gv.setv(nd[n],"shape","record")
     res = gv.setv(nd[n],"label",tostring(k) .. " " .. "inner" .. 
             "\|id:" .. tostring(head.id) .. 
             "\|subtype:" .. tostring(head.subtype) .. 
             "\|attr:" .. string.gsub(tostring(head.attr),"([><])","\\\%1") .. 
             "\|nucleus:" .. tostring(head.nucleus) .. 
             "\|sub:" .. tostring(head.sub) .. 
             "\|sup:" .. tostring(head.sup) .. 
             "\|next:" .. string.gsub(tostring(head.next),"([><])","\\\%1") .. 
             "\|prev:" .. string.gsub(tostring(head.prev),"([><])","\\\%1"))
    end
    if id == node.id("radical") then
     k = k + 1
     nd[n] = gv.node(g,ndlbl)     res = gv.setv(nd[n],"shape","record")
     res = gv.setv(nd[n],"label",tostring(k) .. " " .. "radical" .. 
             "\|id:" .. tostring(head.id) .. 
             "\|subtype:" .. tostring(head.subtype) .. 
             "\|attr:" .. string.gsub(tostring(head.attr),"([><])","\\\%1") .. 
             "\|nucleus:" .. tostring(head.nucleus) .. 
             "\|sub:" .. tostring(head.sub) .. 
             "\|sup:" .. tostring(head.sup) .. 
             "\|left:" .. tostring(head.left) .. 
             "\|degree:" .. tostring(head.degree) .. 
             "\|next:" .. string.gsub(tostring(head.next),"([><])","\\\%1") .. 
             "\|prev:" .. string.gsub(tostring(head.prev),"([><])","\\\%1"))
    end
    if id == node.id("fraction") then
     k = k + 1
     nd[n] = gv.node(g,ndlbl)     res = gv.setv(nd[n],"shape","record")
     res = gv.setv(nd[n],"label",tostring(k) .. " " .. "fraction" .. 
             "\|id:" .. tostring(head.id) .. 
             "\|subtype:" .. tostring(head.subtype) .. 
             "\|attr:" .. string.gsub(tostring(head.attr),"([><])","\\\%1") .. 
             "\|width:" .. tostring(head.width) .. 
             "\|num:" .. tostring(head.num) .. 
             "\|denom:" .. tostring(head.denom) .. 
             "\|left:" .. tostring(head.left) .. 
             "\|right:" .. tostring(head.right) .. 
             "\|next:" .. string.gsub(tostring(head.next),"([><])","\\\%1") .. 
             "\|prev:" .. string.gsub(tostring(head.prev),"([><])","\\\%1"))
    end
    if id == node.id("under") then
     k = k + 1
     nd[n] = gv.node(g,ndlbl)     res = gv.setv(nd[n],"shape","record")
     res = gv.setv(nd[n],"label",tostring(k) .. " " .. "under" .. 
             "\|id:" .. tostring(head.id) .. 
             "\|subtype:" .. tostring(head.subtype) .. 
             "\|attr:" .. string.gsub(tostring(head.attr),"([><])","\\\%1") .. 
             "\|nucleus:" .. tostring(head.nucleus) .. 
             "\|sub:" .. tostring(head.sub) .. 
             "\|sup:" .. tostring(head.sup) .. 
             "\|next:" .. string.gsub(tostring(head.next),"([><])","\\\%1") .. 
             "\|prev:" .. string.gsub(tostring(head.prev),"([><])","\\\%1"))
    end
    if id == node.id("over") then
     k = k + 1
     nd[n] = gv.node(g,ndlbl)     res = gv.setv(nd[n],"shape","record")
     res = gv.setv(nd[n],"label",tostring(k) .. " " .. "over" .. 
             "\|id:" .. tostring(head.id) .. 
             "\|subtype:" .. tostring(head.subtype) .. 
             "\|attr:" .. string.gsub(tostring(head.attr),"([><])","\\\%1") .. 
             "\|nucleus:" .. tostring(head.nucleus) .. 
             "\|sub:" .. tostring(head.sub) .. 
             "\|sup:" .. tostring(head.sup) .. 
             "\|next:" .. string.gsub(tostring(head.next),"([><])","\\\%1") .. 
             "\|prev:" .. string.gsub(tostring(head.prev),"([><])","\\\%1"))
    end
    if id == node.id("accent") then
     k = k + 1
     nd[n] = gv.node(g,ndlbl)     res = gv.setv(nd[n],"shape","record")
     res = gv.setv(nd[n],"label",tostring(k) .. " " .. "accent" .. 
             "\|id:" .. tostring(head.id) .. 
             "\|subtype:" .. tostring(head.subtype) .. 
             "\|attr:" .. string.gsub(tostring(head.attr),"([><])","\\\%1") .. 
             "\|nucleus:" .. tostring(head.nucleus) .. 
             "\|sub:" .. tostring(head.sub) .. 
             "\|sup:" .. tostring(head.sup) .. 
             "\|accent:" .. tostring(head.accent) .. 
             "\|bot_accent:" .. tostring(head.bot_accent) .. 
             "\|next:" .. string.gsub(tostring(head.next),"([><])","\\\%1") .. 
             "\|prev:" .. string.gsub(tostring(head.prev),"([><])","\\\%1"))
    end
    if id == node.id("vcenter") then
     k = k + 1
     nd[n] = gv.node(g,ndlbl)     res = gv.setv(nd[n],"shape","record")
     res = gv.setv(nd[n],"label",tostring(k) .. " " .. "vcenter" .. 
             "\|id:" .. tostring(head.id) .. 
             "\|subtype:" .. tostring(head.subtype) .. 
             "\|attr:" .. string.gsub(tostring(head.attr),"([><])","\\\%1") .. 
             "\|nucleus:" .. tostring(head.nucleus) .. 
             "\|sub:" .. tostring(head.sub) .. 
             "\|sup:" .. tostring(head.sup) .. 
             "\|next:" .. string.gsub(tostring(head.next),"([><])","\\\%1") .. 
             "\|prev:" .. string.gsub(tostring(head.prev),"([><])","\\\%1"))
    end
    if id == node.id("fence") then
     k = k + 1
     nd[n] = gv.node(g,ndlbl)     res = gv.setv(nd[n],"shape","record")
     res = gv.setv(nd[n],"label",tostring(k) .. " " .. "fence" .. 
             "\|id:" .. tostring(head.id) .. 
             "\|subtype:" .. tostring(head.subtype) .. 
             "\|attr:" .. string.gsub(tostring(head.attr),"([><])","\\\%1") .. 
             "\|delim:" .. tostring(head.delim) .. 
             "\|prev:" .. string.gsub(tostring(head.prev),"([><])","\\\%1") .. 
             "\|next:" .. string.gsub(tostring(head.next),"([><])","\\\%1"))
    end
    if id == node.id("math_char") then
     k = k + 1
     nd[n] = gv.node(g,ndlbl)     res = gv.setv(nd[n],"shape","record")
     res = gv.setv(nd[n],"label",tostring(k) .. " " .. "math_char" .. 
             "\|id:" .. tostring(head.id) .. 
             "\|subtype:" .. tostring(head.subtype) .. 
             "\|attr:" .. string.gsub(tostring(head.attr),"([><])","\\\%1") .. 
             "\|fam:" .. tostring(head.fam) .. 
             "\|char:" .. tostring(head.char) .. 
             "\|next:" .. string.gsub(tostring(head.next),"([><])","\\\%1") .. 
             "\|prev:" .. string.gsub(tostring(head.prev),"([><])","\\\%1"))
    end
    if id == node.id("sub_box") then
     k = k + 1
     nd[n] = gv.node(g,ndlbl)     res = gv.setv(nd[n],"shape","record")
     res = gv.setv(nd[n],"label",tostring(k) .. " " .. "sub_box" .. 
             "\|id:" .. tostring(head.id) .. 
             "\|subtype:" .. tostring(head.subtype) .. 
             "\|attr:" .. string.gsub(tostring(head.attr),"([><])","\\\%1") .. 
             "\|list:" .. string.gsub(tostring(head.list),"([><])","\\\%1") .. 
             "\|prev:" .. string.gsub(tostring(head.prev),"([><])","\\\%1") .. 
             "\|next:" .. string.gsub(tostring(head.next),"([><])","\\\%1"))
    end
    if id == node.id("sub_mlist") then
     k = k + 1
     nd[n] = gv.node(g,ndlbl)     res = gv.setv(nd[n],"shape","record")
     res = gv.setv(nd[n],"label",tostring(k) .. " " .. "sub_mlist" .. 
             "\|id:" .. tostring(head.id) .. 
             "\|subtype:" .. tostring(head.subtype) .. 
             "\|attr:" .. string.gsub(tostring(head.attr),"([><])","\\\%1") .. 
             "\|list:" .. string.gsub(tostring(head.list),"([><])","\\\%1") .. 
             "\|prev:" .. string.gsub(tostring(head.prev),"([><])","\\\%1") .. 
             "\|next:" .. string.gsub(tostring(head.next),"([><])","\\\%1"))
    end
    if id == node.id("math_text_char") then
     k = k + 1
     nd[n] = gv.node(g,ndlbl)     res = gv.setv(nd[n],"shape","record")
     res = gv.setv(nd[n],"label",tostring(k) .. " " .. "math_text_char" .. 
             "\|id:" .. tostring(head.id) .. 
             "\|subtype:" .. tostring(head.subtype) .. 
             "\|attr:" .. string.gsub(tostring(head.attr),"([><])","\\\%1") .. 
             "\|fam:" .. tostring(head.fam) .. 
             "\|char:" .. tostring(head.char) .. 
             "\|next:" .. string.gsub(tostring(head.next),"([><])","\\\%1") .. 
             "\|prev:" .. string.gsub(tostring(head.prev),"([><])","\\\%1"))
    end
    if id == node.id("delim") then
     k = k + 1
     nd[n] = gv.node(g,ndlbl)     res = gv.setv(nd[n],"shape","record")
     res = gv.setv(nd[n],"label",tostring(k) .. " " .. "delim" .. 
             "\|id:" .. tostring(head.id) .. 
             "\|subtype:" .. tostring(head.subtype) .. 
             "\|attr:" .. string.gsub(tostring(head.attr),"([><])","\\\%1") .. 
             "\|small_fam:" .. tostring(head.small_fam) .. 
             "\|small_char:" .. tostring(head.small_char) .. 
             "\|large_fam:" .. tostring(head.large_fam) .. 
             "\|large_char:" .. tostring(head.large_char) .. 
             "\|next:" .. string.gsub(tostring(head.next),"([><])","\\\%1") .. 
             "\|prev:" .. string.gsub(tostring(head.prev),"([><])","\\\%1"))
    end
    if id == node.id("margin_kern") then
     k = k + 1
     nd[n] = gv.node(g,ndlbl)     res = gv.setv(nd[n],"shape","record")
     res = gv.setv(nd[n],"label",tostring(k) .. " " .. "margin_kern" .. 
             "\|id:" .. tostring(head.id) .. 
             "\|subtype:" .. tostring(head.subtype) .. 
             "\|attr:" .. string.gsub(tostring(head.attr),"([><])","\\\%1") .. 
             "\|width:" .. tostring(head.width) .. 
             "\|glyph:" .. tostring(head.glyph) .. 
             "\|next:" .. string.gsub(tostring(head.next),"([><])","\\\%1") .. 
             "\|prev:" .. string.gsub(tostring(head.prev),"([><])","\\\%1"))
    end
    if id == node.id("glyph") then
     k = k + 1
     nd[n] = gv.node(g,ndlbl)     res = gv.setv(nd[n],"shape","record")
     res = gv.setv(nd[n],"label",tostring(k) .. " " .. "glyph" .. 
             "\|id:" .. tostring(head.id) .. 
             "\|subtype:" .. tostring(head.subtype) .. 
             "\|attr:" .. string.gsub(tostring(head.attr),"([><])","\\\%1") .. 
             "\|char:" .. tostring(head.char) .. 
             "\|font:" .. tostring(head.font) .. 
             "\|lang:" .. tostring(head.lang) .. 
             "\|left:" .. tostring(head.left) .. 
             "\|right:" .. tostring(head.right) .. 
             "\|uchyph:" .. tostring(head.uchyph) .. 
             "\|components:" .. string.gsub(tostring(head.components),"([><])","\\\%1") .. 
             "\|xoffset:" .. tostring(head.xoffset) .. 
             "\|yoffset:" .. tostring(head.yoffset) .. 
             "\|prev:" .. string.gsub(tostring(head.prev),"([><])","\\\%1") .. 
             "\|next:" .. string.gsub(tostring(head.next),"([><])","\\\%1"))
    end
    if id == node.id("align_record") then
     k = k + 1
     nd[n] = gv.node(g,ndlbl)     res = gv.setv(nd[n],"shape","record")
     res = gv.setv(nd[n],"label",tostring(k) .. " " .. "align_record" .. 
             "\|id:" .. tostring(head.id) .. 
             "\|subtype:" .. tostring(head.subtype) .. 
             "\|next:" .. string.gsub(tostring(head.next),"([><])","\\\%1"))
    end
    if id == node.id("pseudo_file") then
     k = k + 1
     nd[n] = gv.node(g,ndlbl)     res = gv.setv(nd[n],"shape","record")
     res = gv.setv(nd[n],"label",tostring(k) .. " " .. "pseudo_file" .. 
             "\|id:" .. tostring(head.id) .. 
             "\|subtype:" .. tostring(head.subtype) .. 
             "\|next:" .. string.gsub(tostring(head.next),"([><])","\\\%1"))
    end
    if id == node.id("pseudo_line") then
     k = k + 1
     nd[n] = gv.node(g,ndlbl)     res = gv.setv(nd[n],"shape","record")
     res = gv.setv(nd[n],"label",tostring(k) .. " " .. "pseudo_line" .. 
             "\|id:" .. tostring(head.id) .. 
             "\|subtype:" .. tostring(head.subtype) .. 
             "\|next:" .. string.gsub(tostring(head.next),"([><])","\\\%1"))
    end
    if id == node.id("page_insert") then
     k = k + 1
     nd[n] = gv.node(g,ndlbl)     res = gv.setv(nd[n],"shape","record")
     res = gv.setv(nd[n],"label",tostring(k) .. " " .. "page_insert" .. 
             "\|id:" .. tostring(head.id) .. 
             "\|subtype:" .. tostring(head.subtype) .. 
             "\|height:" .. tostring(head.height) .. 
             "\|last_ins_ptr:" .. tostring(head.last_ins_ptr) .. 
             "\|best_ins_ptr:" .. tostring(head.best_ins_ptr) .. 
             "\|next:" .. string.gsub(tostring(head.next),"([><])","\\\%1") .. 
             "\|prev:" .. string.gsub(tostring(head.prev),"([><])","\\\%1"))
    end
    if id == node.id("split_insert") then
     k = k + 1
     nd[n] = gv.node(g,ndlbl)     res = gv.setv(nd[n],"shape","record")
     res = gv.setv(nd[n],"label",tostring(k) .. " " .. "split_insert" .. 
             "\|id:" .. tostring(head.id) .. 
             "\|subtype:" .. tostring(head.subtype) .. 
             "\|height:" .. tostring(head.height) .. 
             "\|last_ins_ptr:" .. tostring(head.last_ins_ptr) .. 
             "\|best_ins_ptr:" .. tostring(head.best_ins_ptr) .. 
             "\|broken_ptr:" .. tostring(head.broken_ptr) .. 
             "\|broken_ins:" .. tostring(head.broken_ins) .. 
             "\|next:" .. string.gsub(tostring(head.next),"([><])","\\\%1") .. 
             "\|prev:" .. string.gsub(tostring(head.prev),"([><])","\\\%1"))
    end
    if id == node.id("expr_stack") then
     k = k + 1
     nd[n] = gv.node(g,ndlbl)     res = gv.setv(nd[n],"shape","record")
     res = gv.setv(nd[n],"label",tostring(k) .. " " .. "expr_stack" .. 
             "\|id:" .. tostring(head.id) .. 
             "\|subtype:" .. tostring(head.subtype) .. 
             "\|next:" .. string.gsub(tostring(head.next),"([><])","\\\%1"))
    end
    if id == node.id("nested_list") then
     k = k + 1
     nd[n] = gv.node(g,ndlbl)     res = gv.setv(nd[n],"shape","record")
     res = gv.setv(nd[n],"label",tostring(k) .. " " .. "nested_list" .. 
             "\|id:" .. tostring(head.id) .. 
             "\|subtype:" .. tostring(head.subtype) .. 
             "\|next:" .. string.gsub(tostring(head.next),"([><])","\\\%1"))
    end
    if id == node.id("span") then
     k = k + 1
     nd[n] = gv.node(g,ndlbl)     res = gv.setv(nd[n],"shape","record")
     res = gv.setv(nd[n],"label",tostring(k) .. " " .. "span" .. 
             "\|id:" .. tostring(head.id) .. 
             "\|subtype:" .. tostring(head.subtype) .. 
             "\|next:" .. string.gsub(tostring(head.next),"([><])","\\\%1"))
    end
    if id == node.id("attribute") then
     k = k + 1
     nd[n] = gv.node(g,ndlbl)     res = gv.setv(nd[n],"shape","record")
     res = gv.setv(nd[n],"label",tostring(k) .. " " .. "attribute" .. 
             "\|id:" .. tostring(head.id) .. 
             "\|subtype:" .. tostring(head.subtype) .. 
             "\|number:" .. tostring(head.number) .. 
             "\|value:" .. tostring(head.value) .. 
             "\|prev:" .. string.gsub(tostring(head.prev),"([><])","\\\%1") .. 
             "\|next:" .. string.gsub(tostring(head.next),"([><])","\\\%1"))
    end
    if id == node.id("glue_spec") then
     k = k + 1
     nd[n] = gv.node(g,ndlbl)     res = gv.setv(nd[n],"shape","record")
     res = gv.setv(nd[n],"label",tostring(k) .. " " .. "glue_spec" .. 
             "\|id:" .. tostring(head.id) .. 
             "\|subtype:" .. tostring(head.subtype) .. 
             "\|width:" .. tostring(head.width) .. 
             "\|stretch:" .. tostring(head.stretch) .. 
             "\|shrink:" .. tostring(head.shrink) .. 
             "\|stretch_order:" .. tostring(head.stretch_order) .. 
             "\|shrink_order:" .. tostring(head.shrink_order) .. 
             "\|ref_count:" .. tostring(head.ref_count) .. 
             "\|next:" .. string.gsub(tostring(head.next),"([><])","\\\%1") .. 
             "\|prev:" .. string.gsub(tostring(head.prev),"([><])","\\\%1"))
    end
    if id == node.id("attribute_list") then
     k = k + 1
     nd[n] = gv.node(g,ndlbl)     res = gv.setv(nd[n],"shape","record")
     res = gv.setv(nd[n],"label",tostring(k) .. " " .. "attribute_list" .. 
             "\|id:" .. tostring(head.id) .. 
             "\|subtype:" .. tostring(head.subtype) .. 
             "\|next:" .. string.gsub(tostring(head.next),"([><])","\\\%1") .. 
             "\|prev:" .. string.gsub(tostring(head.prev),"([><])","\\\%1"))
    end
    if id == node.id("action") then
     k = k + 1
     nd[n] = gv.node(g,ndlbl)     res = gv.setv(nd[n],"shape","record")
     res = gv.setv(nd[n],"label",tostring(k) .. " " .. "action" .. 
             "\|id:" .. tostring(head.id) .. 
             "\|subtype:" .. tostring(head.subtype) .. 
             "\|action_type:" .. tostring(head.action_type) .. 
             "\|named_id:" .. tostring(head.named_id) .. 
             "\|action_id:" .. tostring(head.action_id) .. 
             "\|file:" .. tostring(head.file) .. 
             "\|new_window:" .. tostring(head.new_window) .. 
             "\|data:" .. tostring(head.data) .. 
             "\|ref_count:" .. tostring(head.ref_count) .. 
             "\|prev:" .. string.gsub(tostring(head.prev),"([><])","\\\%1") .. 
             "\|next:" .. string.gsub(tostring(head.next),"([><])","\\\%1"))
    end
    if id == node.id("temp") then
     k = k + 1
     nd[n] = gv.node(g,ndlbl)     res = gv.setv(nd[n],"shape","record")
     res = gv.setv(nd[n],"label",tostring(k) .. " " .. "temp" .. 
             "\|id:" .. tostring(head.id) .. 
             "\|subtype:" .. tostring(head.subtype) .. 
             "\|next:" .. string.gsub(tostring(head.next),"([><])","\\\%1"))
    end
    if id == node.id("align_stack") then
     k = k + 1
     nd[n] = gv.node(g,ndlbl)     res = gv.setv(nd[n],"shape","record")
     res = gv.setv(nd[n],"label",tostring(k) .. " " .. "align_stack" .. 
             "\|id:" .. tostring(head.id) .. 
             "\|subtype:" .. tostring(head.subtype) .. 
             "\|next:" .. string.gsub(tostring(head.next),"([><])","\\\%1"))
    end
    if id == node.id("movement_stack") then
     k = k + 1
     nd[n] = gv.node(g,ndlbl)     res = gv.setv(nd[n],"shape","record")
     res = gv.setv(nd[n],"label",tostring(k) .. " " .. "movement_stack" .. 
             "\|id:" .. tostring(head.id) .. 
             "\|subtype:" .. tostring(head.subtype) .. 
             "\|next:" .. string.gsub(tostring(head.next),"([><])","\\\%1"))
    end
    if id == node.id("if_stack") then
     k = k + 1
     nd[n] = gv.node(g,ndlbl)     res = gv.setv(nd[n],"shape","record")
     res = gv.setv(nd[n],"label",tostring(k) .. " " .. "if_stack" .. 
             "\|id:" .. tostring(head.id) .. 
             "\|subtype:" .. tostring(head.subtype) .. 
             "\|next:" .. string.gsub(tostring(head.next),"([><])","\\\%1"))
    end
    if id == node.id("unhyphenated") then
     k = k + 1
     nd[n] = gv.node(g,ndlbl)     res = gv.setv(nd[n],"shape","record")
     res = gv.setv(nd[n],"label",tostring(k) .. " " .. "unhyphenated" .. 
             "\|id:" .. tostring(head.id) .. 
             "\|subtype:" .. tostring(head.subtype) .. 
             "\|next:" .. string.gsub(tostring(head.next),"([><])","\\\%1"))
    end
    if id == node.id("hyphenated") then
     k = k + 1
     nd[n] = gv.node(g,ndlbl)     res = gv.setv(nd[n],"shape","record")
     res = gv.setv(nd[n],"label",tostring(k) .. " " .. "hyphenated" .. 
             "\|id:" .. tostring(head.id) .. 
             "\|subtype:" .. tostring(head.subtype) .. 
             "\|next:" .. string.gsub(tostring(head.next),"([><])","\\\%1"))
    end
    if id == node.id("delta") then
     k = k + 1
     nd[n] = gv.node(g,ndlbl)     res = gv.setv(nd[n],"shape","record")
     res = gv.setv(nd[n],"label",tostring(k) .. " " .. "delta" .. 
             "\|id:" .. tostring(head.id) .. 
             "\|subtype:" .. tostring(head.subtype) .. 
             "\|next:" .. string.gsub(tostring(head.next),"([><])","\\\%1"))
    end
    if id == node.id("passive") then
     k = k + 1
     nd[n] = gv.node(g,ndlbl)     res = gv.setv(nd[n],"shape","record")
     res = gv.setv(nd[n],"label",tostring(k) .. " " .. "passive" .. 
             "\|id:" .. tostring(head.id) .. 
             "\|subtype:" .. tostring(head.subtype) .. 
             "\|next:" .. string.gsub(tostring(head.next),"([><])","\\\%1"))
    end
    if id == node.id("shape") then
     k = k + 1
     nd[n] = gv.node(g,ndlbl)     res = gv.setv(nd[n],"shape","record")
     res = gv.setv(nd[n],"label",tostring(k) .. " " .. "shape" .. 
             "\|id:" .. tostring(head.id) .. 
             "\|subtype:" .. tostring(head.subtype) .. 
             "\|next:" .. string.gsub(tostring(head.next),"([><])","\\\%1"))
    end
    if id == node.id("hlist") then
     k = k + 1
     nd[n] = gv.node(g,ndlbl)     res = gv.setv(nd[n],"shape","record")
     res = gv.setv(nd[n],"label",tostring(k) .. " " .. "hlist" .. 
             "\|id:" .. tostring(head.id) .. 
             "\|subtype:" .. tostring(head.subtype) .. 
             "\|attr:" .. string.gsub(tostring(head.attr),"([><])","\\\%1") .. 
             "\|width:" .. tostring(head.width) .. 
             "\|depth:" .. tostring(head.depth) .. 
             "\|height:" .. tostring(head.height) .. 
             "\|dir:" .. tostring(head.dir) .. 
             "\|shift:" .. tostring(head.shift) .. 
             "\|glue_order:" .. tostring(head.glue_order) .. 
             "\|glue_sign:" .. tostring(head.glue_sign) .. 
             "\|glue_set:" .. tostring(head.glue_set) .. 
             "\|list:" .. string.gsub(tostring(head.list),"([><])","\\\%1") .. 
             "\|prev:" .. string.gsub(tostring(head.prev),"([><])","\\\%1") .. 
             "\|next:" .. string.gsub(tostring(head.next),"([><])","\\\%1"))
    end
    if id == node.id("fake") then
     k = k + 1
     nd[n] = gv.node(g,ndlbl)
     res = gv.setv(nd[n],"shape","record")
     res = gv.setv(nd[n],"label",tostring(k) .. " " .. "fake" .. 
             "\|id:" .. tostring(head.id) .. 
             "\|subtype:" .. tostring(head.subtype) .. 
             "\|next:" .. string.gsub(tostring(head.next),"([><])","\\\%1"))
    end
        kd[k] = nd[n] 
        e1 = gv.edge(kd[k-1],kd[k])
        if id == node.id('hlist')  or id == node.id('vlist') then
           %% If we want to connect nested (h|v)list
           %% 
           %%e = gv.edge(nd[n-1],nd[n])
           %%gv.setv(e,'arrowhead','diamond')
           nodesprint(head.list,( n or 0) +1)
        end
        head = head.next
    end
end

local head  = tex.box[#1]
nodesprint(head,0)
r = gv.layout(g,"dot")
r = gv.render(g,'pdf',string.format('box\%d.pdf',#1))
\stopluacode%%
}

\starttext
\startTEXpage
\setbox0=\hbox{\TeX}
\hbox to 29cm{\strut\hss\copy0\hss}
\StudyBox{0}%
\externalfigure[box0.pdf][width=29cm]
\stopTEXpage
\stoptext

Graphviz.png

R

R is a language and environment for statistical computing and graphics (see here) .
RPy is a very simple, yet robust, Python interface to the R Programming Language (see here) .
As example, let's try to plot a discrete distribution of probability for a set of pseudorandom number (around 100000 samples) .

import rpy2.robjects as robjects
import rpy2.rinterface as rinterface

class density(object):

   def __init__(self,samples,outpdf,w,h,kernel):
     self.samples = samples
     self.outpdf= outpdf
     self.kernel = kernel
     self.width=w
     self.height=h

   def run(self):
        r = robjects.r
        data = [int(k.strip()) for k in file(self.samples,'r').readlines()]
        x = robjects.IntVector(data)
        r.pdf(file=self.outpdf,width=self.width,height=self.height)
        z = r.density(x,kernel=self.kernel)
        #r.plot(z[0],z[1],xlab='',ylab='',xlim=robjects.IntVector([0,2**16-1]))
        r.plot(z[0],z[1],xlab='',ylab='')
        r['dev.off']()


if __name__ == '__main__' :
   dens = density('u-random-int','test-001.pdf',10,7,'o')
   dens.run()


\startluacode
function testR(samples,outpdf,w,h,kernel)
  require("python")
  testR  = python.import("test-R") 
  dens = testR.density(samples,outpdf,w,h,kernel)
  dens.run()
end
\stopluacode

\def\plotdenstiy[#1]{%
\getparameters[R][#1]%
\expanded{\ctxlua{testR("\Rsamples","\Routpdf",\Rwidth,\Rheight,"\Rkernel")}}%
}

\setupbodyfont[sans,10pt]
\starttext
\startTEXpage
\plotdenstiy[samples={u-random-int},outpdf={test-001.pdf},width={10},height={7},kernel={o}]
\setupcombinations[location=top]
\startcombination[1*2]
{\vbox{\hsize=400bp
This is a density plot of around {\tt 100 000} random numbers  between $0$ and $2^{16}-1$ generated from {\tt \hbox{/dev/urandom}}}}{}
{\externalfigure[test-001.pdf][width={400bp}]}{}
\stopcombination
\stopTEXpage
\stoptext

And here is the plot

Test-R.png

dbxml

From site (see here) :

Oracle Berkeley DB XML is an open source, embeddable XML database with XQuery-based access to documents stored in containers and indexed based on their content. Oracle Berkeley DB XML is built on top of Oracle Berkeley DB and inherits its rich features and attributes. Like Oracle Berkeley DB, it runs in process with the application with no need for human administration. Oracle Berkeley DB XML adds a document parser, XML indexer and XQuery engine on top of Oracle Berkeley DB to enable the fastest, most efficient retrieval of data.

As test, we can use a dump from wikiversity (see here) .


Build the cointainer

First we build the container 'Data.dbxml' in the directory "wikienv" (that must exists) :

"""
---
"""
from bsddb3.db import *
from dbxml import *
import sys
import re
import time

def createEnvironment(home):
    """Create DBEnv and initialize XmlManager"""
    try:
        environment = DBEnv()
        #
        environment.set_cachesize(0,512 * 1024 *1024,1)
        environment.set_lk_max_lockers(10000)
        environment.set_lk_max_locks(10000)
        environment.set_lk_max_objects(10000)
        # initialize DBEnv for transactions
        environment.open(home, DB_RECOVER|DB_CREATE|DB_INIT_LOCK|DB_DSYNC_LOG|
                         DB_INIT_LOG|DB_INIT_MPOOL|DB_INIT_TXN, 0)
    except DBError, exc:
        print exc
        sys.exit()
    try:
        mgr = XmlManager(environment, 0)
        mgr.setDefaultPageSize(4096)
    except XmlException, se:
        print xe
        sys.exit()
    return mgr

def createContainer(mgr, containerName, flags):
    """create/open a node container"""
    
    try:
        uc = mgr.createUpdateContext()
        container =  mgr.openContainer(containerName,
                                       flags|DB_CREATE,
                                       XmlContainer.WholedocContainer)
        container.addIndex("","title","edge-element-substring-string",uc)
        container.addIndex("","username","edge-element-substring-string",uc)
        container.addIndex("","text","edge-element-substring-string",uc)
        return container
    except XmlException, ex:
        print ex
        sys.exit()


def loadcontent(mgr, container,content,printmsg,k):
    """ -- """
    id= re.compile(r"<id>(.*)</id>")
    title = re.compile(r"<title>(.*)</title>",re.MULTILINE|re.DOTALL)

    id_text = id.search(content,re.MULTILINE|re.DOTALL).group(1)
    title_text = title.search(content).group(1)
    docName = '_'.join(title_text.split()) + '_' +id_text
    txn = False
    try:
        # all Container modification operations need XmlUpdateContext
        uc = mgr.createUpdateContext()
        # create XmlTransaction for the operation
        txn = mgr.createTransaction()
        # use the DBXML_GEN_NAME flag to make sure this
        # succeeds by creating a new, unique name
        # Use a try/except block to allow the transaction to
        # be aborted in the proper scope upon error
        try:
            docName = container.putDocument(txn, docName, 
                                       content, uc, 
				       DBXML_GEN_NAME)
            txn.commit()
        except XmlException, ex:
            print k,ex
            txn.abort()
        if printmsg:
            # now, get the document in a new transaction
            txn = mgr.createTransaction()
            doc = container.getDocument(txn, docName)
            name = doc.getName()
            docContent = doc.getContentAsString()
            txn.commit() # done with data
            # print the name and content
            print name
            pass  
    except XmlException, inst:
        print inst
        if txn:
            txn.abort()

# "main"
def main():
    home = "wikienv"
    # some configuration...
    containerName = "Data.dbxml"
    # initialize...
    
    mgr = createEnvironment(home)
    # create/open a transactional container
    container = createContainer(mgr, containerName, 
                           DBXML_TRANSACTIONAL)

    startpage = re.compile(r"^\s*<page>\s*$")
    endpage = re.compile(r"^\s*</page>\s*$")
    id= re.compile(r"<id>(.*)</id>")
    title = re.compile(r"<title>(.*)</title>",re.MULTILINE|re.DOTALL)
    text = re.compile(r"<text ([^>]*)>(.*)</text>",re.MULTILINE|re.DOTALL)
    k,k1,k2 = 0,0,0
    startcollect = False
    #src = file("enwiki-latest-pages-articles.xml","rb")
    src = file("enwikiversity-20090627-pages-articles.xml",'rb')
    for  line in src:
        try: 
         k1 = k1 +1 
	except:
	 k1 = 0 
        if divmod(k1,10000)[0]>0 and divmod(k1,10000)[1] == 0  : 
          print "k1=%012d,k=%012d ,sleep 1 sec." % (k1,k)
	  #time.sleep(1)
        if startcollect and endpage.match(line) is None: 
            temp = ''.join((temp,line))
            continue
        if startpage.match(line) is not None: 
            temp = line
            startcollect = True
            pos = src.tell()
            continue
        if endpage.match(line) is not None: 
            content  = ''.join((temp,line))
            startcollect = False
            if title.search(content) is not None and  id.search(content) is not None:
                #title_text = title.search(temp).group(1)
                #id_text , content_len = id.search(temp,re.MULTILINE|re.DOTALL).group(1), len(temp) 
                #text_text = ((text.search(temp) is not None and text.search(temp).group(2)) or '' )+ ' ' + title_text
                #keywords = [kk.lower() for kk in re.split("\W",text_text) 
                #            if len(kk) >4 and kk.lower() != 'redirect' 
                #                and kk.lower() != 'disambiguation' ]
                #keywords.append(title_text) 
                #keywords = list(set(keywords))
                #keywords.sort()
                printmesg = False
                if divmod(k,100)[1] == 0 and divmod(k,100)[0] >0: 
                    print "%012d sync" %k
                    container.sync()
                    #del container 
                    #container =  mgr.openContainer(containerName,DBXML_TRANSACTIONAL)
                #if divmod(k,1200)[1] == 0 and divmod(k,1200)[0] == 1:
                    #print k,title_text,id_text ,pos,content_len,keywords#,temp
                    #printmesg = True
                    #print '%09d insert data...'%k, 
                    #return
                loadcontent(mgr,container,content,printmesg,k)
            k = k+1 
    src.close()

if __name__ == "__main__":
    main()

Make pdf

We use this modules wikidbxml_queryTxn.py to retrive a page, given a title (it can be also used as basis to build more complex queries, but for now it's adeguate ):

from  lxml import etree
from bsddb3.db import *
from dbxml import *
import StringIO
import fcntl
import os
import pprint
import sys
import time
import mwlib.docbookwriter
from mwlib.dummydb import DummyDB
from mwlib.uparser import parseString


def getXML(title,res):
    db = DummyDB()
    r = parseString(title=title, raw=res, wikidb=db)
    dbw = mwlib.docbookwriter.DocBookWriter()
    dbw.writeBook(r)
    pprint.pprint( dbw.getTree() )
    return dbw.asstring()


def getConTeXt(title,res):
    db = DummyDB()
    r = parseString(title=title, raw=res, wikidb=db)
    dbw = mwlib.docbookwriter.DocBookWriter()
    dbw.writeBook(r)
    article =  dbw.getTree()
    res = []

    def managepara(c,res):
        if c.tag == 'para' and (c.text is not None):
            res.append(c.text.strip()+r"\par")
        if c.tag == 'para' and (len(c.getchildren())>0):
            for c1 in c.iterchildren('ulink'):
                res.append(r'cfr~\type{%s}\par' %c1.get('url'))


    def managetable(c,res):
        if c.tag == 'informaltable' :
            res.append(r'\bTABLE')
            for row in c.iterchildren():
                res.append(r'\bTR')
                for col in row.iterchildren():
                    res.append(r'\bTD '+ col.text.strip()+ r'\eTD')
                res.append(r'\eTR')
            res.append(r'\eTABLE')

    def section_content(c,res):
        managepara(c,res)
        if c.tag == 'section' :
            subsection = c
            subsection_title = subsection.find("sectioninfo").find("title").text.strip()
            res.append(r"\subsection{%s}" % subsection_title)
            for sc in subsection.iterchildren():
                subsection_content(sc,res)

    def subsection_content(c,res):
        managepara(c,res)
        if c.tag == 'section' :
            subsubsection = c
            subsubsection_title = subsubsection.find("sectioninfo").find("title").text.strip()
            res.append(r"\subsubsection{%s}" % subsubsection_title)
            for sc in subsubsection.iterchildren():
                subsubsection_content(sc,res)
        if c.tag == 'informaltable' :
            managetable(c,res)


    def subsubsection_content(c,res):
        managepara(c,res)


    chapter_title = article.find("articleinfo").find("title").text
    res.append(r"\chapter{%s}" % chapter_title)

    section = article.find("section")
    section_title = section.find("sectioninfo").find("title").text
    res.append(r"\section{%s}" % section_title)

    for c in section.iterchildren():
        section_content(c,res)
    
    #pprint.pprint(res)
    return '\n'.join(res)





def query(env=None,mgr=None,container=None,querystring='Foo'):
    """ Always check with queryPlan
        (for example matches is not optimized for indexes)
    """
    anID = env.lock_id()
    lock = env.lock_get(anID, "shared lock", DB_LOCK_READ)
    updateContext = mgr.createUpdateContext();
    try:
        txn = mgr.createTransaction()
        resultsContext = mgr.createQueryContext()
        #queryString = "collection('%s')/page[contains(title,'%s')]" % (container.getName(),data)
        #queryString = "collection('%s')%s" % (container.getName(),querystring)
        results = mgr.query(txn, querystring, resultsContext)
        res = [res.asString() for res in results]
        txn.commit()
        return res 
        #print "START",book_name
        ##
        ##
    except XmlException, inst:
        txn.abort()
	print "XmlException (", inst.exceptionCode,"): ", inst.what#,'name=',theName
	if inst.exceptionCode == DATABASE_ERROR:
	    print "Database error code:",inst.dbError
    env.lock_put(lock)
    env.lock_id_free(anID)
    print 'OK exit'

def getArtitleByTitle(title):
    pass
    env = DBEnv()
    env.set_cachesize(0, 64 * 1024 * 1024, 1)
    path2DbEnv ='wikienv'
    env.open(path2DbEnv,DB_THREAD|DB_REGISTER|DB_RECOVER|DB_INIT_MPOOL|DB_CREATE|DB_INIT_LOCK|DB_INIT_LOG|DB_INIT_TXN, 0)
    mgr = XmlManager(env,0)
    containerTxn = mgr.createTransaction()
    theContainer = "Data.dbxml"
    container = mgr.openContainer(containerTxn, theContainer)
    containerTxn.commit()
    ##
    lockfile = open("lock.kmgr", "w")
    fcntl.flock(lockfile, fcntl.LOCK_EX)
    try:
        res = set()
        querystring = 'collection("%s")/page[contains(title,"%s")]/revision/text/text()'  % (theContainer,title)
        res = res.union(query(env,mgr,container,querystring=querystring))
        res = ''.join(list(res)).decode('utf8')
        #res = getXML(title,res)
        #open('res.dbk','w').write( " ".join(res.split()) )
        res = getConTeXt(title,res)
        return res 
    except Exception,e:
        print "error on read:" ,e
    fcntl.flock(lockfile, fcntl.LOCK_UN)
    lockfile.close()

def writeres(title,preamble,postamble,filename):

    res = getArtitleByTitle(title=title)
    if res is not None :
        res = res.replace('&',r'\&')
        res = res.replace('#',r'\#')
    else:
        res = ''  
    open(filename,'wb').write( '\n'.join((preamble,res,postamble)) )

    pass

if __name__ == '__main__':

    preamble = r"""\usetypescriptfile[type-gentium]
\usetypescript[gentium]
\setupbodyfont[gentium,10pt]
\setuppapersize[A5][A5]
\setuplayout[height=middle,topspace=1cm,header={2\lineheight},footer=0pt,backspace=1cm,margin=1cm, width=middle]
\starttext"""

    postamble = r"""\stoptext"""


    title="Primary mathematics/Numbers"
    filename = 'res.tex'
    
    writeres(title,preamble,postamble,filename)

And in the end mkiv wrapper:

\usetypescriptfile[type-gentium]
\usetypescript[gentium]
\setupbodyfont[gentium,10pt]
\setuppapersize[A5][A5]
\setuplayout[height=middle,topspace=1cm,header={2\lineheight},footer=0pt,backspace=1cm,margin=1cm, width=middle]


\startluacode
function testdbxml(title,preamble,postamble,filename)
  require("python")
  pg = python.globals()
  wikiversity = python.import("wikidbxml_queryTxn")
  wikiversity.writeres(title,preamble,postamble,filename)  
end
\stopluacode

\def\testdbxml[#1]{%
\getparameters[dbxml][#1]%
\ctxlua{testdbxml("\csname dbxmltitle\endcsname","\csname dbxmlpreamble\endcsname",
                  "\csname dbxmlpostamble\endcsname","\csname dbxmlfilename\endcsname")}%
\input \csname dbxmlfilename\endcsname %
}



\starttext
\testdbxml[title={Primary mathematics/Numbers},
           preamble={},
           postamble={},               
           filename={testres.tex}]
\stoptext

Here here the result:

Dbxml-1.png
Dbxml-2.png Dbxml-3.png

One can also use sqlite that comes with python to query for titles category.db made from (for example) enwikiversity-20090627-category.sql, so reports are more simpler: just put this in python code above, right before 'if __name__' ...:

import sqlite3
def querycategory(title):
    conn = sqlite3.connect('category.db')
    c = conn.cursor()
    t = (title,)
    c.execute('select cat_title from category where cat_title like "%%%s%%" ;' % t)
    res = [row[0] for row in c]
    conn.commit()
    c.close()
    return res


def simplereports(title):
    res = querycategory(title)
    j = 0
    for r in res:
        g = r.replace('_',' ')
        print g
        title= g.encode('utf8')
        filename = 'reps%04d.tex' % j
        writeres(title,'','',filename)
        j = j+1
    return j

Similary, add this to tex code

\startluacode
function listtitles(title)
  require("python")
  pg = python.globals()
  wikiversity = python.import("wikidbxml_queryTxn")
  r = wikiversity.querycategory(title)  
  local j = 0
  local res = r[j] or {}
  while res do
   local d = string.format("\%s\\par",string.gsub(tostring(res),'_',' '))
   tex.sprint(tex.ctxcatcodes,d)
   j = j+1
   res = r[j]
  end
end
\stopluacode


\startluacode
function simplereports(title)
  require("python")
  pg = python.globals()
  wikiversity = python.import("wikidbxml_queryTxn")
  r = wikiversity.simplereports(title)  
  local j = tonumber(r)
  for v = 0,j-1 do
   local d = string.format("\\input reps\%04d ",v)
   tex.sprint(tex.ctxcatcodes,d)
  end   
  print( j )
end
\stopluacode

and test it with

\starttext
{\bfb Query for  'geometr':}
\ctxlua{listtitles("geometr")}%
\ctxlua{simplereports("geometr")}%
\stoptext

(query results are stored in reps0001.tex ,reps0002.tex ,..and so on.)