User:Luigi.scarso/luatex lunatic

Introduction

---

!! W A R N I N G !!

---

!! THIS CODE IS HIGHLY EXPERIMENTAL !!

---

!! THIS CODE WORKS ONLY UNDER LINUX !!

---

luatex_lunatic is a modification of lua side of luatex to host a python interpreter inside lua (see lunatic-python) .

I have made a set of patches because :

• by design, lua in luatex doesn't permit dynamic loading ("Dynamic loading of .so and .dll files is disabled on all platforms." see "LUA changes" in manual/luatexref-t.pdf of src and/or svn dist.)

• to prevent symbols collisions between luatex and an arbitrary library

Having a python interpreter hosted in luatex can make easy to use existing python's bindings (also Python releases includes at least from 2.5 a ctypes module that permit a binding to a .so without using SWIG or similar, similar to lua's loadlib) .

As general rule, I want the smallest set of patches ; so, for example, I have choose to not use a system libpng, even if it's easy to modify build.sh.linux to do so.

Also, I'm not concerned about portability: so I will talk only about Linux .

Please note that luatex_lunatic was born to answer to (my) question "Can I apply lunatic to luatex ?" and nothing else.

Every others meanings can be interestring, but are not mine , and I will not partecipate to any discussion about that.

How to

I prefear to put a bash script that can help to compile a luatex-lunatic binary. I have put all patches files here It's clear that some skills are needed, so I hope that this is barrier to avoid useless questions . Of course, I will try to correct all errors .

The trick is "hide all, then unhide what is needed" : in build.sh.linux change

CFLAGS="-g -O2 -Wno-write-strings"
CXXFLAGS=$CFLAGS

in

CFLAGS="-g -O2 -Wno-write-strings -fvisibility=hidden"
CXXFLAGS="$CFLAGS -fvisibility-inlines-hidden"

and build all. After that, make only what is needed (lua51) and re-compile luatex .

##
## Sec. 0
## Safe exit 
## uncomment exit to go on
exit

##
##
## Sec. 1
## Setup
## 
##
if [! -d Python-2.6.1 ];
then
$HOME_LUN="/opt/luatex/luatex-lunatic"
svn checkout svn://scm.foundry.supelec.fr/svn/luatex/tags/beta-0.35.0
#svn checkout svn://scm.foundry.supelec.fr/svn/luatex/trunk
bzr branch lp:lunatic-python
wget http://www.python.org/ftp/python/2.6.1/Python-2.6.1.tar.bz2
tar -xjvf Python-2.6.1.tar.bz2 
cd  Python-2.6.1
./configure --prefix=$HOME_LUN --enable-unicode=ucs4 --enable-shared
make && make install
cp python2.6.conf /etc/ld.so.conf.d
ldconfig ;
fi
##
## install setuptools
if [ ! -x bin/easy_install ];
then
echo "Please install setuptools"
echo "from  http://pypi.python.org/pypi/setuptools"
echo "in $HOME_LUN" 
echo "Please, pay attention to use $HOME_LUN/bin/python , not system python !" 
echo "Eventually modify first lines of setuptools to accomodate this,"
echo "or modify \$PATH=$HOME_LUN/bin:\$PATH" 
exit;
fi
##
##
## Sec 2
## Patch and build luatex
##
##
cd $HOME_LUN
#ln -s trunk luatex
##
## build lunatic
##
ln -s beta-0.35.0 luatex
patch -N --backup <0000_lunatic-python_setup.py.patch  lunatic-python/setup.py
cd lunatic-python
../bin/python setup.py build
../bin/python setup.py install --root=/ --prefix=$HOME_LUN --install-script=$HOME_LUN/bin
cd $HOME_LUN
##
## This patch set 
## CFLAGS="-g -O2 -Wno-write-strings -fvisibility=hidden"
## to avoid symbols conflicts
patch -N --backup <0500_build.sh.linux.patch luatex/build.sh.linux
##
## These patches 
## permits dynamic loading of shared libs.
##
patch -N --backup <1000_luatex_src_libs_lua51_loadlib.c.patch luatex/src/libs/lua51/loadlib.c
patch -N --backup <2000_luatex_src_libs_lua51_Makefile.patch  luatex/src/libs/lua51/Makefile
patch -N --backup <3000_luatex_src_texk_web2c_configure.patch luatex/src/texk/web2c/configure
##
## Build luatex
##
cd luatex
./build.sh.linux &> out 
cd $HOME_LUN
mkdir tests
##
## "Unhide" visibility of lua51
##
cd luatex-lunatic/luatex/build/libs/lua51
make -B posix
##
## Re-compile luatex
##
cd $HOME_LUN
cd luatex-lunatic/luatex/build/texk/web2c
g++ -o luatex  luatexini.o luatex0.o luatex1.o luatex2.o luatex3.o luatexextra.o luatex-pool.o   luatexdir/libpdf.a ../../libs/libpng/libpng.a ../../libs/zlib/libz.a ../../libs/xpdf/xpdf/libxpdf.a ../../libs/xpdf/goo/libGoo.a ../../libs/xpdf/fofi/libfofi.a ../../libs/md5/md5.o ../../libs/obsdcompat/libopenbsd-compat.a ../../libs/lua51/liblua.a ../../libs/slnunicode/slnunico.o  ../../libs/luazip/src/luazip.o ../../libs/zziplib/zzip/libzzip.a ../../libs/luafilesystem/src/lfs.o ../../libs/luasocket/src/socket.a ../../libs/luapeg/lpeg.o ../../libs/luamd5/md5lib.o ../../libs/luamd5/md5.o  ../../libs/luazlib/lgzip.o ../../libs/luazlib/lzlib.o ../../libs/luafontforge/libff.a ../../libs/luaprofiler/libprofiler.a mpdir/lmplib.o mpdir/.libs/libmplib.a   lib/lib.a ../kpathsea/.libs/libkpathsea.a -lm   -nodefaultlibs -Wl,-Bstatic -lstdc++ -Wl,-Bdynamic -ldl -lm -lgcc_eh -lgcc -lc -lgcc_eh -lgcc -Wl,-E -ldl -lreadline -lhistory -lncurses
##
##

The last line is:

g++ -o luatex luatexini.o luatex0.o luatex1.o luatex2.o luatex3.o luatexextra.o luatex-pool.o luatexdir/libpdf.a ../../libs/libpng/libpng.a ../../libs/zlib/libz.a ../../libs/xpdf/xpdf/libxpdf.a ../../libs/xpdf/goo/libGoo.a ../../libs/xpdf/fofi/libfofi.a ../../libs/md5/md5.o ../../libs/obsdcompat/libopenbsd-compat.a ../../libs/lua51/liblua.a ../../libs/slnunicode/slnunico.o ../../libs/luazip/src/luazip.o ../../libs/zziplib/zzip/libzzip.a ../../libs/luafilesystem/src/lfs.o ../../libs/luasocket/src/socket.a ../../libs/luapeg/lpeg.o ../../libs/luamd5/md5lib.o ../../libs/luamd5/md5.o ../../libs/luazlib/lgzip.o ../../libs/luazlib/lzlib.o ../../libs/luafontforge/libff.a ../../libs/luaprofiler/libprofiler.a mpdir/lmplib.o mpdir/.libs/libmplib.a lib/lib.a ../kpathsea/.libs/libkpathsea.a -lm -nodefaultlibs -Wl,-Bstatic -lstdc++ -Wl,-Bdynamic -ldl -lm -lgcc_eh -lgcc -lc -lgcc_eh -lgcc -Wl,-E -ldl -lreadline -lhistory -lncurses

Python packages

• numpy ✔
• scipy ✔
• matplot ✔
• odfpy ✔
• PIL, python imaging library ✔ PIL
• TO FIX ; pygegl (I like its syntax, but need too much and it's easy only in python)
• TODO: binding of libtiff with ctypes (or use gdal ?)
• TODO :gle (for Massimiliano "Max" Dominici, GUIT)
• TODO : using the binding of VIPS

Bindings

• ghostscript 8.64 ✔
• graphviz 2.22.0 ✔
• ImageMagick-6.4.9 with pythonmagickwand ✔
• fontforge 20090224 ✔ Useful to check symbols collision, and if one want to play with the last fontforge, eg to draw the outline of a glyph .
• R-2.8.1 with rpy2-2.0.3 (For Maurizio "Mau" Himmelman , GUIT) ✔ need an example with output in pdf
• quantalib 0.9.7 ✔ need an example with output in pdf
• dbxml-2.4.16 ✔

Dedicated systems

SageMath

Given that sagemath is rooted on a CPtyhon , we can try to "use sagemath as luatex plugin" instead of "use latex as external process of sagemath".

I don't want to mix with luatex-lunatic, so I put all stuffs under lunatic-python-SAGEMATH .

$HOME_LUN=/opt/luatex/luatex-lunatic
cd $HOME_LUN
cp -r lunatic-python lunatic-python-SAGEMATH
cd lunatic-python-SAGEMATH
##
## We prepare new bridge
##
rm -r build
$HOME_LUN/sage/local/bin/python setup.py build
cd $HOME_LUN
mkdir tests-SAGEMATH
##
## I have already installed prev. python.so, I don't want mess things
##
ln -s ../lunatic-python-SAGEMATH/build/lib.linux-i686-2.5/python.so
ln -s  ../luatex/build/texk/web2c/luatex
##
## now we need to setup sagemath
##
cd $HOME_LUA/sage/local/bin
. sage-env
cd $HOME_LUA

We need also a first run of sagemath, if you never run it:

cd sage
./sage
##
## exit from sage
##

That is .

Now a simple test:

function test_SAGEMATH()
  require("python")
  python.execute("from sage.all_cmdline import *")
  dir = python.eval("dir()")
  print(dir)    
end
test_SAGEMATH()

$>luatex test-SAGEMATH.lua

You should see something like this:

['AA', 'AbelianGroup', 'AbelianGroupElement', 'AbelianGroupMap', 'AbelianGroupMorphism', 'AbelianGroupMorphism_id', 'AbelianGroup_class', 'AbelianGroup_subgroup', 'AbelianGroups', 'AbelianMonoids', 'AbelianSemigroups', 'AbelianVariety', 'AffineGeometryDesign', 'AffineSpace', 'Algebra', 'AlgebraElement', 'AlgebraIdeals', 'AlgebraModules', 'AlgebraicField', 'AlgebraicNumber', 'AlgebraicReal', 'AlgebraicRealField', 'Algebras', 'AllCusps', 'AllExactCovers', 'Alphabet', 'AlphabeticStrings', 'AlternatingGroup', 'AlternatingSignMatrices', 'AntichainPoset', 'Arrangements', 'AtkinModularCorrespondenceDatabase', 'AtkinModularPolynomialDatabase', 'AugmentedLatticeDiagramFilling', 'Axiom', 'BCHCode', 'Bessel', 'BinaryGolayCode', 'BinaryQF', 'BinaryQF_reduced_representatives', 'BinaryReedMullerCode', 'BinaryStrings', 'BinaryTree', 'BipartiteGraph', 'BlockDesign', 'BlockDesign_generic', 'BooleanLattice', 'BooleanPolynomialRing', 'CC', 'CDF', 'CFF', 'CIF'

.....

true', 'ttest', 'tuples', 'twinprime', 'twisted', 'two_squares', 'ultraspherical', 'uniform', 'union', 'uniq', 'unordered_tuples', 'unpickle_function', 'unset_verbose_files', 'unsigned_infinity', 'upgrade', 'valuation', 'var', 'vecsmall_to_intlist', 'vector', 'verbose', 'version', 'vert_to_ieq', 'victor_miller_basis', 'view', 'volume_hamming', 'vonmisesvariate', 'walltime', 'walsh_matrix', 'wave', 'weakref', 'weibullvariate', 'wiki', 'wiki_create_instance', 'with_statement', 'word_problem', 'words', 'wrapper', 'wronskian', 'x', 'xgcd', 'xinterval', 'xlcm', 'xmrange', 'xmrange_iter', 'xsrange', 'zero', 'zero_matrix', 'zeta', 'zeta_symmetric', 'zeta_zeros']

Root (CERN) (Maybe)

See here and here

ConTeXt mkIV examples

Here I will collect some tex snippets, just to show some ideas.

Python Imaging Library (PIL)

\startluacode
function testPIL(imageorig,imagesepia)
  require("python")
  PIL_Image = python.import("PIL.Image") 
  PIL_ImageOps = python.import("PIL.ImageOps") 
  python.execute([[
def make_linear_ramp(white):
    ramp = []
    r, g, b = white
    for i in range(255):
        ramp.extend((r*i/255, g*i/255, b*i/255))
    return ramp
]])
        
    -- make sepia ramp (tweak color as necessary)
    sepia = python.eval("make_linear_ramp((255, 240, 192))") 
    im = PIL_Image.open(imageorig)
    
    -- convert to grayscale
    if not(im.mode == "L")
     then 
        im = im.convert("L")
    end
    
    --  optional: apply contrast enhancement here, e.g.
    im = PIL_ImageOps.autocontrast(im)

    --  apply sepia palette
    im.putpalette(sepia)

    --  convert back to RGB so we can save it as JPEG
    --  (alternatively, save it in PNG or similar)
    im = im.convert("RGB")

    im.save(imagesepia)
end
\stopluacode

\def\SepiaImage#1#2{%
\ctxlua{testPIL("#1","#2")}%
\startcombination[2*1]
{\externalfigure[#1]}{\ss Orig.}
{\externalfigure[#2]}{\ss Sepia}
\stopcombination
}


\starttext
\startTEXpage
\SepiaImage{lena.jpg}{lena-sepia.jpg}
\stopTEXpage
\stoptext