Difference between revisions of "Running Mark IV"

From ConTeXt wiki
(preliminary content (more to come))
 
(more text (still not done))
Line 11: Line 11:
 
is the replacement for [[mktexlsr]]
 
is the replacement for [[mktexlsr]]
  
   context --make --compile
+
   context --make
  
 
is the replacement for [[fmtutil]]
 
is the replacement for [[fmtutil]]
 
  mtxrun --script fonts --reload
 
 
is needed if you want to use fonts by font name (as opposed to file names)
 
  
  
Line 23: Line 19:
  
 
=== Introduction ===
 
=== Introduction ===
 +
 
When you run
 
When you run
  
Line 29: Line 26:
 
both the script 'context' and the Mark IV code inside the format have to find various files. To make this happen as fast as possible, a file name database is used. There is also some configuration information and some generated font data is cached because extracting the OpenType information is a relatively lengthy and memory-consuming process.
 
both the script 'context' and the Mark IV code inside the format have to find various files. To make this happen as fast as possible, a file name database is used. There is also some configuration information and some generated font data is cached because extracting the OpenType information is a relatively lengthy and memory-consuming process.
  
All of this data stored inside the 'texmf cache'.
+
All of this data stored inside the 'texmf cache'. To keep this cache up to date, you have to (re)run
 +
 
 +
  luatools --generate
 +
 
 +
whenever you install or remove files from your tex installation that have to be found by Mark IV, like after updating context itself, or installing extra font packages from [[TeX Live]] or [[CTAN]]. System-wide fonts don't count, these are always re-discovered at runtime. {{todo| check that}}
  
 
=== Cache location ===
 
=== Cache location ===
Line 41: Line 42:
 
But there is a catch: luatools has to decide where on the harddisk the file name database should be stored. This needs extra care, because a runtime command like 'context myfile' does not look at texmf.cnf.  
 
But there is a catch: luatools has to decide where on the harddisk the file name database should be stored. This needs extra care, because a runtime command like 'context myfile' does not look at texmf.cnf.  
  
The logic used for finding the cache location is as follows:
+
The internal logic used for finding the cache location is as follows:
  
 
* If there is an environment variable named TEXMFCACHE, this will point to the desired harddisk location
 
* If there is an environment variable named TEXMFCACHE, this will point to the desired harddisk location
* Otherwise, if the web2c configuration directory contains a file named texmfcnf.lua, this file is read and used to discover the desired location (see below)
+
* Otherwise, if the web2c configuration directory contains a file named texmfcnf.lua, this file is read and used to discover the desired location. '''this is the recommended approach, see below'''
* If that doesn't work either, a list of 'likely' environment variables is tested, and the first variable that is set is assumed to point to the correct location. This list is currently:
+
* If that doesn't work either, a list of 'likely' environment variables is tested, and the first variable that is set is assumed to point to the correct location.  
 +
* As a last resort, the local directory is used.
 +
 
 +
 
 +
For completeness: the list of tested environment variables is currently:
  
 
   TMPDIR, TEMPDIR, TMP, TEMP, HOME, HOMEPATH
 
   TMPDIR, TEMPDIR, TMP, TEMP, HOME, HOMEPATH
  
but there have been variants forms
+
but there have been quite some variations on this in the past.
 +
 
 +
=== texmfcnf.lua ===
 +
 
 +
Files named texmfcnf.lua can appear anywhere on the search path where there can also be a [[texmf.cnf]] present. Multiple files can be present. They are read sequentially using the same logic as for [[texmf.ncf]]: earlier files fix values that cannot be overruled by later ones. All texmfcnf.lua files are read before all texmf.cnf files, and values defined in texmfcnf.lua cannot be overruled by texmf.cnf files.
 +
 
 +
The format of texmfcnf.lua is pretty simple: it is a Lua script that has to return a single table with keys for the variables it wants to define. Mark IV will interpret specific keys in the table and do something special (unknown keys are ignored).
 +
 
 +
Besides the 'normal' texmf.cnf keys, the most important key is TEXMFCACHE. Here is a simple one-line example texmfcnf.lua to set up the texmf cache to be inside your private texlive 2008 directory:
 +
 
 +
  return { TEXMFCACHE =  '$HOME/.texlive2008/texmf-var' }
 +
 
 +
Note: Windows users should replace $HOME with $USERPROFILE. Both unix and windows users of [[TeX Live]] 2008 and users of Mark IV beta's released after Jul 10, 2008 can use ~ instead.
 +
 
 +
== Generating and updating formats ==
 +
 
 +
Before first use and after each update to either the [[context]] distribution or the [[luatex]] executable you will have to regenerate the formats. The command for that is:
 +
 
 +
  context --make cont-en
 +
 
 +
This will generate a new Mark IV format with the English interface.  Without an explicit format argument, it will generate four formats: Mark IV cont-en, Matk IV cont-nl, luatex+mptopdf, and luatex+plain.
 +
 
 +
The generated formats are stored inside the texmf cache.
 +
 
 +
== Using fonts ==
 +
 
 +
Context Mark IV can use non-tex fonts (in OpenType, TrueType or Type1 format) directly. The first time that you run
 +
 
 +
  context myfile.tex
 +
 
 +
on a file that uses such non-traditional fonts (which is probably always), the actual fonts themselves have to be loaded in luatex's embedded section of fontforge to extract their metric information. This removes the need for tfm and vf files on your disk, but it is also a somewhat lengthy and often memory-hungry process.
 +
 
 +
That is why the results of that internal preprocessing are also saved inside the texmf cache. Future runs using the same font (not just this file, but all files using that particular font) can then use the cached version of the metrics instead.
 +
 
 +
=== Font name database ===
 +
 
 +
Because Luatex does not know about the names of fonts all by itself,  Mark IV uses a database of 'known font names'
 +
to allow you to specify fonts by their internal name instead of the name of the disk file. The mtxrun script named
 +
'fonts' maintains this database. At any time, you can run
 +
 
 +
  mtxrun --script fonts --list
 +
 
 +
to get the list of currently known fonts.
 +
 
 +
  mtxrun --script fonts --help
 +
 
 +
will show some other options.
 +
 
 +
Previously,
 +
 
 +
  mtxrun --script fonts --reload
 +
 
 +
was always needed if you wanted to use new fonts by their font name (as opposed to their file names).
 +
 
 +
In the latest beta (juli 1008) versions of Mark IV this is no longer the case. In these new versions, the reload command is run automatically whenever a named font cannot be found.
 +
 
 +
Nowadays, you only have to do the reload manually when you have installed updates to already installed fonts that cause the font and/or file names to change. This normally only happens for fonts that are still in development like [[TeX Gyre]] and [[Latin Modern]].

Revision as of 11:57, 10 July 2008

Introduction

Context Mark IV does not use the kpathsea library to find files and configuration settings. It follows that running mktexlsr or fmtutil has no effect. This page explains the details of installing and maintaining Mark IV.


Quickstart

 luatools --generate

is the replacement for mktexlsr

 context --make

is the replacement for fmtutil


Filename and Configuration databases

Introduction

When you run

 context myfile

both the script 'context' and the Mark IV code inside the format have to find various files. To make this happen as fast as possible, a file name database is used. There is also some configuration information and some generated font data is cached because extracting the OpenType information is a relatively lengthy and memory-consuming process.

All of this data stored inside the 'texmf cache'. To keep this cache up to date, you have to (re)run

 luatools --generate

whenever you install or remove files from your tex installation that have to be found by Mark IV, like after updating context itself, or installing extra font packages from TeX Live or CTAN. System-wide fonts don't count, these are always re-discovered at runtime.


TODO: check that (See: To-Do List)


Cache location

When

 luatools --generate

is run, it creates the file name database. To prevent duplication of settings as much as possible, it parses texmf.cnf as the source for discovering at what directory trees need to be indexed in the database. This ensures that while Mark IV does not actually use kpathsea, it is still compatible with how kpathsea handles things.

But there is a catch: luatools has to decide where on the harddisk the file name database should be stored. This needs extra care, because a runtime command like 'context myfile' does not look at texmf.cnf.

The internal logic used for finding the cache location is as follows:

  • If there is an environment variable named TEXMFCACHE, this will point to the desired harddisk location
  • Otherwise, if the web2c configuration directory contains a file named texmfcnf.lua, this file is read and used to discover the desired location. this is the recommended approach, see below
  • If that doesn't work either, a list of 'likely' environment variables is tested, and the first variable that is set is assumed to point to the correct location.
  • As a last resort, the local directory is used.


For completeness: the list of tested environment variables is currently:

 TMPDIR, TEMPDIR, TMP, TEMP, HOME, HOMEPATH

but there have been quite some variations on this in the past.

texmfcnf.lua

Files named texmfcnf.lua can appear anywhere on the search path where there can also be a texmf.cnf present. Multiple files can be present. They are read sequentially using the same logic as for texmf.ncf: earlier files fix values that cannot be overruled by later ones. All texmfcnf.lua files are read before all texmf.cnf files, and values defined in texmfcnf.lua cannot be overruled by texmf.cnf files.

The format of texmfcnf.lua is pretty simple: it is a Lua script that has to return a single table with keys for the variables it wants to define. Mark IV will interpret specific keys in the table and do something special (unknown keys are ignored).

Besides the 'normal' texmf.cnf keys, the most important key is TEXMFCACHE. Here is a simple one-line example texmfcnf.lua to set up the texmf cache to be inside your private texlive 2008 directory:

 return { TEXMFCACHE =  '$HOME/.texlive2008/texmf-var' }

Note: Windows users should replace $HOME with $USERPROFILE. Both unix and windows users of TeX Live 2008 and users of Mark IV beta's released after Jul 10, 2008 can use ~ instead.

Generating and updating formats

Before first use and after each update to either the context distribution or the luatex executable you will have to regenerate the formats. The command for that is:

 context --make cont-en

This will generate a new Mark IV format with the English interface. Without an explicit format argument, it will generate four formats: Mark IV cont-en, Matk IV cont-nl, luatex+mptopdf, and luatex+plain.

The generated formats are stored inside the texmf cache.

Using fonts

Context Mark IV can use non-tex fonts (in OpenType, TrueType or Type1 format) directly. The first time that you run

 context myfile.tex

on a file that uses such non-traditional fonts (which is probably always), the actual fonts themselves have to be loaded in luatex's embedded section of fontforge to extract their metric information. This removes the need for tfm and vf files on your disk, but it is also a somewhat lengthy and often memory-hungry process.

That is why the results of that internal preprocessing are also saved inside the texmf cache. Future runs using the same font (not just this file, but all files using that particular font) can then use the cached version of the metrics instead.

Font name database

Because Luatex does not know about the names of fonts all by itself, Mark IV uses a database of 'known font names' to allow you to specify fonts by their internal name instead of the name of the disk file. The mtxrun script named 'fonts' maintains this database. At any time, you can run

 mtxrun --script fonts --list

to get the list of currently known fonts.

 mtxrun --script fonts --help

will show some other options.

Previously,

 mtxrun --script fonts --reload

was always needed if you wanted to use new fonts by their font name (as opposed to their file names).

In the latest beta (juli 1008) versions of Mark IV this is no longer the case. In these new versions, the reload command is run automatically whenever a named font cannot be found.

Nowadays, you only have to do the reload manually when you have installed updates to already installed fonts that cause the font and/or file names to change. This normally only happens for fonts that are still in development like TeX Gyre and Latin Modern.