Difference between revisions of "User:Luigi.scarso"

From Wiki
Jump to navigation Jump to search
 
(7 intermediate revisions by the same user not shown)
Line 1: Line 1:
<h1>The database</h1>
+
== Some materials ==
 +
=== Npp for ConTeXt ===
  
The <span tag="MKIV" style="font-style:sans;">bibTEX</span> format is rather popular in the <span tag="MKIV" style="font-style:sans;">TEX</span> community and even with its shortcomings it will stay around for a while. Many publication websites can export and many tools are available to work with this database format. It is rather simple and looks a bit like <span tag="MKIV" style="font-style:sans;">Lua</span> tables. Unfortunately the content can be polluted with non-standardized <span tag="MKIV" style="font-style:sans;">TEX</span> commands which complicates pre- or postprocessing outside <span tag="MKIV" style="font-style:sans;">TEX</span>. In that sense a <span tag="MKIV" style="font-style:sans;">bibTEX</span> database is often not coded neutrally. Some limitations, like the use of commands to encode accented characters root in the <span tag="MKIV" style="font-style:sans;">ascii</span> world and can be bypassed by using <span tag="MKIV" style="font-style:sans;">utf</span> instead (as handled somewhat in <span tag="MKIV" style="font-style:sans;">LATEX</span> through extensions such as <tt style="color:rgb(0,102,102);font-size:100%;" >bibtex8</tt>).
+
[https://github.com/luigiScarso/context-npp This] is a notepad++ plugin for ConTeXt. Original work by Jason Yu, current developer Luigi Scarso.  
<p></p>
 
The normal way to deal with a bibliography is to refer to entries using a unique tag or key. When a list of entries is typeset, this reference can be used for linking purposes. The typeset list can be processed and sorted using the <tt style="color:rgb(0,102,102);font-size:100%;" >bibtex</tt> program that converts the database into something more <span tag="MKIV" style="font-style:sans;">TEX</span> friendly (a <tt style="color:rgb(0,102,102);font-size:100%;" >.bbl</tt> file). I never used the program myself (nor bibliographies) so I will not go into too much detail here, if only because all I say can be wrong.
 
<p></p>
 
In <span tag="MKIV" style="font-style:sans;">ConTEXt</span> we no longer use the <tt style="color:rgb(0,102,102);font-size:100%;" >bibtex</tt> program: we just use database files and deal with the necessary manipulations directly in <span tag="MKIV" style="font-style:sans;">ConTEXt</span>. One or more such databases can be used and combined with additional entries defined within the document. We can have several such datasets active at the same time.
 
<p></p>
 
A <span tag="MKIV" style="font-style:sans;">bibTEX</span> file looks like this:
 
   
 
<pre style="color:rgb(102,0,102);font-size:100%">@Article{sometag,
 
    author  = "An Author and Another One",
 
    title  = "A hopefully meaningful title",
 
    journal = maps,
 
    volume  = "25",
 
    number  = "2",
 
    pages  = "5--9",
 
    month  = mar,
 
    year    = "2013",
 
    ISSN    = "1234-5678",
 
}
 
</pre>
 
<p></p>
 
Normally a value is given between quotes (or curly brackets) but single words are also OK (there is no real benefit in not using quotes, so we advise to always use them). There can be many more fields and instead of strings one can use predefined shortcuts. The title for example quite often contains <span tag="MKIV" style="font-style:sans;">TEX</span> macros. Some fields, like <tt style="color:rgb(0,102,102);font-size:100%;" >pages</tt> have funny characters such as the endash (typically as <tt style="color:rgb(0,102,102);font-size:100%;" >--</tt>) so we have a mixture of data and typesetting directives. If you are covering non--english references, you often need characters that are not in the <span tag="MKIV" style="font-style:sans;">ascii</span> subset but <span tag="MKIV" style="font-style:sans;">ConTEXt</span> is quite happy with <span tag="MKIV" style="font-style:sans;">utf</span>. If your database file uses old-fashioned <span tag="MKIV" style="font-style:sans;">TEX</span> accent commands then these will be internally converted automatically to <span tag="MKIV" style="font-style:sans;">utf</span>. Commands (macros) are converted to an indirect call, which is quite robust.
 
<p></p>
 
The <span tag="MKIV" style="font-style:sans;">bibTEX</span> files are loaded in memory as <span tag="MKIV" style="font-style:sans;">Lua</span> table but can be converted to <span tag="MKIV" style="font-style:sans;">xml</span> so that we can access them in a more flexible way, but that is a subject for specialists.
 
<p></p>
 
In the old <span tag="MKIV" style="font-style:sans;">MkII</span> setup we have two kinds of entries: the ones that come from the <span tag="MKIV" style="font-style:sans;">bibTEX</span> run and user supplied ones. We no longer rely on <span tag="MKIV" style="font-style:sans;">bibTEX</span> output but we do still support the user supplied definitions. These were in fact prepared in a way that suits the processing of <span tag="MKIV" style="font-style:sans;">bibTEX</span> generated entries. The next variant reflects the <span tag="MKIV" style="font-style:sans;">ConTEXt</span> recoding of the old <span tag="MKIV" style="font-style:sans;">bibTEX</span> output.
 
   
 
<pre style="color:rgb(102,0,102);font-size:100%">\startpublication[k=Hagen:Second,t=article,a={Hans Hagen},y=2013,s=HH01]
 
    \artauthor[]{Hans}[H.]{}{Hagen}
 
    \arttitle{Who knows more?}
 
    \journal{MyJournal}
 
    \pubyear{2013}
 
    \month{8}
 
    \volume{1}
 
    \issue{3}
 
    \issn{1234-5678}
 
    \pages{123--126}
 
\stoppublication
 
</pre>
 
<p></p>
 
The split <tt style="color:rgb(0,102,102);font-size:100%;" >\artauthor</tt> fields are collapsed into a single <tt style="color:rgb(0,102,102);font-size:100%;" >author</tt> field as we deal with the splitting later when it gets parsed in <span tag="MKIV" style="font-style:sans;">Lua</span>. The <tt style="color:rgb(0,102,102);font-size:100%;" >\artauthor</tt> syntax is only kept around for backward compatibility with the previous use of <span tag="MKIV" style="font-style:sans;">bibTEX</span>.
 
<p></p>
 
In the new setup we support these variants as well:
 
   
 
<pre style="color:rgb(102,0,102);font-size:100%">\startpublication[k=Hagen:Third,t=article]
 
    \author{Hans Hagen}
 
    \title{Who knows who?}
 
    ...
 
\stoppublication
 
</pre>
 
<p></p>
 
and
 
   
 
<pre style="color:rgb(102,0,102);font-size:100%">\startpublication[tag=Hagen:Third,category=article]
 
    \author{Hans Hagen}
 
    \title{Who knows who?}
 
    ...
 
\stoppublication
 
</pre>
 
<p></p>
 
and
 
   
 
<pre style="color:rgb(102,0,102);font-size:100%">\startpublication
 
    \tag{Hagen:Third}
 
    \category{article}
 
    \author{Hans Hagen}
 
    \title{Who knows who?}
 
    ...
 
\stoppublication
 
</pre>
 
<p></p>
 
Because internally the entries are <span tag="MKIV" style="font-style:sans;">Lua</span> tables, we also support loading of <span tag="MKIV" style="font-style:sans;">Lua</span> based definitions:
 
   
 
<pre style="color:rgb(102,0,102);font-size:100%">return {
 
    ["Hagen:First"] = {
 
        author  = "Hans Hagen",
 
        category = "article",
 
        issn    = "1234-5678",
 
        issue    = "3",
 
        journal  = "MyJournal",
 
        month    = "8",
 
        pages    = "123--126",
 
        tag      = "Hagen:First",
 
        title    = "Who knows nothing?",
 
        volume  = "1",
 
        year    = "2013",
 
    },
 
}
 
</pre>
 
<p></p>
 
Files set up like this can be loaded too. The following <span tag="MKIV" style="font-style:sans;">xml</span> input is rather close to this, and is also accepted as input.
 
   
 
<pre style="color:rgb(102,0,102);font-size:100%"><?xml version="2.0" standalone="yes" ?>
 
<bibtex>
 
    <entry tag="Hagen:First" category="article">
 
        <field name="author">Hans Hagen</field>
 
        <field name="category">article</field>
 
        <field name="issn">1234-5678</field>
 
        <field name="issue">3</field>
 
        <field name="journal">MyJournal</field>
 
        <field name="month">8</field>
 
        <field name="pages">123--126</field>
 
        <field name="tag">Hagen:First</field>
 
        <field name="title">Who knows nothing?</field>
 
        <field name="volume">1</field>
 
        <field name="year">2013</field>
 
    </entry>
 
</bibtex>
 
</pre>
 
<p></p>
 
Todo: Add some remarks about loading EndNote and RIS formats, but first we need to complete the tag mapping (on Alan’s plate).
 
<p></p>
 
So the user has a rather wide choice of formatting style for bibliography database files.
 
   
 
You can load more data than you actually need. Only entries that are referred to explicitly through the <tt style="color:rgb(0,102,102);font-size:100%;" >\cite</tt> and <tt style="color:rgb(0,102,102);font-size:100%;" >\nocite</tt> commands will be shown in lists. We will cover these details later.
 
 
 
<h1>Commands in entries</h1>
 
  
One unfortunate aspect commonly found in <span tag="MKIV" style="font-style:sans;">bibTEX</span> files is that they often contain <span tag="MKIV" style="font-style:sans;">TEX</span> commands. Even worse is that there is no standard on what these commands can be and what they mean, at least not formally, as <span tag="MKIV" style="font-style:sans;">bibTEX</span> is a program intended to be used with many variants of <span tag="MKIV" style="font-style:sans;">TEX</span> style: plain, <span tag="MKIV" style="font-style:sans;">LATEX</span>, and others. This means that we need to define our use of these typesetting commands. However, in most cases, they are just abbreviations or font switches and these are often known. Therefore, <span tag="MKIV" style="font-style:sans;">ConTEXt</span> will try to resolve them before reporting an issue. In the log file there is a list of commands that has been seen in the loaded databases. For instance, loading <tt style="color:rgb(0,102,102);font-size:100%;" >tugboat.bib</tt> gives a long list of commands of which we show a small set here:
+
The plugin is available here: [[File:Npp-for-ConTeXt.zip]].
   
+
Manual under doc/npp-context-manual.pdf of the archive.
<pre style="color:rgb(102,0,102);font-size:100%">publications > start used btx commands
 
publications > standard CONTEXT 1 known
 
publications > standard ConTeXt 4 known
 
publications > standard TeXLive 3 KNOWN
 
publications > standard eTeX    1 known
 
publications > standard hbox    6 known
 
publications > standard sltt    1 unknown
 
publications > stop used btxcommands
 
</pre>
 
<p></p>
 
You can define unknown commands, or overload existing definitions in the following way:
 
   
 
<pre style="color:rgb(102,0,102);font-size:100%">\definebtxcommand\TUB {TUGboat}
 
\definebtxcommand\sltt{\tt}
 
\definebtxcommand\<#1>{\type{#1}}
 
</pre>
 
<p></p>
 
Unknown commands do not stall processing, but their names are then typeset in a mono- spaced font so they probably stand out for proofreading. You can access the commands with <tt style="color:rgb(0,102,102);font-size:100%;" >\btxcommand{...}</tt>, as in:
 
   
 
<pre style="color:rgb(102,0,102);font-size:100%">commands like \btxcommand{MySpecialCommand} are handled in an indirect way
 
</pre>
 
<p></p>
 
As this is an undefined command we get: “commands like MySpecialCommand are handled in an indirect way”.
 
<p></p>
 
??
 
   
 
   
 
<h1>Datasets</h1>
 
 
 
Normally in a document you will use only one bibliographic database, whether or not distributed over multiple files. Nevertheless we support multiple databases as well which is why we talk of datasets instead. A dataset is loaded with the <tt style="color:rgb(0,102,102);font-size:100%;" >\usebtxdataset</tt> command. Although currently it is not necessary to define a (default) dataset you can best do this because in the future we might provide more options. Here are some examples:
 
   
 
<pre style="color:rgb(102,0,102);font-size:100%">\definebtxdataset[standard]
 
\usebtxdataset[standard][tugboat.bib]
 
\usebtxdataset[standard][mtx-bibtex-output.xml]
 
\usebtxdataset[standard][test-001-btx-standard.lua]
 
</pre>
 
<p></p>
 
These three suffixes are understood by the loader. Here the dataset has the name <tt style="color:rgb(0,102,102);font-size:100%;" >standard</tt> and the three database files are merged, where later entries having the same tag overload previous ones. Definitions in the document source (coded in <span tag="MKIV" style="font-style:sans;">TEX</span> speak) are also added, and they are saved for successive runs. This means that if you load and define entries, they will be known at a next run beforehand, so that references to them are independent of when loading and definitions take place.
 
<div style="border:thin solid black;" >
 
<span style="font-style:oblique;" > setup definition setupbtxdataset </span >
 
</div>
 
<div style="border:thin solid black;" >
 
<span style="font-style:oblique;" > setup definition definebtxdataset </span >
 
</div>
 
<div style="border:thin solid black;" >
 
<span style="font-style:oblique;" > setup definition usebtxdataset </span >
 
</div>
 
<p></p>
 
In this document we use some example databases, so let’s load one of them now:
 
   
 
<pre style="color:rgb(102,0,102);font-size:100%">\definebtxdataset[example]
 
\usebtxdataset[example][mkiv-publications.bib]
 
</pre>
 
<p></p>
 
You can ask for an overview of entries in a dataset with:
 
   
 
<pre style="color:rgb(102,0,102);font-size:100%">\showbtxdatasetfields[example]
 
</pre>
 
<p></p>
 
this gives:
 
   
 
<table>
 
<tr><td>tag</td><td>category</td><td>fields</td></tr>
 
<tr><td>demo-001</td><td>book</td><td>author index title year</td></tr>
 
<tr><td>demo-002</td><td>book</td><td>crossref index year</td></tr>
 
<tr><td>demo-003</td><td>book</td><td>author comment index title year</td></tr>
 
<tr><td>demo-004</td><td>book</td><td>author comment index title year</td></tr>
 
<tr><td>demo-005</td><td>book</td><td>author doi index pages serial title url year</td></tr>
 
</table>
 
 
 
<p></p>
 
You can set the current active dataset with
 
   
 
<pre style="color:rgb(102,0,102);font-size:100%">\setbtxdataset[standard]
 
</pre>
 
<p></p>
 
but most publication-related commands accept optional arguments that denote the dataset and references to entries can be prefixed with a dataset identifier.. More about that later.
 
<p></p>
 
Sometimes you want to check a database. One way of doing that is the following:
 
   
 
<pre style="color:rgb(102,0,102);font-size:100%">\definebtxdataset[check]
 
\usebtxdataset[check][mkiv-publications-check.bib]
 
\showbtxdatasetcompleteness[check]
 
</pre>
 
<p></p>
 
The database like like this:
 
<p></p>
 
The completeness check shows (with green field names) the required fields and when one is missing this is indicated in red. In blue we show what gets inherited.
 
   
 
   
 
<h1>Renderings</h1>
 
 
 
A list of publications can be rendered at any place in the document. A database can be much larger than needed for a document. The same is true for the fields that make up an entry. Here is the list of fields that are currently handled, but of course there can be additional ones:
 
<p></p>
 
<tt style="color:rgb(0,102,102);font-size:100%;" >abstract</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >address</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >annotate</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >assignee</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >author</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >bibnumber</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >booktitle</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >chapter</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >comment</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >country</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >day</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >dayfiled</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >doi</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >edition</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >editor</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >eprint</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >howpublished</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >institution</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >isbn</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >issn</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >journal</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >key</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >keyword</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >keywords</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >language</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >lastchecked</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >month</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >monthfiled</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >names</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >nationality</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >note</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >notes</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >number</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >organization</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >pages</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >publisher</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >revision</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >school</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >series</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >size</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >title</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >type</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >url</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >volume</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >year</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >yearfiled</tt>
 
<p></p>
 
If you want to see what publications are in the database, the easiest way is to ask for a complete list:
 
   
 
<pre style="color:rgb(102,0,102);font-size:100%">\definebtxrendering
 
  [example]
 
  [dataset=example,
 
  method=local,
 
  alternative=apa]
 
\placelistofpublications % \placebtxrendering
 
  [example]
 
  [criterium=all]
 
</pre>
 
<p></p>
 
This gives:1 Hagen, H. and Otten, T. (1996). Typesetting education documents.2 Scarso, L. (2021). Designing high speed trains.3 author (year). title. pages p.
 
<p></p>
 
The rendering itself is somewhat complex to set up because we have not only many different standards but also many fields that can be set up. This means that there are several commands involved. Often there is a prescribed style to render bibliographic descriptions, for example <tt style="color:rgb(0,102,102);font-size:100%;" >apa</tt>. A rendering is setup and defined with:
 
<div style="border:thin solid black;" >
 
<span style="font-style:oblique;" > setup definition setupbtxrendering </span >
 
</div>
 
<div style="border:thin solid black;" >
 
<span style="font-style:oblique;" > setup definition definebtxrendering </span >
 
</div>
 
<p></p>
 
And a list of such descriptions is generated with:
 
<div style="border:thin solid black;" >
 
<span style="font-style:oblique;" > setup definition placebtxrendering </span >
 
</div>
 
<p></p>
 
A dataset can have all kind of entries:
 
<p></p>
 
Each has its own rendering variant. To keep things simple we have their settings separated. However, these settings are shared for all rendering alternatives. In practice this is seldom a problem in a publication as only one rendering alternative will be active. If this be not sufficient, you can always group local settings in a setup and hook that into the specific rendering.
 
<div style="border:thin solid black;" >
 
<span style="font-style:oblique;" > setup definition setupbtxlistvariant </span >
 
</div>
 
<div style="border:thin solid black;" >
 
<span style="font-style:oblique;" > setup definition definebtxlistvariant </span >
 
</div>
 
<p></p>
 
Examples of list variants are:
 
<p></p>
 
<tt style="color:rgb(102,0,102);font-size:100%;" >setupbtxlistvariant : artauthor</tt>
 
   
 
<table>
 
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >no specific settings</tt></td><td></td></tr>
 
</table>
 
 
 
<p></p>
 
<tt style="color:rgb(102,0,102);font-size:100%;" >setupbtxlistvariant : author</tt>
 
   
 
<table>
 
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >no specific settings</tt></td><td></td></tr>
 
</table>
 
 
 
<tt style="color:rgb(102,0,102);font-size:100%;" >setupbtxlistvariant : editor</tt>
 
   
 
<table>
 
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >no specific settings</tt></td><td></td></tr>
 
</table>
 
 
 
<p></p>
 
The exact rendering of list entries is determined by the <tt style="color:rgb(0,102,102);font-size:100%;" >alternative</tt> key and defaults to <tt style="color:rgb(0,102,102);font-size:100%;" >apa</tt> which uses definitions from <tt style="color:rgb(0,102,102);font-size:100%;" >publ-imp-apa.mkiv</tt>. If you look at that file you will see that each category has its own setup. You may also notice that additional tests are needed to make sure that empty fields don’t trigger separators and such.
 
<p></p>
 
There are a couple of accessors and helpers to get the job done. When you want to fetch a field from the current entry you use <tt style="color:rgb(0,102,102);font-size:100%;" >\btxfield</tt>. In most cases you want to make sure this field has a value, for instance because you don’t want fences or punctuation that belongs to a field.
 
   
 
<pre style="color:rgb(102,0,102);font-size:100%">\btxdoif {title} {
 
    \bold{\btxfield{title}},
 
}
 
</pre>
 
<p></p>
 
There are three test macros:
 
   
 
<pre style="color:rgb(102,0,102);font-size:100%">\btxdoifelse{fieldname}{action when found}{action when not found}
 
\btxdoif    {fieldname}{action when found}
 
\btxdoifnot {fieldname}                  {action when not found}
 
</pre>
 
<p></p>
 
An extra conditional is available for testing interactivity:
 
   
 
<pre style="color:rgb(102,0,102);font-size:100%">\btxdoifelseinteraction{action when true}{action when false}
 
</pre>
 
<p></p>
 
In addition there is also a conditional <tt style="color:rgb(0,102,102);font-size:100%;" >\btxinteractive</tt> which is more efficient, although in practice efficiency is not so important here.
 
<p></p>
 
There are three commands to flush data:
 
   
 
<table>
 
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >\btxfield</tt></td><td>fetch a explicit field (e.g. <tt style="color:rgb(0,102,102);font-size:100%;" >year</tt>)</td></tr>
 
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >\btxdetail</tt></td><td>fetch a derived field (e.g. <tt style="color:rgb(0,102,102);font-size:100%;" >short</tt>)</td></tr>
 
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >\btxflush</tt></td><td>fetch a derived or explicit field</td></tr>
 
</table>
 
 
 
<p></p>
 
Normally you can use <tt style="color:rgb(0,102,102);font-size:100%;" >\btxfield</tt> or <tt style="color:rgb(0,102,102);font-size:100%;" >\btxflush</tt> as derived fields just like analyzed author fields are flushed in a special way.
 
<p></p>
 
You can improve readability by using setups, for instance:
 
   
 
<pre style="color:rgb(102,0,102);font-size:100%">\btxdoifelse {author} {
 
    \btxsetup{btx:apa:author:yes}
 
} {
 
    \btxsetup{btx:apa:author:nop}
 
}
 
</pre>
 
<p></p>
 
Keep in mind that normally you don’t need to mess with definitions like this because standard rendering styles are provided. These styles use a few helpers that inject symbols but also take care of leading and trailing spaces:
 
   
 
<table>
 
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >\btxspace</tt></td><td>before after</td></tr>
 
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >\btxperiod</tt></td><td>before. after</td></tr>
 
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >\btxcomma</tt></td><td>before, after</td></tr>
 
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >\btxlparent</tt></td><td>before (after</td></tr>
 
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >\btxrparent</tt></td><td>before) after</td></tr>
 
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >\btxlbracket</tt></td><td>before [after</td></tr>
 
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >\btxrbracket</tt></td><td>before] after</td></tr>
 
</table>
 
 
 
<p></p>
 
So, the previous example setup can be rewritten as:
 
   
 
<pre style="color:rgb(102,0,102);font-size:100%">\btxdoif {title} {
 
    \bold{\btxfield{title}}
 
    \btxcomma
 
}
 
</pre>
 
<p></p>
 
There is a special command for rendering a (combination) of authors:
 
   
 
<pre style="color:rgb(102,0,102);font-size:100%">\btxflushauthor{author}
 
\btxflushauthor{editor}
 
\btxflushauthor[inverted]{editor}
 
</pre>
 
<p></p>
 
Instead of the last one you can also use:
 
   
 
<pre style="color:rgb(102,0,102);font-size:100%">\btxflushauthorinverted{editor}
 
</pre>
 
<p></p>
 
You can use a (configurable) default or pass directives: Valid directives are
 
   
 
<table>
 
<tr><td>conversion</td><td>rendering</td></tr>
 
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >inverted</tt></td><td>the Frog jr, Kermit</td></tr>
 
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >invertedshort</tt></td><td>the Frog jr, K</td></tr>
 
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >normal</tt></td><td>Kermit, the Frog, jr</td></tr>
 
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >normalshort</tt></td><td>K, the Frog, jr</td></tr>
 
</table>
 
 
 
   
 
   
 
<h1>Citations</h1>
 
 
 
Citations are references to bibliographic entries that normally show up in lists someplace in the document: at the end of a chapter, in an appendix, at the end of an article, etc. We discussed the rendering of these lists in the previous chapter. A citation is normally pretty short as its main purpose is to refer uniquely to a more detailed description. But, there are several ways to refer, which is why the citation subsystem is configurable and extensible. Just look at the following commands:
 
   
 
<pre style="color:rgb(102,0,102);font-size:100%">\cite[author][example::demo-003]
 
\cite[authoryear][example::demo-003]
 
\cite[authoryears][example::demo-003]
 
\cite[author][example::demo-003,demo-004]
 
\cite[authoryear][example::demo-003,demo-004]
 
\cite[authoryears][example::demo-003,demo-004]
 
\cite[author][example::demo-004,demo-003]
 
\cite[authoryear][example::demo-004,demo-003]
 
\cite[authoryears][example::demo-004,demo-003]
 
</pre>
 
   
 
<li>
 
<ul>(Hans Hagen and Ton Otten)</ul>
 
<ul>(Hans Hagen and Ton Otten (1996))</ul>
 
<ul>(Hans Hagen and Ton Otten, 1996)</ul>
 
<ul>(Hans Hagen and Ton Otten, Luigi Scarso)</ul>
 
<ul>(Hans Hagen and Ton Otten (1996), Luigi Scarso (2021))</ul>
 
<ul>(Hans Hagen and Ton Otten, 1996, Luigi Scarso, 2021)</ul>
 
<ul>(Luigi Scarso, Hans Hagen and Ton Otten)</ul>
 
<ul>(Luigi Scarso (2021), Hans Hagen and Ton Otten (1996))</ul>
 
<ul>(Luigi Scarso, 2021, Hans Hagen and Ton Otten, 1996)</ul>
 
</li>
 
<p></p>
 
The first argument is optional.
 
<div style="border:thin solid black;" >
 
<span style="font-style:oblique;" > setup definition cite </span >
 
</div>
 
<p></p>
 
You can tune the way a citation shows up:
 
   
 
<pre style="color:rgb(102,0,102);font-size:100%">\setupbtxcitevariant[author]    [sorttype=author,color=darkyellow]
 
\setupbtxcitevariant[authoryear] [sorttype=author,color=darkyellow]
 
\setupbtxcitevariant[authoryears][sorttype=author,color=darkyellow]
 
\cite[author][example::demo-004,demo-003]
 
\cite[authoryear][example::demo-004,demo-003]
 
\cite[authoryears][example::demo-004,demo-003]
 
</pre>
 
<p></p>
 
Here we sort the authors and color the citation:
 
   
 
<li>
 
<ul>(Hans Hagen and Ton Otten, Luigi Scarso)</ul>
 
<ul>(Hans Hagen and Ton Otten (1996), Luigi Scarso (2021))</ul>
 
<ul>(Hans Hagen and Ton Otten, 1996, Luigi Scarso, 2021)</ul>
 
</li>
 
<p></p>
 
For reasons of backward compatibility the <tt style="color:rgb(0,102,102);font-size:100%;" >\cite</tt> command is a bit picky about spaces between the two arguments, of which the first is optional. This is a consequence of allowing its use with the key specified between curly brackets as is the traditional practice. (We do encourage users to adopt the more coherent <span tag="MKIV" style="font-style:sans;">ConTEXt</span> syntax by using square brackets for keywords and reserving curly brackets to regroup text to be typeset.)
 
<p></p>
 
The <tt style="color:rgb(0,102,102);font-size:100%;" >\citation</tt> command is synonymous but is more flexible with respect to spacing of its arguments:
 
   
 
<pre style="color:rgb(102,0,102);font-size:100%">\citation[author]    [example::demo-004,demo-003]
 
\citation[authoryear] [example::demo-004,demo-003]
 
\citation[authoryears][example::demo-004,demo-003]
 
</pre>
 
<p></p>
 
There is a whole bunch of cite options and more can be easily defined.  
 
   
 
<table>
 
<tr><td>key</td><td>rendering</td></tr>
 
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >author</tt></td><td>(author)</td></tr>
 
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >authornum</tt></td><td>[author [btx error 1]]</td></tr>
 
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >authoryear</tt></td><td>(author (year))</td></tr>
 
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >authoryears</tt></td><td>(author, year)</td></tr>
 
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >doi</tt></td><td>[todo: doi]</td></tr>
 
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >key</tt></td><td>[demo-005]</td></tr>
 
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >none</tt></td><td></td></tr>
 
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >num</tt></td><td>[[btx error 1]]</td></tr>
 
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >page</tt></td><td>pages</td></tr>
 
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >serial</tt></td><td>[5]</td></tr>
 
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >short</tt></td><td>[aut00]</td></tr>
 
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >type</tt></td><td>[book]</td></tr>
 
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >url</tt></td><td>[todo: url]</td></tr>
 
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >year</tt></td><td>(year)</td></tr>
 
</table>
 
 
 
<p></p>
 
Because we are dealing with database input and because we generally need to manipulate entries, much of the work is delegated to <span tag="MKIV" style="font-style:sans;">Lua</span>. This makes it easier to maintain and extend the code. Of course <span tag="MKIV" style="font-style:sans;">TEX</span> still does the rendering. The typographic details are controlled by parameters but not all are used in all variants. As with most <span tag="MKIV" style="font-style:sans;">ConTEXt</span> commands, it starts out with a general setup command:
 
<div style="border:thin solid black;" >
 
<span style="font-style:oblique;" > setup definition setupbtxcitevariant </span >
 
</div>
 
<p></p>
 
On top of that we can define instances that inherit either from a given parent or from the topmost setup.
 
<div style="border:thin solid black;" >
 
<span style="font-style:oblique;" > setup definition definebtxcitevariant </span >
 
</div>
 
<p></p>
 
But, specific variants can have them overloaded:
 
<p></p>
 
<tt style="color:rgb(102,0,102);font-size:100%;" >setupbtxcitevariant : author</tt>
 
   
 
<table>
 
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >right</tt></td><td><tt style="color:rgb(0,102,102);font-size:100%;" >)</tt></td></tr>
 
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >middle</tt></td><td><tt style="color:rgb(0,102,102);font-size:100%;" >, </tt></td></tr>
 
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >left</tt></td><td><tt style="color:rgb(0,102,102);font-size:100%;" >(</tt></td></tr>
 
</table>
 
 
 
<p></p>
 
<tt style="color:rgb(102,0,102);font-size:100%;" >setupbtxcitevariant : authornum</tt>
 
   
 
<table>
 
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >right</tt></td><td><tt style="color:rgb(0,102,102);font-size:100%;" >]</tt></td></tr>
 
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >middle</tt></td><td><tt style="color:rgb(0,102,102);font-size:100%;" >, </tt></td></tr>
 
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >left</tt></td><td><tt style="color:rgb(0,102,102);font-size:100%;" >[</tt></td></tr>
 
</table>
 
 
 
<p></p>
 
<tt style="color:rgb(102,0,102);font-size:100%;" >setupbtxcitevariant : authoryear</tt>
 
   
 
<table>
 
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >compress</tt></td><td><tt style="color:rgb(0,102,102);font-size:100%;" >yes</tt></td></tr>
 
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >inbetween</tt></td><td><tt style="color:rgb(0,102,102);font-size:100%;" >, </tt></td></tr>
 
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >right</tt></td><td><tt style="color:rgb(0,102,102);font-size:100%;" >)</tt></td></tr>
 
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >middle</tt></td><td><tt style="color:rgb(0,102,102);font-size:100%;" >, </tt></td></tr>
 
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >left</tt></td><td><tt style="color:rgb(0,102,102);font-size:100%;" >(</tt></td></tr>
 
</table>
 
 
 
<p></p>
 
<tt style="color:rgb(102,0,102);font-size:100%;" >setupbtxcitevariant : authoryears</tt>
 
   
 
<table>
 
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >compress</tt></td><td><tt style="color:rgb(0,102,102);font-size:100%;" >yes</tt></td></tr>
 
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >inbetween</tt></td><td><tt style="color:rgb(0,102,102);font-size:100%;" >, </tt></td></tr>
 
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >right</tt></td><td><tt style="color:rgb(0,102,102);font-size:100%;" >)</tt></td></tr>
 
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >middle</tt></td><td><tt style="color:rgb(0,102,102);font-size:100%;" >, </tt></td></tr>
 
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >left</tt></td><td><tt style="color:rgb(0,102,102);font-size:100%;" >(</tt></td></tr>
 
</table>
 
 
 
<p></p>
 
<tt style="color:rgb(102,0,102);font-size:100%;" >setupbtxcitevariant : doi</tt>
 
   
 
<table>
 
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >right</tt></td><td><tt style="color:rgb(0,102,102);font-size:100%;" >]</tt></td></tr>
 
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >left</tt></td><td><tt style="color:rgb(0,102,102);font-size:100%;" >[</tt></td></tr>
 
</table>
 
 
 
<p></p>
 
<tt style="color:rgb(102,0,102);font-size:100%;" >setupbtxcitevariant : key</tt>
 
   
 
<table>
 
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >right</tt></td><td><tt style="color:rgb(0,102,102);font-size:100%;" >]</tt></td></tr>
 
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >left</tt></td><td><tt style="color:rgb(0,102,102);font-size:100%;" >[</tt></td></tr>
 
</table>
 
 
 
<p></p>
 
<tt style="color:rgb(102,0,102);font-size:100%;" >setupbtxcitevariant : none</tt>
 
   
 
<table>
 
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >no specific settings</tt></td><td></td></tr>
 
</table>
 
 
 
<p></p>
 
<tt style="color:rgb(102,0,102);font-size:100%;" >setupbtxcitevariant : num</tt>
 
   
 
<table>
 
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >compress</tt></td><td><tt style="color:rgb(0,102,102);font-size:100%;" >yes</tt></td></tr>
 
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >inbetween</tt></td><td><tt style="color:rgb(0,102,102);font-size:100%;" >--</tt></td></tr>
 
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >right</tt></td><td><tt style="color:rgb(0,102,102);font-size:100%;" >]</tt></td></tr>
 
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >left</tt></td><td><tt style="color:rgb(0,102,102);font-size:100%;" >[</tt></td></tr>
 
</table>
 
 
 
<p></p>
 
<tt style="color:rgb(102,0,102);font-size:100%;" >setupbtxcitevariant : page</tt>
 
   
 
<table>
 
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >inbetween</tt></td><td><tt style="color:rgb(0,102,102);font-size:100%;" >–</tt></td></tr>
 
</table>
 
 
 
<p></p>
 
<tt style="color:rgb(102,0,102);font-size:100%;" >setupbtxcitevariant : serial</tt>
 
   
 
<table>
 
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >right</tt></td><td><tt style="color:rgb(0,102,102);font-size:100%;" >]</tt></td></tr>
 
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >left</tt></td><td><tt style="color:rgb(0,102,102);font-size:100%;" >[</tt></td></tr>
 
</table>
 
 
 
<p></p>
 
<tt style="color:rgb(102,0,102);font-size:100%;" >setupbtxcitevariant : short</tt>
 
   
 
<table>
 
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >right</tt></td><td><tt style="color:rgb(0,102,102);font-size:100%;" >]</tt></td></tr>
 
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >left</tt></td><td><tt style="color:rgb(0,102,102);font-size:100%;" >[</tt></td></tr>
 
</table>
 
 
 
<p></p>
 
<tt style="color:rgb(102,0,102);font-size:100%;" >setupbtxcitevariant : type</tt>
 
   
 
<table>
 
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >right</tt></td><td><tt style="color:rgb(0,102,102);font-size:100%;" >]</tt></td></tr>
 
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >left</tt></td><td><tt style="color:rgb(0,102,102);font-size:100%;" >[</tt></td></tr>
 
</table>
 
 
 
<p></p>
 
<tt style="color:rgb(102,0,102);font-size:100%;" >setupbtxcitevariant : url</tt>
 
   
 
<table>
 
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >right</tt></td><td><tt style="color:rgb(0,102,102);font-size:100%;" >]</tt></td></tr>
 
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >left</tt></td><td><tt style="color:rgb(0,102,102);font-size:100%;" >[</tt></td></tr>
 
</table>
 
 
 
<p></p>
 
<tt style="color:rgb(102,0,102);font-size:100%;" >setupbtxcitevariant : year</tt>
 
   
 
<table>
 
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >right</tt></td><td><tt style="color:rgb(0,102,102);font-size:100%;" >)</tt></td></tr>
 
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >left</tt></td><td><tt style="color:rgb(0,102,102);font-size:100%;" >(</tt></td></tr>
 
</table>
 
 
 
A citation variant is defined in several steps and if you really want to know the dirty details, you should look into the <tt style="color:rgb(0,102,102);font-size:100%;" >publ-imp-*.mkiv</tt> files. Here we stick to the concept.
 
   
 
<pre style="color:rgb(102,0,102);font-size:100%">\startsetups btx:cite:author
 
    \btxcitevariant{author}
 
\stopsetups
 
</pre>
 
<p></p>
 
You can overload such setups if needed, but that only makes sense when you cannot configure the rendering with parameters. The <tt style="color:rgb(0,102,102);font-size:100%;" >\btxcitevariant</tt> command is one of the build in accessors and it calls out to <span tag="MKIV" style="font-style:sans;">Lua</span> where more complex manipulation takes place if needed. If no manipulation is known, the field with the same name (if found) will be flushed. A command like <tt style="color:rgb(0,102,102);font-size:100%;" >\btxcitevariant</tt> assumes that a dataset and specific tag has been set. This is normally done in the wrapper macros, like <tt style="color:rgb(0,102,102);font-size:100%;" >\cite</tt>. For special purposes you can use these commands
 
   
 
<pre style="color:rgb(102,0,102);font-size:100%">\setbtxdataset[example]
 
\setbtxentry[hh2013]
 
</pre>
 
<p></p>
 
But don’t expect too much support for such low level rendering control.
 
<p></p>
 
Unless you use <tt style="color:rgb(0,102,102);font-size:100%;" >criterium=all</tt> only publications that are cited will end up in the lists. You can force a citation into a list using <tt style="color:rgb(0,102,102);font-size:100%;" >\usecitation</tt>, for example:
 
   
 
<pre style="color:rgb(102,0,102);font-size:100%">\usecitation[example::demo-004,demo-003]
 
</pre>
 
<p></p>
 
This command has two synonyms: <tt style="color:rgb(0,102,102);font-size:100%;" >\nocite</tt> and <tt style="color:rgb(0,102,102);font-size:100%;" >\nocitation</tt> so you can choose whatever fits you best.
 
<div style="border:thin solid black;" >
 
<span style="font-style:oblique;" > setup definition nocite </span >
 
</div>
 
   
 
   
 
<h1>The LUA view</h1>
 
 
 
Because we manage data at the <span tag="MKIV" style="font-style:sans;">Lua</span> end it is tempting to access it there for other purposes. This is fine as long as you keep in mind that aspects of the implementation may change over time, although this is unlikely once the modules become stable.
 
<p></p>
 
The entries are collected in datasets and each set has a unique name. In this document we have the set named <tt style="color:rgb(0,102,102);font-size:100%;" >example</tt>. A dataset table has several fields, and probably the one of most interest is the <tt style="color:rgb(0,102,102);font-size:100%;" >luadata</tt> field. Each entry in this table describes a publication:
 
   
 
<pre style="color:rgb(102,0,102);font-size:100%">t={
 
["author"]="Hans Hagen",
 
  ["category"]="book",
 
["index"]=1,
 
["tag"]="demo-001",
 
["title"]="\\btxcmd{BIBTEX}, the \\btxcmd{CONTEXT}\\ way",
 
["year"]="2013",
 
}
 
</pre>
 
This is <tt style="color:rgb(0,102,102);font-size:100%;" >publications.datasets.example.luadata["demo-001"]</tt>. There can be a companion entry in the parallel <tt style="color:rgb(0,102,102);font-size:100%;" >details</tt> table.
 
   
 
<pre style="color:rgb(102,0,102);font-size:100%">t={
 
["author"]={
 
  {
 
  ["firstnames"]={ "Hans" },
 
  ["initials"]={ "H" },
 
  ["original"]="Hans Hagen",
 
  ["surnames"]={ "Hagen" },
 
  ["vons"]={},
 
  },
 
},
 
["short"]="Hag13",
 
}
 
</pre>
 
These details are accessed as <tt style="color:rgb(0,102,102);font-size:100%;" >publications.datasets.example.details["demo-001"]</tt> and by using a separate table we can overload fields in the original entry without losing the original.
 
<p></p>
 
You can loop over the entries using regular <span tag="MKIV" style="font-style:sans;">Lua</span> code combined with <span tag="MKIV" style="font-style:sans;">MkIV</span> helpers:
 
   
 
<pre style="color:rgb(102,0,102);font-size:100%">local dataset = publications.datasets.example
 
context.starttabulate { "|l|l|l|" }
 
for tag, entry in table.sortedhash(dataset.luadata) do
 
    local detail = dataset.details[tag] or { }
 
    context.NC() context.type(tag)
 
    context.NC() context(detail.short)
 
    context.NC() context(entry.title)
 
    context.NC() context.NR()
 
end
 
context.stoptabulate()
 
</pre>
 
<p></p>
 
This results in:
 
   
 
<table>
 
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >demo-001</tt></td><td>Hag13</td><td><span tag="MKIV" style="font-style:sans;">bibTEX</span>, the <span tag="MKIV" style="font-style:sans;">ConTEXt</span> way</td></tr>
 
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >demo-002</tt></td><td>Hag14</td><td><span tag="MKIV" style="font-style:sans;">bibTEX</span>, the <span tag="MKIV" style="font-style:sans;">ConTEXt</span> way</td></tr>
 
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >demo-003</tt></td><td>HO96</td><td>Typesetting education documents</td></tr>
 
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >demo-004</tt></td><td>Sca21</td><td>Designing high speed trains</td></tr>
 
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >demo-005</tt></td><td>aut00</td><td>title</td></tr>
 
</table>
 
 
 
<p></p>
 
You can manipulate a dataset after loading. Of course this assumes that you know what kind of content you have and what you need for rendering. As example we load a small dataset.
 
   
 
<pre style="color:rgb(102,0,102);font-size:100%">\definebtxdataset[drumming]
 
\usebtxdataset[drumming][mkiv-publications.lua]
 
</pre>
 
<p></p>
 
Because we’re going to do some <span tag="MKIV" style="font-style:sans;">Lua</span>, we could also have loaded the dataset with:
 
   
 
<pre style="color:rgb(102,0,102);font-size:100%">publications.load("drumming","mkiv-publications.lua","lua")
 
</pre>
 
<p></p>
 
The dataset has three entries:
 
<p></p>
 
As you can see, we can have a subtitle. We will combine the title and subtitle into one:
 
   
 
<pre style="color:rgb(102,0,102);font-size:100%">\startluacode
 
for tag, entry in next, publications.datasets.drumming.luadata do
 
    if entry.subtitle then
 
        if entry.title then
 
            entry.title = entry.title .. ", " .. entry.subtitle
 
        else
 
            entry.title = entry.subtitle
 
        end
 
        entry.subtitle = nil
 
        logs.report("btx","combining title and subtitle of entry tagged %a",tag)
 
    end
 
end
 
\stopluacode
 
</pre>
 
<p></p>
 
We can now typeset the entries with:
 
   
 
<pre style="color:rgb(102,0,102);font-size:100%">\definebtxrendering[drumming][dataset=drumming,method=dataset]
 
\placebtxrendering[drumming]
 
</pre>
 
<p></p>
 
Because we just want to show the entries, and have no citations that force them to be shown, we have to the <tt style="color:rgb(0,102,102);font-size:100%;" >method</tt> to <tt style="color:rgb(0,102,102);font-size:100%;" >dataset</tt>.1
 
   
 
 
 
 
 
<h1>The XML view</h1>
 
 
 
The <tt style="color:rgb(0,102,102);font-size:100%;" >luadata</tt> table can be converted into an <span tag="MKIV" style="font-style:sans;">xml</span> representation. This is a follow up on earlier experiments with an <span tag="MKIV" style="font-style:sans;">xml</span>-only approach. I decided in the end to stick to a <span tag="MKIV" style="font-style:sans;">Lua</span> approach and provide some simple <span tag="MKIV" style="font-style:sans;">xml</span> support in addition.
 
<p></p>
 
Once a dataset is accessible as <span tag="MKIV" style="font-style:sans;">xml</span> tree, you can use the regular <tt style="color:rgb(0,102,102);font-size:100%;" >\xml...</tt> commands. We start with loading a dataset, in this case from just one file.
 
   
 
<pre style="color:rgb(102,0,102);font-size:100%">\usebtxdataset[tugboat][tugboat.bib]
 
</pre>
 
<p></p>
 
The dataset has to be converted to <span tag="MKIV" style="font-style:sans;">xml</span>:
 
   
 
<pre style="color:rgb(102,0,102);font-size:100%">\convertbtxdatasettoxml[tugboat]
 
</pre>
 
<p></p>
 
The tree is now accessible by its root reference <tt style="color:rgb(0,102,102);font-size:100%;" >btx:tugboat</tt>. If we want simple field access we can use a few setups:
 
   
 
<pre style="color:rgb(102,0,102);font-size:100%">\startxmlsetups btx:initialize
 
    \xmlsetsetup{#1}{bibtex|entry|field}{btx:*}
 
    \xmlmain{#1}
 
\stopxmlsetups
 
\startxmlsetups btx:field
 
    \xmlflushcontext{#1}
 
\stopxmlsetups
 
\xmlsetup{btx:tugboat}{btx:initialize}
 
</pre>
 
<p></p>
 
The two setups are predefined in the core already, but you might want to change them. They are applied in for instance:
 
   
 
<pre style="color:rgb(102,0,102);font-size:100%">\starttabulate[|||]
 
    \NC \type {tag}  \NC \xmlfirst {btx:tugboat}
 
        {/bibtex/entry[string.find(@tag,'Hagen')]/attribute('tag')}
 
    \NC \NR
 
    \NC \type {title} \NC \xmlfirst {btx:tugboat}
 
        {/bibtex/entry[string.find(@tag,'Hagen')]/field[@name='title']}
 
    \NC \NR
 
\stoptabulate
 
</pre>
 
   
 
<table>
 
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >tag</tt></td><td>Hagen:TB17-1-54</td></tr>
 
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >title</tt></td><td>PPCHTEX: typesetting chemical formulas in TEX</td></tr>
 
</table>
 
 
 
   
 
<pre style="color:rgb(102,0,102);font-size:100%">\startxmlsetups btx:demo
 
    \xmlcommand
 
        {#1}
 
        {/bibtex/entry[string.find(@tag,'Hagen')][1]}{btx:table}
 
\stopxmlsetups
 
\startxmlsetups btx:table
 
\starttabulate[|||]
 
    \NC \type {tag}  \NC \xmlatt{#1}{tag} \NC \NR
 
    \NC \type {title} \NC \xmlfirst{#1}{/field[@name='title']} \NC \NR
 
\stoptabulate
 
\stopxmlsetups
 
\xmlsetup{btx:tugboat}{btx:demo}
 
</pre>
 
   
 
<table>
 
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >tag</tt></td><td>Hagen:TB17-1-54</td></tr>
 
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >title</tt></td><td>PPCHTEX: typesetting chemical formulas in TEX</td></tr>
 
</table>
 
 
 
<p></p>
 
Here is another example:
 
   
 
<pre style="color:rgb(102,0,102);font-size:100%">\startxmlsetups btx:row
 
    \NC \xmlatt{#1}{tag}
 
    \NC \xmlfirst{#1}{/field[@name='title']}
 
    \NC \NR
 
\stopxmlsetups
 
\startxmlsetups btx:demo
 
    \xmlfilter {#1} {
 
        /bibtex
 
        /entry[@category='article']
 
        /field[@name='author' and (find(text(),'Knuth') or find(text(),'DEK'))]
 
        /../command(btx:row)
 
    }
 
\stopxmlsetups
 
\starttabulate[|||]
 
    \xmlsetup{btx:tugboat}{btx:demo}
 
\stoptabulate
 
</pre>
 
   
 
<table>
 
<tr><td>Knuth:TB10-1-31</td><td>Typesetting Concrete Mathematics</td></tr>
 
<tr><td>Knuth:TB10-1-8</td><td>TEX would find it difficult …</td></tr>
 
<tr><td>Knuth:TB10-3-325</td><td>The new versions of TEX and MF</td></tr>
 
<tr><td>Knuth:TB10-4-529</td><td>The errors of TEX</td></tr>
 
<tr><td>Knuth:TB11-1-13</td><td>Virtual Fonts: More Fun for Grand Wizards</td></tr>
 
<tr><td>Knuth:TB11-2-165</td><td>Exercises for TEX: The Program</td></tr>
 
<tr><td>Knuth:TB11-4-489</td><td>The future of TEX and MF</td></tr>
 
<tr><td>Knuth:TB11-4-497</td><td>Arthur Lee Samuel, 1901--1990</td></tr>
 
<tr><td>Knuth:TB11-4-499</td><td>Answers to Exercises for TEX: The Program</td></tr>
 
<tr><td>Knuth:TB12-2-313</td><td>Fixed-point glue setting: Errata</td></tr>
 
<tr><td>Knuth:TB14-4-387</td><td>Icons for TEX and MF</td></tr>
 
<tr><td>Knuth:TB17-1-29</td><td>Important message regarding CM fonts</td></tr>
 
<tr><td>Knuth:TB2-3-5</td><td>The current state of things</td></tr>
 
<tr><td>Knuth:TB3-1-10</td><td>Fixed-point glue settingDashan example of WEB</td></tr>
 
<tr><td>Knuth:TB31-2-121</td><td>An Earthshaking Announcement</td></tr>
 
<tr><td>Knuth:TB4-2-64</td><td>A note on hyphenation</td></tr>
 
<tr><td>Knuth:TB5-1-4</td><td>TEX incunabula</td></tr>
 
<tr><td>Knuth:TB5-1-67</td><td>Comments on quality in publishing</td></tr>
 
<tr><td>Knuth:TB5-2-105</td><td>A course on MF programming</td></tr>
 
<tr><td>Knuth:TB6-1-36</td><td>Recipes and fractions</td></tr>
 
<tr><td>Knuth:TB7-2-101</td><td>The TEX logo in various fonts</td></tr>
 
<tr><td>Knuth:TB7-2-95</td><td>Remarks to celebrate the publication of Computers & Typesetting</td></tr>
 
<tr><td>Knuth:TB8-1-14</td><td>Mixing right-to-left texts with left-to-right texts</td></tr>
 
<tr><td>Knuth:TB8-1-6</td><td>It happened: announcement of TEX 2.1</td></tr>
 
<tr><td>Knuth:TB8-1-73</td><td>Problem for a Saturday afternoon</td></tr>
 
<tr><td>Knuth:TB8-2-135</td><td>Fonts for digital halftones</td></tr>
 
<tr><td>Knuth:TB8-2-210</td><td>Saturday morning problemDashsolution</td></tr>
 
<tr><td>Knuth:TB8-2-217</td><td>Reply: Printing out selected pages</td></tr>
 
<tr><td>Knuth:TB8-3-309</td><td>Macros for Jill</td></tr>
 
<tr><td>Knuth:TB9-2-152</td><td>A Punk Meta-Font</td></tr>
 
</table>
 
 
 
<p></p>
 
A more extensive example is the following. Of course this assumes that you know what <span tag="MKIV" style="font-style:sans;">xml</span> support mechanisms and macros are available.
 
   
 
<pre style="color:rgb(102,0,102);font-size:100%">\startxmlsetups btx:getkeys
 
    \xmladdsortentry{btx}{#1}{\xmlfilter{#1}{/field[@name='author']/text()}}
 
    \xmladdsortentry{btx}{#1}{\xmlfilter{#1}{/field[@name='year'  ]/text()}}
 
    \xmladdsortentry{btx}{#1}{\xmlatt{#1}{tag}}
 
\stopxmlsetups
 
\startxmlsetups btx:sorter
 
    \xmlresetsorter{btx}
 
  % \xmlfilter{#1}{entry/command(btx:getkeys)}
 
    \xmlfilter{#1}{
 
        /bibtex
 
        /entry[@category='article']
 
        /field[@name='author' and find(text(),'Knuth')]
 
        /../command(btx:getkeys)}
 
    \xmlsortentries{btx}
 
    \starttabulate[||||]
 
        \xmlflushsorter{btx}{btx:entry:flush}
 
    \stoptabulate
 
\stopxmlsetups
 
\startxmlsetups btx:entry:flush
 
    \NC \xmlfilter{#1}{/field[@name='year'  ]/context()}
 
    \NC \xmlatt{#1}{tag}
 
    \NC \xmlfilter{#1}{/field[@name='author']/context()}
 
    \NC \NR
 
\stopxmlsetups
 
\xmlsetup{btx:tugboat}{btx:sorter}
 
</pre>
 
   
 
<table>
 
<tr><td>1984</td><td>Knuth:TB5-1-67</td><td>Don Knuth</td></tr>
 
<tr><td>1984</td><td>Knuth:TB5-1-4</td><td>Donald E. Knuth</td></tr>
 
<tr><td>1984</td><td>Knuth:TB5-2-105</td><td>Donald E. Knuth</td></tr>
 
<tr><td>1985</td><td>Knuth:TB6-1-36</td><td>Donald E. Knuth</td></tr>
 
<tr><td>1986</td><td>Knuth:TB7-2-101</td><td>Donald E. Knuth</td></tr>
 
<tr><td>1987</td><td>Knuth:TB8-2-135</td><td>Donald E. Knuth</td></tr>
 
<tr><td>1987</td><td>Knuth:TB8-3-309</td><td>Donald E. Knuth</td></tr>
 
<tr><td>1988</td><td>Knuth:TB9-2-152</td><td>Donald E. Knuth</td></tr>
 
<tr><td>1989</td><td>Knuth:TB10-3-325</td><td>Donald E. Knuth</td></tr>
 
<tr><td>1989</td><td>Knuth:TB10-4-529</td><td>Donald E. Knuth</td></tr>
 
<tr><td>1990</td><td>Knuth:TB11-4-489</td><td>Donald E. Knuth</td></tr>
 
<tr><td>1993</td><td>Knuth:TB14-4-387</td><td>Donald E. Knuth</td></tr>
 
<tr><td>1996</td><td>Knuth:TB17-1-29</td><td>Donald E. Knuth</td></tr>
 
<tr><td>1987</td><td>Knuth:TB8-1-14</td><td>Donald Knuth and Pierre MacKay</td></tr>
 
<tr><td>1981</td><td>Knuth:TB2-3-5</td><td>Donald Knuth</td></tr>
 
<tr><td>1982</td><td>Knuth:TB3-1-10</td><td>Donald Knuth</td></tr>
 
<tr><td>1983</td><td>Knuth:TB4-2-64</td><td>Donald Knuth</td></tr>
 
<tr><td>1986</td><td>Knuth:TB7-2-95</td><td>Donald Knuth</td></tr>
 
<tr><td>1987</td><td>Knuth:TB8-1-6</td><td>Donald Knuth</td></tr>
 
<tr><td>1987</td><td>Knuth:TB8-1-73</td><td>Donald Knuth</td></tr>
 
<tr><td>1987</td><td>Knuth:TB8-2-210</td><td>Donald Knuth</td></tr>
 
<tr><td>1987</td><td>Knuth:TB8-2-217</td><td>Donald Knuth</td></tr>
 
<tr><td>1989</td><td>Knuth:TB10-1-8</td><td>Donald Knuth</td></tr>
 
<tr><td>1989</td><td>Knuth:TB10-1-31</td><td>Donald Knuth</td></tr>
 
<tr><td>1990</td><td>Knuth:TB11-1-13</td><td>Donald Knuth</td></tr>
 
<tr><td>1990</td><td>Knuth:TB11-2-165</td><td>Donald Knuth</td></tr>
 
<tr><td>1990</td><td>Knuth:TB11-4-497</td><td>Donald Knuth</td></tr>
 
<tr><td>1990</td><td>Knuth:TB11-4-499</td><td>Donald Knuth</td></tr>
 
<tr><td>1991</td><td>Knuth:TB12-2-313</td><td>Donald Knuth</td></tr>
 
<tr><td>2010</td><td>Knuth:TB31-2-121</td><td>Donald Knuth</td></tr>
 
</table>
 
 
 
<p></p>
 
The original data is stored in a <span tag="MKIV" style="font-style:sans;">Lua</span> table, hashed by tag. Starting with <span tag="MKIV" style="font-style:sans;">Lua</span> 5.2 each run of <span tag="MKIV" style="font-style:sans;">Lua</span> gets a different ordering of such a hash. In older versions, when you looped over a hash, the order was undefined, but the same as long as you used the same binary. This had the advantage that successive runs, something we often have in document processing gave consistent results. In today’s <span tag="MKIV" style="font-style:sans;">Lua</span> we need to do much more sorting of hashes before we loop, especially when we save multi--pass data. It is for this reason that the <span tag="MKIV" style="font-style:sans;">xml</span> tree is sorted by hash key by default. That way lookups (especially the first of a set) give consistent outcomes.
 
 
 
 
 
<h1>Standards</h1>
 
 
 
The rendering of bibliographic entries is often standardized and prescribed by the publisher. If you submit an article to a journal, normally it will be reformatted (or even re- keyed) and the rendering will happen at the publishers end. In that case it may not matter how entries were rendered when writing the publication, because the publisher will do it his or her way. This means that most users probably will stick to the standard <span tag="MKIV" style="font-style:sans;">apa</span> rules and for them we provide some configuration. Because we use setups it is easy to overload specifics. If you really want to tweak, best look in the files that deal with it.
 
<p></p>
 
Many standards exist and support for other renderings may be added to the core. Interested users are invited to develop and to test alternate standard renderings according to their needs.
 
<p></p>
 
Todo: maybe a list of categories and fields.
 
 
 
 
 
<h1>Cleaning up</h1>
 
 
 
Although the <span tag="MKIV" style="font-style:sans;">bibTEX</span> format is reasonably well defined, in practice there are many ways to organize the data. For instance, one can use predefined string constants that get used (either or not combined with other strings) later on. A string can be enclosed in curly braces or double quotes. The strings can contain <span tag="MKIV" style="font-style:sans;">TEX</span> commands but these are not standardized. The databases often have somewhat complex ways to deal with special characters and the use of braces in their definition is also not normalized.
 
<p></p>
 
The most complex to deal with are the fields that contain names of people. At some point it might be needed to split a combination of names into individual ones that then get split into title, first name, optional inbetweens, surname(s) and additional: <tt style="color:rgb(0,102,102);font-size:100%;" >Prof. Dr. Alfred B. C. von Kwik Kwak Jr. II and P. Q. Olet</tt> is just one example of this. The convention seems to be not to use commas but <tt style="color:rgb(0,102,102);font-size:100%;" >and</tt> to separate names (often each name will be specified as lastname, firstname).
 
<p></p>
 
We don’t see it as challenge nor as a duty to support all kinds of messy definitions. Of course we try to be somewhat tolerant, but you will be sure to get better results if you use nicely setup, consistent databases.
 
<p></p>
 
Todo: maybe some examples of bad.
 
 
 
 
 
<h1>Transition</h1>
 
 
 
In the original bibliography support module usage was as follows (example taken from the contextgarden wiki):
 
   
 
<pre style="color:rgb(102,0,102);font-size:100%">% engine=pdftex
 
\usemodule[bib]
 
\usemodule[bibltx]
 
\setupbibtex
 
  [database=xampl]
 
\setuppublications
 
  [numbering=yes]
 
\starttext
 
    As \cite [article-full] already indicated, bibtex is a \LATEX||centric
 
    program.
 
    \completepublications
 
\stoptext
 
</pre>
 
<p></p>
 
For <span tag="MKIV" style="font-style:sans;">MkIV</span> the modules were partly rewritten and ended up in the core so the two commands were no longer needed. The overhead associated with the automatic loading of the bibliography macros can be neglected these days, so standardized modules such as <tt style="color:rgb(0,102,102);font-size:100%;" >bib</tt> are all being moved to the core and do not need to be explicitly loaded.
 
<p></p>
 
The first <tt style="color:rgb(0,102,102);font-size:100%;" >\setupbibtex</tt> command in this example is needed to bootstrap the process: it tells what database has to be processed by <span tag="MKIV" style="font-style:sans;">bibTEX</span> between runs. The second <tt style="color:rgb(0,102,102);font-size:100%;" >\setuppublications</tt> command is optional. Each citation (tagged with <tt style="color:rgb(0,102,102);font-size:100%;" >\cite</tt>) ends up in the list of publications.
 
<p></p>
 
In the new approach we no longer use <span tag="MKIV" style="font-style:sans;">bibTEX</span>so we don’t need to setup <span tag="MKIV" style="font-style:sans;">bibTEX</span>. Instead we define dataset(s). We also no longer set up publications with one command, but have split that up in rendering-, list-, and cite-variants. The basic <tt style="color:rgb(0,102,102);font-size:100%;" >\cite</tt> command remains. The above example becomes:
 
   
 
<pre style="color:rgb(102,0,102);font-size:100%">\definebtxdataset
 
  [document]
 
\usebtxdataset
 
  [document]
 
  [mybibfile.bib]
 
\definebtxrendering
 
  [document]
 
\setupbtxrendering
 
  [document]
 
  [numbering=yes]
 
\starttext
 
    As \cite [article-full] already indicated, bibtex is a \LATEX||centric
 
    program.
 
    \completebtxrendering[document]
 
\stoptext
 
</pre>
 
<p></p>
 
So, we have a few more commands to set up things. If you intend to use just a single dataset and rendering, the above preamble can be simplified to:
 
   
 
<pre style="color:rgb(102,0,102);font-size:100%">\usebtxdataset
 
  [mybibfile.bib]
 
\setupbtxrendering
 
  [numbering=yes]
 
</pre>
 
<p></p>
 
But keep in mind that compared to the old <span tag="MKIV" style="font-style:sans;">MkII</span> derived method we have moved some of the options to the rendering, list and cite setup variants.
 
<p></p>
 
Another difference is now the use of lists. When you define a rendering, you also define a list. However, all entries are collected in a common list tagged <tt style="color:rgb(0,102,102);font-size:100%;" >btx</tt>. Although you will normally configure a rendering you can still set some properties of lists, but in that case you need to prefix the list identifier. In the case of the above example this is <tt style="color:rgb(0,102,102);font-size:100%;" >btx:document</tt>.
 
 
 
 
 
<h1>MLBIBTEX</h1>
 
 
 
Todo: how to plug in <span tag="MKIV" style="font-style:sans;">MLbibTEX</span> for sorting and other advanced operations.
 
 
 
 
 
<h1>Extensions</h1>
 
 
 
As <span tag="MKIV" style="font-style:sans;">TEX</span> and <span tag="MKIV" style="font-style:sans;">Lua</span> are both open and accessible in <span tag="MKIV" style="font-style:sans;">ConTEXt</span> it is possible to extend the functionality of the bibliography related code. For instance, you can add extra loaders.
 
   
 
<pre style="color:rgb(102,0,102);font-size:100%">function publications.loaders.myformat(dataset,filename)
 
    local t = { }
 
    -- Load data from 'filename' and convert it to a Lua table 't' with
 
    -- the key as hash entry and fields conforming the luadata table
 
    -- format.
 
    loaders.lua(dataset,t)
 
end
 
</pre>
 
<p></p>
 
This then permits loading a database (into a dataset) with the command:
 
   
 
<pre style="color:rgb(102,0,102);font-size:100%">\usebtxdataset[standard][myfile.myformat]
 
</pre>
 
<p></p>
 
The <tt style="color:rgb(0,102,102);font-size:100%;" >myformat</tt> suffix is recognized automatically. If you want to use another suffix, you can do this:
 
   
 
<pre style="color:rgb(102,0,102);font-size:100%">\usebtxdataset[standard][myformat::myfile.txt]
 
</pre>
 
 
 
 
 
<h1>Notes</h1>
 
 
 
The move from external <span tag="MKIV" style="font-style:sans;">bibTEX</span> processing to internal processing has the advantage that we stay within the same run. In the traditional approach we had roughly the following steps:
 
   
 
<div>
 
<div><span>• </span><span>the first run information is collected and written to file</span></div>
 
<div><span>• </span><span>after that run the <span tag="MKIV" style="font-style:sans;">bibTEX</span> program converts that file to another one</span></div>
 
<div><span>• </span><span>successive runs use that data for references and producing lists</span></div>
 
</div>
 
 
 
<p></p>
 
In the <span tag="MKIV" style="font-style:sans;">MkIV</span> approach the bibliographic database is loaded in memory each run and processing also happens each run. On paper this looks less efficient but as <span tag="MKIV" style="font-style:sans;">Lua</span> is quite fast, in practice performance is much better.
 
<p></p>
 
Probably most demanding is the treatment of authors as we have to analyze names, split multiple authors and reassemble firstnames, vons, surnames and juniors. When we sort by author sorting vectors have to be made which also has a penalty. However, in practice the user will not notice a performance degradation. We did some tests with a list of 500.000 authors, sorted them and typeset them as list (producing some 5400 dense pages in a small font and with small margins). This is typical one of these cases where using <span tag="MKIV" style="font-style:sans;">LuajitTEX</span> saves quite time. On my machine it took just over 100 seconds to get this done. Unfortunately not all operating systems performed equally well: 32 bit versions worked fine, but 64 bit <span tag="MKIV" style="font-style:sans;">linux</span> either crashed (stalled) the machine or ran out of memory rather fast, while <span tag="MKIV" style="font-style:sans;">MacOSX</span> and <span tag="MKIV" style="font-style:sans;">Windows</span> performed fine. In practice you will never run into this, unless you produce massive amounts of bibliographic entries. <span tag="MKIV" style="font-style:sans;">LuaJIT</span> has some benefits but also some drawbacks.
 

Latest revision as of 08:13, 13 February 2018

Some materials

Npp for ConTeXt

This is a notepad++ plugin for ConTeXt. Original work by Jason Yu, current developer Luigi Scarso.

The plugin is available here: File:Npp-for-ConTeXt.zip. Manual under doc/npp-context-manual.pdf of the archive.