Difference between revisions of "User:Luigi.scarso"

From Wiki
Jump to navigation Jump to search
Line 1: Line 1:
__TOC__
+
<h1>The database</h1>
  
 
+
The <span tag="MKIV" style="font-style:sans;">bibTEX</span> format is rather popular in the <span tag="MKIV" style="font-style:sans;">TEX</span> community and even with its shortcomings it will stay around for a while. Many publication websites can export and many tools are available to work with this database format. It is rather simple and looks a bit like <span tag="MKIV" style="font-style:sans;">Lua</span> tables. Unfortunately the content can be polluted with non-standardized <span tag="MKIV" style="font-style:sans;">TEX</span> commands which complicates pre- or postprocessing outside <span tag="MKIV" style="font-style:sans;">TEX</span>. In that sense a <span tag="MKIV" style="font-style:sans;">bibTEX</span> database is often not coded neutrally. Some limitations, like the use of commands to encode accented characters root in the <span tag="MKIV" style="font-style:sans;">ascii</span> world and can be bypassed by using <span tag="MKIV" style="font-style:sans;">utf</span> instead (as handled somewhat in <span tag="MKIV" style="font-style:sans;">LATEX</span> through extensions such as <tt style="color:rgb(0,102,102);font-size:100%;" >bibtex8</tt>).
 
 
 
 
 
 
   
 
<h1>The data­base</h1>
 
 
 
The <span tag="MKIV" style="font-style:sans;">bibTEX</span> for­mat is rather pop­u­lar in the <span tag="MKIV" style="font-style:sans;">TEX</span> com­mu­nity and even with its short­com­ings it will stay around for a while. Many pub­li­ca­tion web­sites can ex­port and many tools are avail­able to work with this data­base for­mat. It is rather sim­ple and looks a bit like <span tag="MKIV" style="font-style:sans;">Lua</span> ta­bles. Un­for­tu­nately the con­tent can be pol­luted with non-stan­dard­ized <span tag="MKIV" style="font-style:sans;">TEX</span> com­mands which com­pli­cates pre- or post­pro­cess­ing out­side <span tag="MKIV" style="font-style:sans;">TEX</span>. In that sense a <span tag="MKIV" style="font-style:sans;">bibTEX</span> data­base is of­ten not coded neu­trally. Some lim­i­ta­tions, like the use of com­mands to en­code ac­cented char­ac­ters root in the <span tag="MKIV" style="font-style:sans;">ascii</span> world and can be by­passed by us­ing <span tag="MKIV" style="font-style:sans;">utf</span> in­stead (as han­dled some­what in <span tag="MKIV" style="font-style:sans;">LATEX</span> through ex­ten­sions such as <tt style="color:rgb(0,102,102);font-size:100%;" >bibtex8</tt>).
 
 
<p></p>
 
<p></p>
The nor­mal way to deal with a bib­li­og­ra­phy is to re­fer to en­tries us­ing a unique tag or key. When a list of en­tries is type­set, this ref­er­ence can be used for link­ing pur­poses. The type­set list can be processed and sorted us­ing the <tt style="color:rgb(0,102,102);font-size:100%;" >bibtex</tt> pro­gram that con­verts the data­base into some­thing more <span tag="MKIV" style="font-style:sans;">TEX</span> friendly (a <tt style="color:rgb(0,102,102);font-size:100%;" >.bbl</tt> file). I never used the pro­gram my­self (nor bib­li­ogra­phies) so I will not go into too much de­tail here, if only be­cause all I say can be wrong.
+
The normal way to deal with a bibliography is to refer to entries using a unique tag or key. When a list of entries is typeset, this reference can be used for linking purposes. The typeset list can be processed and sorted using the <tt style="color:rgb(0,102,102);font-size:100%;" >bibtex</tt> program that converts the database into something more <span tag="MKIV" style="font-style:sans;">TEX</span> friendly (a <tt style="color:rgb(0,102,102);font-size:100%;" >.bbl</tt> file). I never used the program myself (nor bibliographies) so I will not go into too much detail here, if only because all I say can be wrong.
 
<p></p>
 
<p></p>
In <span tag="MKIV" style="font-style:sans;">ConTEXt</span> we no longer use the <tt style="color:rgb(0,102,102);font-size:100%;" >bibtex</tt> pro­gram: we just use data­base files and deal with the nec­es­sary ma­nip­u­la­tions di­rectly in <span tag="MKIV" style="font-style:sans;">ConTEXt</span>. One or more such data­bases can be used and com­bined with ad­di­tional en­tries de­fined within the doc­u­ment. We can have sev­eral such datasets ac­tive at the same time.
+
In <span tag="MKIV" style="font-style:sans;">ConTEXt</span> we no longer use the <tt style="color:rgb(0,102,102);font-size:100%;" >bibtex</tt> program: we just use database files and deal with the necessary manipulations directly in <span tag="MKIV" style="font-style:sans;">ConTEXt</span>. One or more such databases can be used and combined with additional entries defined within the document. We can have several such datasets active at the same time.
 
<p></p>
 
<p></p>
 
A <span tag="MKIV" style="font-style:sans;">bibTEX</span> file looks like this:  
 
A <span tag="MKIV" style="font-style:sans;">bibTEX</span> file looks like this:  
Line 29: Line 22:
 
</pre>
 
</pre>
 
<p></p>
 
<p></p>
Nor­mally a value is given be­tween quotes (or curly brack­ets) but sin­gle words are also OK (there is no real ben­e­fit in not us­ing quotes, so we ad­vise to al­ways use them). There can be many more fields and in­stead of strings one can use pre­de­fined short­cuts. The ti­tle for ex­am­ple quite of­ten con­tains <span tag="MKIV" style="font-style:sans;">TEX</span> macros. Some fields, like <tt style="color:rgb(0,102,102);font-size:100%;" >pages</tt> have funny char­ac­ters such as the en­dash (typ­i­cally as <tt style="color:rgb(0,102,102);font-size:100%;" >--</tt>) so we have a mix­ture of data and type­set­ting di­rec­tives. If you are cov­er­ing non--eng­lish ref­er­ences, you of­ten need char­ac­ters that are not in the <span tag="MKIV" style="font-style:sans;">ascii</span> sub­set but <span tag="MKIV" style="font-style:sans;">ConTEXt</span> is quite happy with <span tag="MKIV" style="font-style:sans;">utf</span>. If your data­base file uses old-fash­ioned <span tag="MKIV" style="font-style:sans;">TEX</span> ac­cent com­mands then these will be in­ter­nally con­verted au­to­mat­i­cally to <span tag="MKIV" style="font-style:sans;">utf</span>. Com­mands (macros) are con­verted to an in­di­rect call, which is quite ro­bust.
+
Normally a value is given between quotes (or curly brackets) but single words are also OK (there is no real benefit in not using quotes, so we advise to always use them). There can be many more fields and instead of strings one can use predefined shortcuts. The title for example quite often contains <span tag="MKIV" style="font-style:sans;">TEX</span> macros. Some fields, like <tt style="color:rgb(0,102,102);font-size:100%;" >pages</tt> have funny characters such as the endash (typically as <tt style="color:rgb(0,102,102);font-size:100%;" >--</tt>) so we have a mixture of data and typesetting directives. If you are covering non--english references, you often need characters that are not in the <span tag="MKIV" style="font-style:sans;">ascii</span> subset but <span tag="MKIV" style="font-style:sans;">ConTEXt</span> is quite happy with <span tag="MKIV" style="font-style:sans;">utf</span>. If your database file uses old-fashioned <span tag="MKIV" style="font-style:sans;">TEX</span> accent commands then these will be internally converted automatically to <span tag="MKIV" style="font-style:sans;">utf</span>. Commands (macros) are converted to an indirect call, which is quite robust.
 
<p></p>
 
<p></p>
The <span tag="MKIV" style="font-style:sans;">bibTEX</span> files are loaded in mem­ory as <span tag="MKIV" style="font-style:sans;">Lua</span> ta­ble but can be con­verted to <span tag="MKIV" style="font-style:sans;">xml</span> so that we can ac­cess them in a more flex­i­ble way, but that is a sub­ject for spe­cial­ists.
+
The <span tag="MKIV" style="font-style:sans;">bibTEX</span> files are loaded in memory as <span tag="MKIV" style="font-style:sans;">Lua</span> table but can be converted to <span tag="MKIV" style="font-style:sans;">xml</span> so that we can access them in a more flexible way, but that is a subject for specialists.
 
<p></p>
 
<p></p>
In the old <span tag="MKIV" style="font-style:sans;">MkII</span> setup we have two kinds of en­tries: the ones that come from the <span tag="MKIV" style="font-style:sans;">bibTEX</span> run and user sup­plied ones. We no longer rely on <span tag="MKIV" style="font-style:sans;">bibTEX</span> out­put but we do still sup­port the user sup­plied de­f­i­n­i­tions. These were in fact pre­pared in a way that suits the pro­cess­ing of <span tag="MKIV" style="font-style:sans;">bibTEX</span> gen­er­ated en­tries. The next vari­ant re­flects the <span tag="MKIV" style="font-style:sans;">ConTEXt</span> re­cod­ing of the old <span tag="MKIV" style="font-style:sans;">bibTEX</span> out­put.  
+
In the old <span tag="MKIV" style="font-style:sans;">MkII</span> setup we have two kinds of entries: the ones that come from the <span tag="MKIV" style="font-style:sans;">bibTEX</span> run and user supplied ones. We no longer rely on <span tag="MKIV" style="font-style:sans;">bibTEX</span> output but we do still support the user supplied definitions. These were in fact prepared in a way that suits the processing of <span tag="MKIV" style="font-style:sans;">bibTEX</span> generated entries. The next variant reflects the <span tag="MKIV" style="font-style:sans;">ConTEXt</span> recoding of the old <span tag="MKIV" style="font-style:sans;">bibTEX</span> output.  
 
      
 
      
 
<pre style="color:rgb(102,0,102);font-size:100%">\startpublication[k=Hagen:Second,t=article,a={Hans Hagen},y=2013,s=HH01]
 
<pre style="color:rgb(102,0,102);font-size:100%">\startpublication[k=Hagen:Second,t=article,a={Hans Hagen},y=2013,s=HH01]
Line 48: Line 41:
 
</pre>
 
</pre>
 
<p></p>
 
<p></p>
The split <tt style="color:rgb(0,102,102);font-size:100%;" >\artauthor</tt> fields are col­lapsed into a sin­gle <tt style="color:rgb(0,102,102);font-size:100%;" >author</tt> field as we deal with the split­ting later when it gets parsed in <span tag="MKIV" style="font-style:sans;">Lua</span>. The <tt style="color:rgb(0,102,102);font-size:100%;" >\artauthor</tt> syn­tax is only kept around for back­ward com­pat­i­bil­ity with the pre­vi­ous use of <span tag="MKIV" style="font-style:sans;">bibTEX</span>.
+
The split <tt style="color:rgb(0,102,102);font-size:100%;" >\artauthor</tt> fields are collapsed into a single <tt style="color:rgb(0,102,102);font-size:100%;" >author</tt> field as we deal with the splitting later when it gets parsed in <span tag="MKIV" style="font-style:sans;">Lua</span>. The <tt style="color:rgb(0,102,102);font-size:100%;" >\artauthor</tt> syntax is only kept around for backward compatibility with the previous use of <span tag="MKIV" style="font-style:sans;">bibTEX</span>.
 
<p></p>
 
<p></p>
In the new setup we sup­port these vari­ants as well:  
+
In the new setup we support these variants as well:  
 
      
 
      
 
<pre style="color:rgb(102,0,102);font-size:100%">\startpublication[k=Hagen:Third,t=article]
 
<pre style="color:rgb(102,0,102);font-size:100%">\startpublication[k=Hagen:Third,t=article]
Line 79: Line 72:
 
</pre>
 
</pre>
 
<p></p>
 
<p></p>
Be­cause in­ter­nally the en­tries are <span tag="MKIV" style="font-style:sans;">Lua</span> ta­bles, we also sup­port load­ing of <span tag="MKIV" style="font-style:sans;">Lua</span> based de­f­i­n­i­tions:
+
Because internally the entries are <span tag="MKIV" style="font-style:sans;">Lua</span> tables, we also support loading of <span tag="MKIV" style="font-style:sans;">Lua</span> based definitions:
 
      
 
      
 
<pre style="color:rgb(102,0,102);font-size:100%">return {
 
<pre style="color:rgb(102,0,102);font-size:100%">return {
Line 98: Line 91:
 
</pre>
 
</pre>
 
<p></p>
 
<p></p>
Files set up like this can be loaded too. The fol­low­ing <span tag="MKIV" style="font-style:sans;">xml</span> in­put is rather close to this, and is also ac­cepted as in­put.  
+
Files set up like this can be loaded too. The following <span tag="MKIV" style="font-style:sans;">xml</span> input is rather close to this, and is also accepted as input.  
 
      
 
      
 
<pre style="color:rgb(102,0,102);font-size:100%"><?xml version="2.0" standalone="yes" ?>
 
<pre style="color:rgb(102,0,102);font-size:100%"><?xml version="2.0" standalone="yes" ?>
Line 118: Line 111:
 
</pre>
 
</pre>
 
<p></p>
 
<p></p>
Todo: Add some re­marks about load­ing End­Note and RIS for­mats, but first we need to com­plete the tag map­ping (on Alan’s plate).
+
Todo: Add some remarks about loading EndNote and RIS formats, but first we need to complete the tag mapping (on Alan’s plate).
 
<p></p>
 
<p></p>
So the user has a rather wide choice of for­mat­ting style for bib­li­og­ra­phy data­base files.
+
So the user has a rather wide choice of formatting style for bibliography database files.
 
      
 
      
You can load more data than you ac­tu­ally need. Only en­tries that are re­ferred to ex­plic­itly through the <tt style="color:rgb(0,102,102);font-size:100%;" >\cite</tt> and <tt style="color:rgb(0,102,102);font-size:100%;" >\nocite</tt> com­mands will be shown in lists. We will cover these de­tails later.
+
You can load more data than you actually need. Only entries that are referred to explicitly through the <tt style="color:rgb(0,102,102);font-size:100%;" >\cite</tt> and <tt style="color:rgb(0,102,102);font-size:100%;" >\nocite</tt> commands will be shown in lists. We will cover these details later.
 
    
 
    
<h1>Com­mands in en­tries</h1>
+
<h1>Commands in entries</h1>
  
One un­for­tu­nate as­pect com­monly found in <span tag="MKIV" style="font-style:sans;">bibTEX</span> files is that they of­ten con­tain <span tag="MKIV" style="font-style:sans;">TEX</span> com­mands. Even worse is that there is no stan­dard on what these com­mands can be and what they mean, at least not for­mally, as <span tag="MKIV" style="font-style:sans;">bibTEX</span> is a pro­gram in­tended to be used with many vari­ants of <span tag="MKIV" style="font-style:sans;">TEX</span> style: plain, <span tag="MKIV" style="font-style:sans;">LATEX</span>, and oth­ers. This means that we need to de­fine our use of these type­set­ting com­mands. How­ever, in most cases, they are just ab­bre­vi­a­tions or font switches and these are of­ten known. There­fore, <span tag="MKIV" style="font-style:sans;">ConTEXt</span> will try to re­solve them be­fore re­port­ing an is­sue. In the log file there is a list of com­mands that has been seen in the loaded data­bases. For in­stance, load­ing <tt style="color:rgb(0,102,102);font-size:100%;" >tugboat.bib</tt> gives a long list of com­mands of which we show a small set here:  
+
One unfortunate aspect commonly found in <span tag="MKIV" style="font-style:sans;">bibTEX</span> files is that they often contain <span tag="MKIV" style="font-style:sans;">TEX</span> commands. Even worse is that there is no standard on what these commands can be and what they mean, at least not formally, as <span tag="MKIV" style="font-style:sans;">bibTEX</span> is a program intended to be used with many variants of <span tag="MKIV" style="font-style:sans;">TEX</span> style: plain, <span tag="MKIV" style="font-style:sans;">LATEX</span>, and others. This means that we need to define our use of these typesetting commands. However, in most cases, they are just abbreviations or font switches and these are often known. Therefore, <span tag="MKIV" style="font-style:sans;">ConTEXt</span> will try to resolve them before reporting an issue. In the log file there is a list of commands that has been seen in the loaded databases. For instance, loading <tt style="color:rgb(0,102,102);font-size:100%;" >tugboat.bib</tt> gives a long list of commands of which we show a small set here:  
 
      
 
      
 
<pre style="color:rgb(102,0,102);font-size:100%">publications > start used btx commands
 
<pre style="color:rgb(102,0,102);font-size:100%">publications > start used btx commands
Line 138: Line 131:
 
</pre>
 
</pre>
 
<p></p>
 
<p></p>
You can de­fine un­known com­mands, or over­load ex­ist­ing de­f­i­n­i­tions in the fol­low­ing way:  
+
You can define unknown commands, or overload existing definitions in the following way:  
 
      
 
      
 
<pre style="color:rgb(102,0,102);font-size:100%">\definebtxcommand\TUB {TUGboat}
 
<pre style="color:rgb(102,0,102);font-size:100%">\definebtxcommand\TUB {TUGboat}
Line 145: Line 138:
 
</pre>
 
</pre>
 
<p></p>
 
<p></p>
Un­known com­mands do not stall pro­cess­ing, but their names are then type­set in a mono- spaced font so they prob­a­bly stand out for proof­read­ing. You can ac­cess the com­mands with <tt style="color:rgb(0,102,102);font-size:100%;" >\btxcommand{...}</tt>, as in:  
+
Unknown commands do not stall processing, but their names are then typeset in a mono- spaced font so they probably stand out for proofreading. You can access the commands with <tt style="color:rgb(0,102,102);font-size:100%;" >\btxcommand{...}</tt>, as in:  
 
      
 
      
 
<pre style="color:rgb(102,0,102);font-size:100%">commands like \btxcommand{MySpecialCommand} are handled in an indirect way
 
<pre style="color:rgb(102,0,102);font-size:100%">commands like \btxcommand{MySpecialCommand} are handled in an indirect way
 
</pre>
 
</pre>
 
<p></p>
 
<p></p>
As this is an un­de­fined com­mand we get: “com­mands like MySpe­cial­Com­mand are han­dled in an in­di­rect way”.
+
As this is an undefined command we get: “commands like MySpecialCommand are handled in an indirect way”.
 
<p></p>
 
<p></p>
 
??
 
??
Line 157: Line 150:
 
<h1>Datasets</h1>
 
<h1>Datasets</h1>
  
Nor­mally in a doc­u­ment you will use only one bib­li­o­graphic data­base, whether or not dis­trib­uted over mul­ti­ple files. Nev­er­the­less we sup­port mul­ti­ple data­bases as well which is why we talk of datasets in­stead. A dataset is loaded with the <tt style="color:rgb(0,102,102);font-size:100%;" >\usebtxdataset</tt> com­mand. Al­though cur­rently it is not nec­es­sary to de­fine a (de­fault) dataset you can best do this be­cause in the fu­ture we might pro­vide more op­tions. Here are some ex­am­ples:  
+
Normally in a document you will use only one bibliographic database, whether or not distributed over multiple files. Nevertheless we support multiple databases as well which is why we talk of datasets instead. A dataset is loaded with the <tt style="color:rgb(0,102,102);font-size:100%;" >\usebtxdataset</tt> command. Although currently it is not necessary to define a (default) dataset you can best do this because in the future we might provide more options. Here are some examples:  
 
      
 
      
 
<pre style="color:rgb(102,0,102);font-size:100%">\definebtxdataset[standard]
 
<pre style="color:rgb(102,0,102);font-size:100%">\definebtxdataset[standard]
Line 165: Line 158:
 
</pre>
 
</pre>
 
<p></p>
 
<p></p>
These three suf­fixes are un­der­stood by the loader. Here the dataset has the name <tt style="color:rgb(0,102,102);font-size:100%;" >standard</tt> and the three data­base files are merged, where later en­tries hav­ing the same tag over­load pre­vi­ous ones. De­f­i­n­i­tions in the doc­u­ment source (coded in <span tag="MKIV" style="font-style:sans;">TEX</span> speak) are also added, and they are saved for suc­ces­sive runs. This means that if you load and de­fine en­tries, they will be known at a next run be­fore­hand, so that ref­er­ences to them are in­de­pen­dent of when load­ing and de­f­i­n­i­tions take place.
+
These three suffixes are understood by the loader. Here the dataset has the name <tt style="color:rgb(0,102,102);font-size:100%;" >standard</tt> and the three database files are merged, where later entries having the same tag overload previous ones. Definitions in the document source (coded in <span tag="MKIV" style="font-style:sans;">TEX</span> speak) are also added, and they are saved for successive runs. This means that if you load and define entries, they will be known at a next run beforehand, so that references to them are independent of when loading and definitions take place.
 
<div style="border:thin solid black;" >
 
<div style="border:thin solid black;" >
 
<span style="font-style:oblique;" > setup definition setupbtxdataset </span >
 
<span style="font-style:oblique;" > setup definition setupbtxdataset </span >
Line 176: Line 169:
 
</div>
 
</div>
 
<p></p>
 
<p></p>
In this doc­u­ment we use some ex­am­ple data­bases, so let’s load one of them now:  
+
In this document we use some example databases, so let’s load one of them now:  
 
      
 
      
 
<pre style="color:rgb(102,0,102);font-size:100%">\definebtxdataset[example]
 
<pre style="color:rgb(102,0,102);font-size:100%">\definebtxdataset[example]
Line 182: Line 175:
 
</pre>
 
</pre>
 
<p></p>
 
<p></p>
You can ask for an overview of en­tries in a dataset with:  
+
You can ask for an overview of entries in a dataset with:  
 
      
 
      
 
<pre style="color:rgb(102,0,102);font-size:100%">\showbtxdatasetfields[example]
 
<pre style="color:rgb(102,0,102);font-size:100%">\showbtxdatasetfields[example]
 
</pre>
 
</pre>
this gives:  
+
<p></p>
 +
this gives:
 
      
 
      
 
<table>
 
<table>
<tr><td>tag</td><td>cat­e­gory</td><td>fields</td></tr>
+
<tr><td>tag</td><td>category</td><td>fields</td></tr>
<tr><td>demo-001</td><td>book</td><td>au­thor in­dex ti­tle year</td></tr>
+
<tr><td>demo-001</td><td>book</td><td>author index title year</td></tr>
<tr><td>demo-002</td><td>book</td><td>cross­ref in­dex year</td></tr>
+
<tr><td>demo-002</td><td>book</td><td>crossref index year</td></tr>
<tr><td>demo-003</td><td>book</td><td>au­thor com­ment in­dex ti­tle year</td></tr>
+
<tr><td>demo-003</td><td>book</td><td>author comment index title year</td></tr>
<tr><td>demo-004</td><td>book</td><td>au­thor com­ment in­dex ti­tle year</td></tr>
+
<tr><td>demo-004</td><td>book</td><td>author comment index title year</td></tr>
<tr><td>demo-005</td><td>book</td><td>au­thor doi in­dex pages se­r­ial ti­tle url year</td></tr>
+
<tr><td>demo-005</td><td>book</td><td>author doi index pages serial title url year</td></tr>
 
</table>
 
</table>
  
 
<p></p>
 
<p></p>
You can set the cur­rent ac­tive dataset with  
+
You can set the current active dataset with  
 
      
 
      
 
<pre style="color:rgb(102,0,102);font-size:100%">\setbtxdataset[standard]
 
<pre style="color:rgb(102,0,102);font-size:100%">\setbtxdataset[standard]
 
</pre>
 
</pre>
 
<p></p>
 
<p></p>
but most pub­li­ca­tion-re­lated com­mands ac­cept op­tional ar­gu­ments that de­note the dataset and ref­er­ences to en­tries can be pre­fixed with a dataset iden­ti­fier.. More about that later.
+
but most publication-related commands accept optional arguments that denote the dataset and references to entries can be prefixed with a dataset identifier.. More about that later.
 +
<p></p>
 +
Sometimes you want to check a database. One way of doing that is the following:
 +
   
 +
<pre style="color:rgb(102,0,102);font-size:100%">\definebtxdataset[check]
 +
\usebtxdataset[check][mkiv-publications-check.bib]
 +
\showbtxdatasetcompleteness[check]
 +
</pre>
 +
<p></p>
 +
The database like like this:
 +
<p></p>
 +
The completeness check shows (with green field names) the required fields and when one is missing this is indicated in red. In blue we show what gets inherited.
 
      
 
      
 
      
 
      
<h1>Ren­der­ings</h1>
+
<h1>Renderings</h1>
  
A list of pub­li­ca­tions can be ren­dered at any place in the doc­u­ment. A data­base can be much larger than needed for a doc­u­ment. The same is true for the fields that make up an en­try. Here is the list of fields that are cur­rently han­dled, but of course there can be ad­di­tional ones:
+
A list of publications can be rendered at any place in the document. A database can be much larger than needed for a document. The same is true for the fields that make up an entry. Here is the list of fields that are currently handled, but of course there can be additional ones:
 
<p></p>
 
<p></p>
 
<tt style="color:rgb(0,102,102);font-size:100%;" >abstract</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >address</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >annotate</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >assignee</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >author</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >bibnumber</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >booktitle</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >chapter</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >comment</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >country</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >day</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >dayfiled</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >doi</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >edition</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >editor</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >eprint</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >howpublished</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >institution</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >isbn</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >issn</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >journal</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >key</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >keyword</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >keywords</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >language</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >lastchecked</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >month</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >monthfiled</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >names</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >nationality</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >note</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >notes</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >number</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >organization</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >pages</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >publisher</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >revision</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >school</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >series</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >size</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >title</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >type</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >url</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >volume</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >year</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >yearfiled</tt>
 
<tt style="color:rgb(0,102,102);font-size:100%;" >abstract</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >address</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >annotate</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >assignee</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >author</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >bibnumber</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >booktitle</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >chapter</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >comment</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >country</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >day</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >dayfiled</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >doi</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >edition</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >editor</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >eprint</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >howpublished</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >institution</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >isbn</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >issn</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >journal</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >key</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >keyword</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >keywords</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >language</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >lastchecked</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >month</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >monthfiled</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >names</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >nationality</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >note</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >notes</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >number</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >organization</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >pages</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >publisher</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >revision</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >school</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >series</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >size</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >title</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >type</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >url</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >volume</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >year</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >yearfiled</tt>
 
<p></p>
 
<p></p>
If you want to see what pub­li­ca­tions are in the data­base, the eas­i­est way is to ask for a com­plete list:  
+
If you want to see what publications are in the database, the easiest way is to ask for a complete list:  
 
      
 
      
 
<pre style="color:rgb(102,0,102);font-size:100%">\definebtxrendering
 
<pre style="color:rgb(102,0,102);font-size:100%">\definebtxrendering
Line 224: Line 229:
 
</pre>
 
</pre>
 
<p></p>
 
<p></p>
This gives:1 Ha­gen, H. and Ot­ten, T. (1996). Type­set­ting ed­u­ca­tion doc­u­ments.2 Scarso, L. (2021). De­sign­ing high speed trains.3 au­thor (year). ti­tle. pages p.
+
This gives:1 Hagen, H. and Otten, T. (1996). Typesetting education documents.2 Scarso, L. (2021). Designing high speed trains.3 author (year). title. pages p.
 
<p></p>
 
<p></p>
The ren­der­ing it­self is some­what com­plex to set up be­cause we have not only many dif­fer­ent stan­dards but also many fields that can be set up. This means that there are sev­eral com­mands in­volved. Of­ten there is a pre­scribed style to ren­der bib­li­o­graphic de­scrip­tions, for ex­am­ple <tt style="color:rgb(0,102,102);font-size:100%;" >apa</tt>. A ren­der­ing is setup and de­fined with:
+
The rendering itself is somewhat complex to set up because we have not only many different standards but also many fields that can be set up. This means that there are several commands involved. Often there is a prescribed style to render bibliographic descriptions, for example <tt style="color:rgb(0,102,102);font-size:100%;" >apa</tt>. A rendering is setup and defined with:
 
<div style="border:thin solid black;" >
 
<div style="border:thin solid black;" >
 
<span style="font-style:oblique;" > setup definition setupbtxrendering </span >
 
<span style="font-style:oblique;" > setup definition setupbtxrendering </span >
Line 234: Line 239:
 
</div>
 
</div>
 
<p></p>
 
<p></p>
And a list of such de­scrip­tions is gen­er­ated with:
+
And a list of such descriptions is generated with:
 
<div style="border:thin solid black;" >
 
<div style="border:thin solid black;" >
 
<span style="font-style:oblique;" > setup definition placebtxrendering </span >
 
<span style="font-style:oblique;" > setup definition placebtxrendering </span >
 
</div>
 
</div>
 
<p></p>
 
<p></p>
A dataset can have all kind of en­tries:
+
A dataset can have all kind of entries:
 
<p></p>
 
<p></p>
<tt style="color:rgb(0,102,102);font-size:100%;" >article</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >book</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >booklet</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >conference</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >inbook</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >incollection</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >inproceedings</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >manual</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >mastersthesis</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >misc</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >phdthesis</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >proceedings</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >techreport</tt>, <tt style="color:rgb(0,102,102);font-size:100%;" >unpublished</tt>
+
Each has its own rendering variant. To keep things simple we have their settings separated. However, these settings are shared for all rendering alternatives. In practice this is seldom a problem in a publication as only one rendering alternative will be active. If this be not sufficient, you can always group local settings in a setup and hook that into the specific rendering.
<p></p>
 
Each has its own ren­der­ing vari­ant. To keep things sim­ple we have their set­tings sep­a­rated. How­ever, these set­tings are shared for all ren­der­ing al­ter­na­tives. In prac­tice this is sel­dom a prob­lem in a pub­li­ca­tion as only one ren­der­ing al­ter­na­tive will be ac­tive. If this be not suf­fi­cient, you can al­ways group lo­cal set­tings in a setup and hook that into the spe­cific ren­der­ing.
 
 
<div style="border:thin solid black;" >
 
<div style="border:thin solid black;" >
 
<span style="font-style:oblique;" > setup definition setupbtxlistvariant </span >
 
<span style="font-style:oblique;" > setup definition setupbtxlistvariant </span >
Line 251: Line 254:
 
</div>
 
</div>
 
<p></p>
 
<p></p>
Ex­am­ples of list vari­ants are:
+
Examples of list variants are:  
 
<p></p>
 
<p></p>
 
<tt style="color:rgb(102,0,102);font-size:100%;" >setupbtxlistvariant : artauthor</tt>  
 
<tt style="color:rgb(102,0,102);font-size:100%;" >setupbtxlistvariant : artauthor</tt>  
Line 266: Line 269:
 
</table>
 
</table>
  
<p></p>
 
 
<tt style="color:rgb(102,0,102);font-size:100%;" >setupbtxlistvariant : editor</tt>  
 
<tt style="color:rgb(102,0,102);font-size:100%;" >setupbtxlistvariant : editor</tt>  
 
      
 
      
Line 274: Line 276:
  
 
<p></p>
 
<p></p>
The ex­act ren­der­ing of list en­tries is de­ter­mined by the <tt style="color:rgb(0,102,102);font-size:100%;" >alternative</tt> key and de­faults to <tt style="color:rgb(0,102,102);font-size:100%;" >apa</tt> which uses de­f­i­n­i­tions from <tt style="color:rgb(0,102,102);font-size:100%;" >publ-imp-apa.mkiv</tt>. If you look at that file you will see that each cat­e­gory has its own setup. You may also no­tice that ad­di­tional tests are needed to make sure that empty fields don’t trig­ger sep­a­ra­tors and such.
+
The exact rendering of list entries is determined by the <tt style="color:rgb(0,102,102);font-size:100%;" >alternative</tt> key and defaults to <tt style="color:rgb(0,102,102);font-size:100%;" >apa</tt> which uses definitions from <tt style="color:rgb(0,102,102);font-size:100%;" >publ-imp-apa.mkiv</tt>. If you look at that file you will see that each category has its own setup. You may also notice that additional tests are needed to make sure that empty fields don’t trigger separators and such.
 
<p></p>
 
<p></p>
There are a cou­ple of ac­ces­sors and helpers to get the job done. When you want to fetch a field from the cur­rent en­try you use <tt style="color:rgb(0,102,102);font-size:100%;" >\btxfield</tt>. In most cases you want to make sure this field has a value, for in­stance be­cause you don’t want fences or punc­tu­a­tion that be­longs to a field.  
+
There are a couple of accessors and helpers to get the job done. When you want to fetch a field from the current entry you use <tt style="color:rgb(0,102,102);font-size:100%;" >\btxfield</tt>. In most cases you want to make sure this field has a value, for instance because you don’t want fences or punctuation that belongs to a field.  
 
      
 
      
 
<pre style="color:rgb(102,0,102);font-size:100%">\btxdoif {title} {
 
<pre style="color:rgb(102,0,102);font-size:100%">\btxdoif {title} {
Line 290: Line 292:
 
</pre>
 
</pre>
 
<p></p>
 
<p></p>
An ex­tra con­di­tional is avail­able for test­ing in­ter­ac­tiv­ity:  
+
An extra conditional is available for testing interactivity:  
 
      
 
      
 
<pre style="color:rgb(102,0,102);font-size:100%">\btxdoifelseinteraction{action when true}{action when false}
 
<pre style="color:rgb(102,0,102);font-size:100%">\btxdoifelseinteraction{action when true}{action when false}
 
</pre>
 
</pre>
 
<p></p>
 
<p></p>
In ad­di­tion there is also a con­di­tional <tt style="color:rgb(0,102,102);font-size:100%;" >\btxinteractive</tt> which is more ef­fi­cient, al­though in prac­tice ef­fi­ciency is not so im­por­tant here.
+
In addition there is also a conditional <tt style="color:rgb(0,102,102);font-size:100%;" >\btxinteractive</tt> which is more efficient, although in practice efficiency is not so important here.
 
<p></p>
 
<p></p>
There are three com­mands to flush data:  
+
There are three commands to flush data:  
 
      
 
      
 
<table>
 
<table>
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >\btxfield</tt></td><td>fetch a ex­plicit field (e.g. <tt style="color:rgb(0,102,102);font-size:100%;" >year</tt>)</td></tr>
+
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >\btxfield</tt></td><td>fetch a explicit field (e.g. <tt style="color:rgb(0,102,102);font-size:100%;" >year</tt>)</td></tr>
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >\btxdetail</tt></td><td>fetch a de­rived field (e.g. <tt style="color:rgb(0,102,102);font-size:100%;" >short</tt>)</td></tr>
+
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >\btxdetail</tt></td><td>fetch a derived field (e.g. <tt style="color:rgb(0,102,102);font-size:100%;" >short</tt>)</td></tr>
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >\btxflush</tt></td><td>fetch a de­rived or ex­plicit field</td></tr>
+
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >\btxflush</tt></td><td>fetch a derived or explicit field</td></tr>
 
</table>
 
</table>
  
 
<p></p>
 
<p></p>
Nor­mally you can use <tt style="color:rgb(0,102,102);font-size:100%;" >\btxfield</tt> or <tt style="color:rgb(0,102,102);font-size:100%;" >\btxflush</tt> as de­rived fields just like an­a­lyzed au­thor fields are flushed in a spe­cial way.
+
Normally you can use <tt style="color:rgb(0,102,102);font-size:100%;" >\btxfield</tt> or <tt style="color:rgb(0,102,102);font-size:100%;" >\btxflush</tt> as derived fields just like analyzed author fields are flushed in a special way.
 
<p></p>
 
<p></p>
You can im­prove read­abil­ity by us­ing se­tups, for in­stance:  
+
You can improve readability by using setups, for instance:  
 
      
 
      
 
<pre style="color:rgb(102,0,102);font-size:100%">\btxdoifelse {author} {
 
<pre style="color:rgb(102,0,102);font-size:100%">\btxdoifelse {author} {
Line 317: Line 319:
 
</pre>
 
</pre>
 
<p></p>
 
<p></p>
Keep in mind that nor­mally you don’t need to mess with de­f­i­n­i­tions like this be­cause stan­dard ren­der­ing styles are pro­vided. These styles use a few helpers that in­ject sym­bols but also take care of lead­ing and trail­ing spaces:  
+
Keep in mind that normally you don’t need to mess with definitions like this because standard rendering styles are provided. These styles use a few helpers that inject symbols but also take care of leading and trailing spaces:
 
      
 
      
 
<table>
 
<table>
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >\btxspace</tt></td><td>be­fore af­ter</td></tr>
+
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >\btxspace</tt></td><td>before after</td></tr>
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >\btxperiod</tt></td><td>be­fore. af­ter</td></tr>
+
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >\btxperiod</tt></td><td>before. after</td></tr>
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >\btxcomma</tt></td><td>be­fore, af­ter</td></tr>
+
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >\btxcomma</tt></td><td>before, after</td></tr>
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >\btxlparent</tt></td><td>be­fore (af­ter</td></tr>
+
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >\btxlparent</tt></td><td>before (after</td></tr>
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >\btxrparent</tt></td><td>be­fore) af­ter</td></tr>
+
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >\btxrparent</tt></td><td>before) after</td></tr>
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >\btxlbracket</tt></td><td>be­fore [af­ter</td></tr>
+
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >\btxlbracket</tt></td><td>before [after</td></tr>
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >\btxrbracket</tt></td><td>be­fore] af­ter</td></tr>
+
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >\btxrbracket</tt></td><td>before] after</td></tr>
 
</table>
 
</table>
  
 
<p></p>
 
<p></p>
So, the pre­vi­ous ex­am­ple setup can be rewrit­ten as:  
+
So, the previous example setup can be rewritten as:  
 
      
 
      
 
<pre style="color:rgb(102,0,102);font-size:100%">\btxdoif {title} {
 
<pre style="color:rgb(102,0,102);font-size:100%">\btxdoif {title} {
Line 338: Line 340:
 
</pre>
 
</pre>
 
<p></p>
 
<p></p>
There is a spe­cial com­mand for ren­der­ing a (com­bi­na­tion) of au­thors:  
+
There is a special command for rendering a (combination) of authors:  
 
      
 
      
 
<pre style="color:rgb(102,0,102);font-size:100%">\btxflushauthor{author}
 
<pre style="color:rgb(102,0,102);font-size:100%">\btxflushauthor{author}
Line 345: Line 347:
 
</pre>
 
</pre>
 
<p></p>
 
<p></p>
In­stead of the last one you can also use:  
+
Instead of the last one you can also use:  
 
      
 
      
 
<pre style="color:rgb(102,0,102);font-size:100%">\btxflushauthorinverted{editor}
 
<pre style="color:rgb(102,0,102);font-size:100%">\btxflushauthorinverted{editor}
 
</pre>
 
</pre>
 
<p></p>
 
<p></p>
You can use a (con­fig­urable) de­fault or pass di­rec­tives: Valid di­rec­tives are  
+
You can use a (configurable) default or pass directives: Valid directives are  
 
      
 
      
 
<table>
 
<table>
<tr><td>con­ver­sion</td><td>ren­der­ing</td></tr>
+
<tr><td>conversion</td><td>rendering</td></tr>
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >inverted</tt></td><td>the Frog jr, Ker­mit</td></tr>
+
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >inverted</tt></td><td>the Frog jr, Kermit</td></tr>
 
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >invertedshort</tt></td><td>the Frog jr, K</td></tr>
 
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >invertedshort</tt></td><td>the Frog jr, K</td></tr>
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >normal</tt></td><td>Ker­mit, the Frog, jr</td></tr>
+
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >normal</tt></td><td>Kermit, the Frog, jr</td></tr>
 
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >normalshort</tt></td><td>K, the Frog, jr</td></tr>
 
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >normalshort</tt></td><td>K, the Frog, jr</td></tr>
 
</table>
 
</table>
Line 362: Line 364:
 
      
 
      
 
      
 
      
<h1>Ci­ta­tions</h1>
+
<h1>Citations</h1>
  
Ci­ta­tions are ref­er­ences to bib­li­o­graphic en­tries that nor­mally show up in lists some­place in the doc­u­ment: at the end of a chap­ter, in an ap­pen­dix, at the end of an ar­ti­cle, etc. We dis­cussed the ren­der­ing of these lists in the pre­vi­ous chap­ter. A ci­ta­tion is nor­mally pretty short as its main pur­pose is to re­fer uniquely to a more de­tailed de­scrip­tion. But, there are sev­eral ways to re­fer, which is why the ci­ta­tion sub­sys­tem is con­fig­urable and ex­ten­si­ble. Just look at the fol­low­ing com­mands:  
+
Citations are references to bibliographic entries that normally show up in lists someplace in the document: at the end of a chapter, in an appendix, at the end of an article, etc. We discussed the rendering of these lists in the previous chapter. A citation is normally pretty short as its main purpose is to refer uniquely to a more detailed description. But, there are several ways to refer, which is why the citation subsystem is configurable and extensible. Just look at the following commands:  
 
      
 
      
 
<pre style="color:rgb(102,0,102);font-size:100%">\cite[author][example::demo-003]
 
<pre style="color:rgb(102,0,102);font-size:100%">\cite[author][example::demo-003]
Line 377: Line 379:
 
</pre>
 
</pre>
 
      
 
      
<div>
+
<li>
:(Hans Ha­gen and Ton Ot­ten)
+
<ul>(Hans Hagen and Ton Otten)</ul>
:(Hans Ha­gen and Ton Ot­ten (1996))
+
<ul>(Hans Hagen and Ton Otten (1996))</ul>
:(Hans Ha­gen and Ton Ot­ten, 1996)
+
<ul>(Hans Hagen and Ton Otten, 1996)</ul>
:(Hans Ha­gen and Ton Ot­ten, Luigi Scarso)
+
<ul>(Hans Hagen and Ton Otten, Luigi Scarso)</ul>
:(Hans Ha­gen and Ton Ot­ten (1996), Luigi Scarso (2021))
+
<ul>(Hans Hagen and Ton Otten (1996), Luigi Scarso (2021))</ul>
:(Hans Ha­gen and Ton Ot­ten, 1996, Luigi Scarso, 2021)
+
<ul>(Hans Hagen and Ton Otten, 1996, Luigi Scarso, 2021)</ul>
:(Luigi Scarso, Hans Ha­gen and Ton Ot­ten)
+
<ul>(Luigi Scarso, Hans Hagen and Ton Otten)</ul>
:(Luigi Scarso (2021), Hans Ha­gen and Ton Ot­ten (1996))
+
<ul>(Luigi Scarso (2021), Hans Hagen and Ton Otten (1996))</ul>
:(Luigi Scarso, 2021, Hans Ha­gen and Ton Ot­ten, 1996)
+
<ul>(Luigi Scarso, 2021, Hans Hagen and Ton Otten, 1996)</ul>
</div>
+
</li>
 
 
 
<p></p>
 
<p></p>
The first ar­gu­ment is op­tional.
+
The first argument is optional.
 
<div style="border:thin solid black;" >
 
<div style="border:thin solid black;" >
 
<span style="font-style:oblique;" > setup definition cite </span >
 
<span style="font-style:oblique;" > setup definition cite </span >
 
</div>
 
</div>
 
<p></p>
 
<p></p>
You can tune the way a ci­ta­tion shows up:  
+
You can tune the way a citation shows up:  
 
      
 
      
 
<pre style="color:rgb(102,0,102);font-size:100%">\setupbtxcitevariant[author]    [sorttype=author,color=darkyellow]
 
<pre style="color:rgb(102,0,102);font-size:100%">\setupbtxcitevariant[author]    [sorttype=author,color=darkyellow]
Line 405: Line 406:
 
</pre>
 
</pre>
 
<p></p>
 
<p></p>
Here we sort the au­thors and color the ci­ta­tion:  
+
Here we sort the authors and color the citation:  
 
      
 
      
<div>
+
<li>
:(Hans Ha­gen and Ton Ot­ten, Luigi Scarso)
+
<ul>(Hans Hagen and Ton Otten, Luigi Scarso)</ul>
:(Hans Ha­gen and Ton Ot­ten (1996), Luigi Scarso (2021))
+
<ul>(Hans Hagen and Ton Otten (1996), Luigi Scarso (2021))</ul>
:(Hans Ha­gen and Ton Ot­ten, 1996, Luigi Scarso, 2021)
+
<ul>(Hans Hagen and Ton Otten, 1996, Luigi Scarso, 2021)</ul>
</div>
+
</li>
 
 
 
<p></p>
 
<p></p>
For rea­sons of back­ward com­pat­i­bil­ity the <tt style="color:rgb(0,102,102);font-size:100%;" >\cite</tt> com­mand is a bit picky about spaces be­tween the two ar­gu­ments, of which the first is op­tional. This is a con­se­quence of al­low­ing its use with the key spec­i­fied be­tween curly brack­ets as is the tra­di­tional prac­tice. (We do en­cour­age users to adopt the more co­her­ent <span tag="MKIV" style="font-style:sans;">ConTEXt</span> syn­tax by us­ing square brack­ets for key­words and re­serv­ing curly brack­ets to re­group text to be type­set.)
+
For reasons of backward compatibility the <tt style="color:rgb(0,102,102);font-size:100%;" >\cite</tt> command is a bit picky about spaces between the two arguments, of which the first is optional. This is a consequence of allowing its use with the key specified between curly brackets as is the traditional practice. (We do encourage users to adopt the more coherent <span tag="MKIV" style="font-style:sans;">ConTEXt</span> syntax by using square brackets for keywords and reserving curly brackets to regroup text to be typeset.)
 
<p></p>
 
<p></p>
The <tt style="color:rgb(0,102,102);font-size:100%;" >\citation</tt> com­mand is syn­ony­mous but is more flex­i­ble with re­spect to spac­ing of its ar­gu­ments:  
+
The <tt style="color:rgb(0,102,102);font-size:100%;" >\citation</tt> command is synonymous but is more flexible with respect to spacing of its arguments:  
 
      
 
      
 
<pre style="color:rgb(102,0,102);font-size:100%">\citation[author]    [example::demo-004,demo-003]
 
<pre style="color:rgb(102,0,102);font-size:100%">\citation[author]    [example::demo-004,demo-003]
Line 423: Line 423:
 
</pre>
 
</pre>
 
<p></p>
 
<p></p>
There is a whole bunch of cite op­tions and more can be eas­ily de­fined.  
+
There is a whole bunch of cite options and more can be easily defined.  
 
      
 
      
 
<table>
 
<table>
<tr><td>key</td><td>ren­der­ing</td></tr>
+
<tr><td>key</td><td>rendering</td></tr>
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >author</tt></td><td>(au­thor)</td></tr>
+
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >author</tt></td><td>(author)</td></tr>
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >authornum</tt></td><td>[au­thor [btx er­ror 1]]</td></tr>
+
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >authornum</tt></td><td>[author [btx error 1]]</td></tr>
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >authoryear</tt></td><td>(au­thor (year))</td></tr>
+
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >authoryear</tt></td><td>(author (year))</td></tr>
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >authoryears</tt></td><td>(au­thor, year)</td></tr>
+
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >authoryears</tt></td><td>(author, year)</td></tr>
 
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >doi</tt></td><td>[todo: doi]</td></tr>
 
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >doi</tt></td><td>[todo: doi]</td></tr>
 
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >key</tt></td><td>[demo-005]</td></tr>
 
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >key</tt></td><td>[demo-005]</td></tr>
 
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >none</tt></td><td></td></tr>
 
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >none</tt></td><td></td></tr>
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >num</tt></td><td>[[btx er­ror 1]]</td></tr>
+
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >num</tt></td><td>[[btx error 1]]</td></tr>
 
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >page</tt></td><td>pages</td></tr>
 
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >page</tt></td><td>pages</td></tr>
 
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >serial</tt></td><td>[5]</td></tr>
 
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >serial</tt></td><td>[5]</td></tr>
Line 444: Line 444:
  
 
<p></p>
 
<p></p>
Be­cause we are deal­ing with data­base in­put and be­cause we gen­er­ally need to ma­nip­u­late en­tries, much of the work is del­e­gated to <span tag="MKIV" style="font-style:sans;">Lua</span>. This makes it eas­ier to main­tain and ex­tend the code. Of course <span tag="MKIV" style="font-style:sans;">TEX</span> still does the ren­der­ing. The ty­po­graphic de­tails are con­trolled by pa­ra­me­ters but not all are used in all vari­ants. As with most <span tag="MKIV" style="font-style:sans;">ConTEXt</span> com­mands, it starts out with a gen­eral setup com­mand:
+
Because we are dealing with database input and because we generally need to manipulate entries, much of the work is delegated to <span tag="MKIV" style="font-style:sans;">Lua</span>. This makes it easier to maintain and extend the code. Of course <span tag="MKIV" style="font-style:sans;">TEX</span> still does the rendering. The typographic details are controlled by parameters but not all are used in all variants. As with most <span tag="MKIV" style="font-style:sans;">ConTEXt</span> commands, it starts out with a general setup command:
 
<div style="border:thin solid black;" >
 
<div style="border:thin solid black;" >
 
<span style="font-style:oblique;" > setup definition setupbtxcitevariant </span >
 
<span style="font-style:oblique;" > setup definition setupbtxcitevariant </span >
 
</div>
 
</div>
 
<p></p>
 
<p></p>
On top of that we can de­fine in­stances that in­herit ei­ther from a given par­ent or from the top­most setup.
+
On top of that we can define instances that inherit either from a given parent or from the topmost setup.
 
<div style="border:thin solid black;" >
 
<div style="border:thin solid black;" >
 
<span style="font-style:oblique;" > setup definition definebtxcitevariant </span >
 
<span style="font-style:oblique;" > setup definition definebtxcitevariant </span >
 
</div>
 
</div>
 
<p></p>
 
<p></p>
But, spe­cific vari­ants can have them over­loaded:  
+
But, specific variants can have them overloaded:  
 
<p></p>
 
<p></p>
 
<tt style="color:rgb(102,0,102);font-size:100%;" >setupbtxcitevariant : author</tt>  
 
<tt style="color:rgb(102,0,102);font-size:100%;" >setupbtxcitevariant : author</tt>  
Line 575: Line 575:
 
</table>
 
</table>
  
A ci­ta­tion vari­ant is de­fined in sev­eral steps and if you re­ally want to know the dirty de­tails, you should look into the <tt style="color:rgb(0,102,102);font-size:100%;" >publ-imp-*.mkiv</tt> files. Here we stick to the con­cept.  
+
A citation variant is defined in several steps and if you really want to know the dirty details, you should look into the <tt style="color:rgb(0,102,102);font-size:100%;" >publ-imp-*.mkiv</tt> files. Here we stick to the concept.  
 
      
 
      
 
<pre style="color:rgb(102,0,102);font-size:100%">\startsetups btx:cite:author
 
<pre style="color:rgb(102,0,102);font-size:100%">\startsetups btx:cite:author
Line 582: Line 582:
 
</pre>
 
</pre>
 
<p></p>
 
<p></p>
You can over­load such se­tups if needed, but that only makes sense when you can­not con­fig­ure the ren­der­ing with pa­ra­me­ters. The <tt style="color:rgb(0,102,102);font-size:100%;" >\btxcitevariant</tt> com­mand is one of the build in ac­ces­sors and it calls out to <span tag="MKIV" style="font-style:sans;">Lua</span> where more com­plex ma­nip­u­la­tion takes place if needed. If no ma­nip­u­la­tion is known, the field with the same name (if found) will be flushed. A com­mand like <tt style="color:rgb(0,102,102);font-size:100%;" >\btxcitevariant</tt> as­sumes that a dataset and spe­cific tag has been set. This is nor­mally done in the wrap­per macros, like <tt style="color:rgb(0,102,102);font-size:100%;" >\cite</tt>. For spe­cial pur­poses you can use these com­mands
+
You can overload such setups if needed, but that only makes sense when you cannot configure the rendering with parameters. The <tt style="color:rgb(0,102,102);font-size:100%;" >\btxcitevariant</tt> command is one of the build in accessors and it calls out to <span tag="MKIV" style="font-style:sans;">Lua</span> where more complex manipulation takes place if needed. If no manipulation is known, the field with the same name (if found) will be flushed. A command like <tt style="color:rgb(0,102,102);font-size:100%;" >\btxcitevariant</tt> assumes that a dataset and specific tag has been set. This is normally done in the wrapper macros, like <tt style="color:rgb(0,102,102);font-size:100%;" >\cite</tt>. For special purposes you can use these commands
 
      
 
      
 
<pre style="color:rgb(102,0,102);font-size:100%">\setbtxdataset[example]
 
<pre style="color:rgb(102,0,102);font-size:100%">\setbtxdataset[example]
Line 588: Line 588:
 
</pre>
 
</pre>
 
<p></p>
 
<p></p>
But don’t ex­pect too much sup­port for such low level ren­der­ing con­trol.
+
But don’t expect too much support for such low level rendering control.
 
<p></p>
 
<p></p>
Un­less you use <tt style="color:rgb(0,102,102);font-size:100%;" >criterium=all</tt> only pub­li­ca­tions that are cited will end up in the lists. You can force a ci­ta­tion into a list us­ing <tt style="color:rgb(0,102,102);font-size:100%;" >\usecitation</tt>, for ex­am­ple:  
+
Unless you use <tt style="color:rgb(0,102,102);font-size:100%;" >criterium=all</tt> only publications that are cited will end up in the lists. You can force a citation into a list using <tt style="color:rgb(0,102,102);font-size:100%;" >\usecitation</tt>, for example:  
 
      
 
      
 
<pre style="color:rgb(102,0,102);font-size:100%">\usecitation[example::demo-004,demo-003]
 
<pre style="color:rgb(102,0,102);font-size:100%">\usecitation[example::demo-004,demo-003]
 
</pre>
 
</pre>
 
<p></p>
 
<p></p>
This com­mand has two syn­onyms: <tt style="color:rgb(0,102,102);font-size:100%;" >\nocite</tt> and <tt style="color:rgb(0,102,102);font-size:100%;" >\nocitation</tt> so you can choose what­ever fits you best.
+
This command has two synonyms: <tt style="color:rgb(0,102,102);font-size:100%;" >\nocite</tt> and <tt style="color:rgb(0,102,102);font-size:100%;" >\nocitation</tt> so you can choose whatever fits you best.
 
<div style="border:thin solid black;" >
 
<div style="border:thin solid black;" >
 
<span style="font-style:oblique;" > setup definition nocite </span >
 
<span style="font-style:oblique;" > setup definition nocite </span >
Line 603: Line 603:
 
<h1>The LUA view</h1>
 
<h1>The LUA view</h1>
  
Be­cause we man­age data at the <span tag="MKIV" style="font-style:sans;">Lua</span> end it is tempt­ing to ac­cess it there for other pur­poses. This is fine as long as you keep in mind that as­pects of the im­ple­men­ta­tion may change over time, al­though this is un­likely once the mod­ules be­come sta­ble.
+
Because we manage data at the <span tag="MKIV" style="font-style:sans;">Lua</span> end it is tempting to access it there for other purposes. This is fine as long as you keep in mind that aspects of the implementation may change over time, although this is unlikely once the modules become stable.
 
<p></p>
 
<p></p>
The en­tries are col­lected in datasets and each set has a unique name. In this doc­u­ment we have the set named <tt style="color:rgb(0,102,102);font-size:100%;" >example</tt>. A dataset ta­ble has sev­eral fields, and prob­a­bly the one of most in­ter­est is the <tt style="color:rgb(0,102,102);font-size:100%;" >luadata</tt> field. Each en­try in this ta­ble de­scribes a pub­li­ca­tion:  
+
The entries are collected in datasets and each set has a unique name. In this document we have the set named <tt style="color:rgb(0,102,102);font-size:100%;" >example</tt>. A dataset table has several fields, and probably the one of most interest is the <tt style="color:rgb(0,102,102);font-size:100%;" >luadata</tt> field. Each entry in this table describes a publication:  
 
      
 
      
 
<pre style="color:rgb(102,0,102);font-size:100%">t={  
 
<pre style="color:rgb(102,0,102);font-size:100%">t={  
Line 616: Line 616:
 
}  
 
}  
 
</pre>
 
</pre>
This is <tt style="color:rgb(0,102,102);font-size:100%;" >publications.datasets.example.luadata["demo-001"]</tt>. There can be a com­pan­ion en­try in the par­al­lel <tt style="color:rgb(0,102,102);font-size:100%;" >details</tt> ta­ble.  
+
This is <tt style="color:rgb(0,102,102);font-size:100%;" >publications.datasets.example.luadata["demo-001"]</tt>. There can be a companion entry in the parallel <tt style="color:rgb(0,102,102);font-size:100%;" >details</tt> table.  
 
      
 
      
 
<pre style="color:rgb(102,0,102);font-size:100%">t={  
 
<pre style="color:rgb(102,0,102);font-size:100%">t={  
Line 631: Line 631:
 
}  
 
}  
 
</pre>
 
</pre>
These de­tails are ac­cessed as <tt style="color:rgb(0,102,102);font-size:100%;" >publications.datasets.example.details["demo-001"]</tt> and by us­ing a sep­a­rate ta­ble we can over­load fields in the orig­i­nal en­try with­out los­ing the orig­i­nal.
+
These details are accessed as <tt style="color:rgb(0,102,102);font-size:100%;" >publications.datasets.example.details["demo-001"]</tt> and by using a separate table we can overload fields in the original entry without losing the original.
 
<p></p>
 
<p></p>
You can loop over the en­tries us­ing reg­u­lar <span tag="MKIV" style="font-style:sans;">Lua</span> code com­bined with <span tag="MKIV" style="font-style:sans;">MkIV</span> helpers:  
+
You can loop over the entries using regular <span tag="MKIV" style="font-style:sans;">Lua</span> code combined with <span tag="MKIV" style="font-style:sans;">MkIV</span> helpers:  
 
      
 
      
 
<pre style="color:rgb(102,0,102);font-size:100%">local dataset = publications.datasets.example
 
<pre style="color:rgb(102,0,102);font-size:100%">local dataset = publications.datasets.example
Line 647: Line 647:
 
</pre>
 
</pre>
 
<p></p>
 
<p></p>
This re­sults in:  
+
This results in:  
 
      
 
      
 
<table>
 
<table>
 
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >demo-001</tt></td><td>Hag13</td><td><span tag="MKIV" style="font-style:sans;">bibTEX</span>, the <span tag="MKIV" style="font-style:sans;">ConTEXt</span> way</td></tr>
 
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >demo-001</tt></td><td>Hag13</td><td><span tag="MKIV" style="font-style:sans;">bibTEX</span>, the <span tag="MKIV" style="font-style:sans;">ConTEXt</span> way</td></tr>
 
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >demo-002</tt></td><td>Hag14</td><td><span tag="MKIV" style="font-style:sans;">bibTEX</span>, the <span tag="MKIV" style="font-style:sans;">ConTEXt</span> way</td></tr>
 
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >demo-002</tt></td><td>Hag14</td><td><span tag="MKIV" style="font-style:sans;">bibTEX</span>, the <span tag="MKIV" style="font-style:sans;">ConTEXt</span> way</td></tr>
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >demo-003</tt></td><td>HO96</td><td>Type­set­ting ed­u­ca­tion doc­u­ments</td></tr>
+
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >demo-003</tt></td><td>HO96</td><td>Typesetting education documents</td></tr>
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >demo-004</tt></td><td>Sca21</td><td>De­sign­ing high speed trains</td></tr>
+
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >demo-004</tt></td><td>Sca21</td><td>Designing high speed trains</td></tr>
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >demo-005</tt></td><td>aut00</td><td>ti­tle</td></tr>
+
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >demo-005</tt></td><td>aut00</td><td>title</td></tr>
 
</table>
 
</table>
  
 
<p></p>
 
<p></p>
You can ma­nip­u­late a dataset af­ter load­ing. Of course this as­sumes that you know what kind of con­tent you have and what you need for ren­der­ing. As ex­am­ple we load a small dataset.  
+
You can manipulate a dataset after loading. Of course this assumes that you know what kind of content you have and what you need for rendering. As example we load a small dataset.  
 
      
 
      
 
<pre style="color:rgb(102,0,102);font-size:100%">\definebtxdataset[drumming]
 
<pre style="color:rgb(102,0,102);font-size:100%">\definebtxdataset[drumming]
Line 664: Line 664:
 
</pre>
 
</pre>
 
<p></p>
 
<p></p>
Be­cause we’re go­ing to do some <span tag="MKIV" style="font-style:sans;">Lua</span>, we could also have loaded the dataset with:  
+
Because we’re going to do some <span tag="MKIV" style="font-style:sans;">Lua</span>, we could also have loaded the dataset with:  
 
      
 
      
 
<pre style="color:rgb(102,0,102);font-size:100%">publications.load("drumming","mkiv-publications.lua","lua")
 
<pre style="color:rgb(102,0,102);font-size:100%">publications.load("drumming","mkiv-publications.lua","lua")
 
</pre>
 
</pre>
 
<p></p>
 
<p></p>
The dataset has three en­tries:
+
The dataset has three entries:
 
<p></p>
 
<p></p>
As you can see, we can have a sub­ti­tle. We will com­bine the ti­tle and sub­ti­tle into one:  
+
As you can see, we can have a subtitle. We will combine the title and subtitle into one:  
 
      
 
      
 
<pre style="color:rgb(102,0,102);font-size:100%">\startluacode
 
<pre style="color:rgb(102,0,102);font-size:100%">\startluacode
Line 688: Line 688:
 
</pre>
 
</pre>
 
<p></p>
 
<p></p>
We can now type­set the en­tries with:  
+
We can now typeset the entries with:  
 
      
 
      
 
<pre style="color:rgb(102,0,102);font-size:100%">\definebtxrendering[drumming][dataset=drumming,method=dataset]
 
<pre style="color:rgb(102,0,102);font-size:100%">\definebtxrendering[drumming][dataset=drumming,method=dataset]
Line 694: Line 694:
 
</pre>
 
</pre>
 
<p></p>
 
<p></p>
Be­cause we just want to show the en­tries, and have no ci­ta­tions that force them to be shown, we have to the <tt style="color:rgb(0,102,102);font-size:100%;" >method</tt> to <tt style="color:rgb(0,102,102);font-size:100%;" >dataset</tt>.1  
+
Because we just want to show the entries, and have no citations that force them to be shown, we have to the <tt style="color:rgb(0,102,102);font-size:100%;" >method</tt> to <tt style="color:rgb(0,102,102);font-size:100%;" >dataset</tt>.1  
 
      
 
      
 
    
 
    
Line 700: Line 700:
 
<h1>The XML view</h1>
 
<h1>The XML view</h1>
  
The <tt style="color:rgb(0,102,102);font-size:100%;" >luadata</tt> ta­ble can be con­verted into an <span tag="MKIV" style="font-style:sans;">xml</span> rep­re­sen­ta­tion. This is a fol­low up on ear­lier ex­per­i­ments with an <span tag="MKIV" style="font-style:sans;">xml</span>-only ap­proach. I de­cided in the end to stick to a <span tag="MKIV" style="font-style:sans;">Lua</span> ap­proach and pro­vide some sim­ple <span tag="MKIV" style="font-style:sans;">xml</span> sup­port in ad­di­tion.
+
The <tt style="color:rgb(0,102,102);font-size:100%;" >luadata</tt> table can be converted into an <span tag="MKIV" style="font-style:sans;">xml</span> representation. This is a follow up on earlier experiments with an <span tag="MKIV" style="font-style:sans;">xml</span>-only approach. I decided in the end to stick to a <span tag="MKIV" style="font-style:sans;">Lua</span> approach and provide some simple <span tag="MKIV" style="font-style:sans;">xml</span> support in addition.
 
<p></p>
 
<p></p>
Once a dataset is ac­ces­si­ble as <span tag="MKIV" style="font-style:sans;">xml</span> tree, you can use the reg­u­lar <tt style="color:rgb(0,102,102);font-size:100%;" >\xml...</tt> com­mands. We start with load­ing a dataset, in this case from just one file.  
+
Once a dataset is accessible as <span tag="MKIV" style="font-style:sans;">xml</span> tree, you can use the regular <tt style="color:rgb(0,102,102);font-size:100%;" >\xml...</tt> commands. We start with loading a dataset, in this case from just one file.  
 
      
 
      
 
<pre style="color:rgb(102,0,102);font-size:100%">\usebtxdataset[tugboat][tugboat.bib]
 
<pre style="color:rgb(102,0,102);font-size:100%">\usebtxdataset[tugboat][tugboat.bib]
 
</pre>
 
</pre>
 
<p></p>
 
<p></p>
The dataset has to be con­verted to <span tag="MKIV" style="font-style:sans;">xml</span>:  
+
The dataset has to be converted to <span tag="MKIV" style="font-style:sans;">xml</span>:  
 
      
 
      
 
<pre style="color:rgb(102,0,102);font-size:100%">\convertbtxdatasettoxml[tugboat]
 
<pre style="color:rgb(102,0,102);font-size:100%">\convertbtxdatasettoxml[tugboat]
 
</pre>
 
</pre>
 
<p></p>
 
<p></p>
The tree is now ac­ces­si­ble by its root ref­er­ence <tt style="color:rgb(0,102,102);font-size:100%;" >btx:tugboat</tt>. If we want sim­ple field ac­cess we can use a few se­tups:  
+
The tree is now accessible by its root reference <tt style="color:rgb(0,102,102);font-size:100%;" >btx:tugboat</tt>. If we want simple field access we can use a few setups:  
 
      
 
      
 
<pre style="color:rgb(102,0,102);font-size:100%">\startxmlsetups btx:initialize
 
<pre style="color:rgb(102,0,102);font-size:100%">\startxmlsetups btx:initialize
Line 724: Line 724:
 
</pre>
 
</pre>
 
<p></p>
 
<p></p>
The two se­tups are pre­de­fined in the core al­ready, but you might want to change them. They are ap­plied in for in­stance:  
+
The two setups are predefined in the core already, but you might want to change them. They are applied in for instance:  
 
      
 
      
 
<pre style="color:rgb(102,0,102);font-size:100%">\starttabulate[|||]
 
<pre style="color:rgb(102,0,102);font-size:100%">\starttabulate[|||]
Line 737: Line 737:
 
      
 
      
 
<table>
 
<table>
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >tag</tt></td><td>Ha­gen:TB17-1-54</td></tr>
+
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >tag</tt></td><td>Hagen:TB17-1-54</td></tr>
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >title</tt></td><td>PPCHTEX: type­set­ting chem­i­cal for­mu­las in TEX</td></tr>
+
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >title</tt></td><td>PPCHTEX: typesetting chemical formulas in TEX</td></tr>
 
</table>
 
</table>
  
Line 757: Line 757:
 
      
 
      
 
<table>
 
<table>
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >tag</tt></td><td>Ha­gen:TB17-1-54</td></tr>
+
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >tag</tt></td><td>Hagen:TB17-1-54</td></tr>
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >title</tt></td><td>PPCHTEX: type­set­ting chem­i­cal for­mu­las in TEX</td></tr>
+
<tr><td><tt style="color:rgb(0,102,102);font-size:100%;" >title</tt></td><td>PPCHTEX: typesetting chemical formulas in TEX</td></tr>
 
</table>
 
</table>
  
 
<p></p>
 
<p></p>
Here is an­other ex­am­ple:  
+
Here is another example:  
 
      
 
      
 
<pre style="color:rgb(102,0,102);font-size:100%">\startxmlsetups btx:row
 
<pre style="color:rgb(102,0,102);font-size:100%">\startxmlsetups btx:row
Line 783: Line 783:
 
      
 
      
 
<table>
 
<table>
<tr><td>Knuth:TB10-1-31</td><td>Type­set­ting Con­crete Math­e­mat­ics</td></tr>
+
<tr><td>Knuth:TB10-1-31</td><td>Typesetting Concrete Mathematics</td></tr>
<tr><td>Knuth:TB10-1-8</td><td>TEX would find it dif­fi­cult …</td></tr>
+
<tr><td>Knuth:TB10-1-8</td><td>TEX would find it difficult …</td></tr>
<tr><td>Knuth:TB10-3-325</td><td>The new ver­sions of TEX and MF</td></tr>
+
<tr><td>Knuth:TB10-3-325</td><td>The new versions of TEX and MF</td></tr>
<tr><td>Knuth:TB10-4-529</td><td>The er­rors of TEX</td></tr>
+
<tr><td>Knuth:TB10-4-529</td><td>The errors of TEX</td></tr>
<tr><td>Knuth:TB11-1-13</td><td>Vir­tual Fonts: More Fun for Grand Wiz­ards</td></tr>
+
<tr><td>Knuth:TB11-1-13</td><td>Virtual Fonts: More Fun for Grand Wizards</td></tr>
<tr><td>Knuth:TB11-2-165</td><td>Ex­er­cises for TEX: The Pro­gram</td></tr>
+
<tr><td>Knuth:TB11-2-165</td><td>Exercises for TEX: The Program</td></tr>
<tr><td>Knuth:TB11-4-489</td><td>The fu­ture of TEX and MF</td></tr>
+
<tr><td>Knuth:TB11-4-489</td><td>The future of TEX and MF</td></tr>
 
<tr><td>Knuth:TB11-4-497</td><td>Arthur Lee Samuel, 1901--1990</td></tr>
 
<tr><td>Knuth:TB11-4-497</td><td>Arthur Lee Samuel, 1901--1990</td></tr>
<tr><td>Knuth:TB11-4-499</td><td>An­swers to Ex­er­cises for TEX: The Pro­gram</td></tr>
+
<tr><td>Knuth:TB11-4-499</td><td>Answers to Exercises for TEX: The Program</td></tr>
<tr><td>Knuth:TB12-2-313</td><td>Fixed-point glue set­ting: Er­rata</td></tr>
+
<tr><td>Knuth:TB12-2-313</td><td>Fixed-point glue setting: Errata</td></tr>
 
<tr><td>Knuth:TB14-4-387</td><td>Icons for TEX and MF</td></tr>
 
<tr><td>Knuth:TB14-4-387</td><td>Icons for TEX and MF</td></tr>
<tr><td>Knuth:TB17-1-29</td><td>Im­por­tant mes­sage re­gard­ing CM fonts</td></tr>
+
<tr><td>Knuth:TB17-1-29</td><td>Important message regarding CM fonts</td></tr>
<tr><td>Knuth:TB2-3-5</td><td>The cur­rent state of things</td></tr>
+
<tr><td>Knuth:TB2-3-5</td><td>The current state of things</td></tr>
<tr><td>Knuth:TB3-1-10</td><td>Fixed-point glue set­ting­Dash an ex­am­ple of WEB</td></tr>
+
<tr><td>Knuth:TB3-1-10</td><td>Fixed-point glue settingDashan example of WEB</td></tr>
<tr><td>Knuth:TB31-2-121</td><td>An Earth­shak­ing An­nounce­ment</td></tr>
+
<tr><td>Knuth:TB31-2-121</td><td>An Earthshaking Announcement</td></tr>
<tr><td>Knuth:TB4-2-64</td><td>A note on hy­phen­ation</td></tr>
+
<tr><td>Knuth:TB4-2-64</td><td>A note on hyphenation</td></tr>
<tr><td>Knuth:TB5-1-4</td><td>TEX in­cunab­ula</td></tr>
+
<tr><td>Knuth:TB5-1-4</td><td>TEX incunabula</td></tr>
<tr><td>Knuth:TB5-1-67</td><td>Com­ments on qual­ity in pub­lish­ing</td></tr>
+
<tr><td>Knuth:TB5-1-67</td><td>Comments on quality in publishing</td></tr>
<tr><td>Knuth:TB5-2-105</td><td>A course on MF pro­gram­ming</td></tr>
+
<tr><td>Knuth:TB5-2-105</td><td>A course on MF programming</td></tr>
<tr><td>Knuth:TB6-1-36</td><td>Recipes and frac­tions</td></tr>
+
<tr><td>Knuth:TB6-1-36</td><td>Recipes and fractions</td></tr>
<tr><td>Knuth:TB7-2-101</td><td>The TEX logo in var­i­ous fonts</td></tr>
+
<tr><td>Knuth:TB7-2-101</td><td>The TEX logo in various fonts</td></tr>
<tr><td>Knuth:TB7-2-95</td><td>Re­marks to cel­e­brate the pub­li­ca­tion of Com­put­ers & Type­set­ting</td></tr>
+
<tr><td>Knuth:TB7-2-95</td><td>Remarks to celebrate the publication of Computers & Typesetting</td></tr>
<tr><td>Knuth:TB8-1-14</td><td>Mix­ing right-to-left texts with left-to-right texts</td></tr>
+
<tr><td>Knuth:TB8-1-14</td><td>Mixing right-to-left texts with left-to-right texts</td></tr>
<tr><td>Knuth:TB8-1-6</td><td>It hap­pened: an­nounce­ment of TEX 2.1</td></tr>
+
<tr><td>Knuth:TB8-1-6</td><td>It happened: announcement of TEX 2.1</td></tr>
<tr><td>Knuth:TB8-1-73</td><td>Prob­lem for a Sat­ur­day af­ter­noon</td></tr>
+
<tr><td>Knuth:TB8-1-73</td><td>Problem for a Saturday afternoon</td></tr>
<tr><td>Knuth:TB8-2-135</td><td>Fonts for dig­i­tal halftones</td></tr>
+
<tr><td>Knuth:TB8-2-135</td><td>Fonts for digital halftones</td></tr>
<tr><td>Knuth:TB8-2-210</td><td>Sat­ur­day morn­ing prob­lem­Dash so­lu­tion</td></tr>
+
<tr><td>Knuth:TB8-2-210</td><td>Saturday morning problemDashsolution</td></tr>
<tr><td>Knuth:TB8-2-217</td><td>Re­ply: Print­ing out se­lected pages</td></tr>
+
<tr><td>Knuth:TB8-2-217</td><td>Reply: Printing out selected pages</td></tr>
 
<tr><td>Knuth:TB8-3-309</td><td>Macros for Jill</td></tr>
 
<tr><td>Knuth:TB8-3-309</td><td>Macros for Jill</td></tr>
 
<tr><td>Knuth:TB9-2-152</td><td>A Punk Meta-Font</td></tr>
 
<tr><td>Knuth:TB9-2-152</td><td>A Punk Meta-Font</td></tr>
Line 816: Line 816:
  
 
<p></p>
 
<p></p>
A more ex­ten­sive ex­am­ple is the fol­low­ing. Of course this as­sumes that you know what <span tag="MKIV" style="font-style:sans;">xml</span> sup­port mech­a­nisms and macros are avail­able.  
+
A more extensive example is the following. Of course this assumes that you know what <span tag="MKIV" style="font-style:sans;">xml</span> support mechanisms and macros are available.  
 
      
 
      
 
<pre style="color:rgb(102,0,102);font-size:100%">\startxmlsetups btx:getkeys
 
<pre style="color:rgb(102,0,102);font-size:100%">\startxmlsetups btx:getkeys
Line 847: Line 847:
 
<table>
 
<table>
 
<tr><td>1984</td><td>Knuth:TB5-1-67</td><td>Don Knuth</td></tr>
 
<tr><td>1984</td><td>Knuth:TB5-1-67</td><td>Don Knuth</td></tr>
<tr><td>1984</td><td>Knuth:TB5-1-4</td><td>Don­ald E. Knuth</td></tr>
+
<tr><td>1984</td><td>Knuth:TB5-1-4</td><td>Donald E. Knuth</td></tr>
<tr><td>1984</td><td>Knuth:TB5-2-105</td><td>Don­ald E. Knuth</td></tr>
+
<tr><td>1984</td><td>Knuth:TB5-2-105</td><td>Donald E. Knuth</td></tr>
<tr><td>1985</td><td>Knuth:TB6-1-36</td><td>Don­ald E. Knuth</td></tr>
+
<tr><td>1985</td><td>Knuth:TB6-1-36</td><td>Donald E. Knuth</td></tr>
<tr><td>1986</td><td>Knuth:TB7-2-101</td><td>Don­ald E. Knuth</td></tr>
+
<tr><td>1986</td><td>Knuth:TB7-2-101</td><td>Donald E. Knuth</td></tr>
<tr><td>1987</td><td>Knuth:TB8-2-135</td><td>Don­ald E. Knuth</td></tr>
+
<tr><td>1987</td><td>Knuth:TB8-2-135</td><td>Donald E. Knuth</td></tr>
<tr><td>1987</td><td>Knuth:TB8-3-309</td><td>Don­ald E. Knuth</td></tr>
+
<tr><td>1987</td><td>Knuth:TB8-3-309</td><td>Donald E. Knuth</td></tr>
<tr><td>1988</td><td>Knuth:TB9-2-152</td><td>Don­ald E. Knuth</td></tr>
+
<tr><td>1988</td><td>Knuth:TB9-2-152</td><td>Donald E. Knuth</td></tr>
<tr><td>1989</td><td>Knuth:TB10-3-325</td><td>Don­ald E. Knuth</td></tr>
+
<tr><td>1989</td><td>Knuth:TB10-3-325</td><td>Donald E. Knuth</td></tr>
<tr><td>1989</td><td>Knuth:TB10-4-529</td><td>Don­ald E. Knuth</td></tr>
+
<tr><td>1989</td><td>Knuth:TB10-4-529</td><td>Donald E. Knuth</td></tr>
<tr><td>1990</td><td>Knuth:TB11-4-489</td><td>Don­ald E. Knuth</td></tr>
+
<tr><td>1990</td><td>Knuth:TB11-4-489</td><td>Donald E. Knuth</td></tr>
<tr><td>1993</td><td>Knuth:TB14-4-387</td><td>Don­ald E. Knuth</td></tr>
+
<tr><td>1993</td><td>Knuth:TB14-4-387</td><td>Donald E. Knuth</td></tr>
<tr><td>1996</td><td>Knuth:TB17-1-29</td><td>Don­ald E. Knuth</td></tr>
+
<tr><td>1996</td><td>Knuth:TB17-1-29</td><td>Donald E. Knuth</td></tr>
<tr><td>1987</td><td>Knuth:TB8-1-14</td><td>Don­ald Knuth and Pierre MacKay</td></tr>
+
<tr><td>1987</td><td>Knuth:TB8-1-14</td><td>Donald Knuth and Pierre MacKay</td></tr>
<tr><td>1981</td><td>Knuth:TB2-3-5</td><td>Don­ald Knuth</td></tr>
+
<tr><td>1981</td><td>Knuth:TB2-3-5</td><td>Donald Knuth</td></tr>
<tr><td>1982</td><td>Knuth:TB3-1-10</td><td>Don­ald Knuth</td></tr>
+
<tr><td>1982</td><td>Knuth:TB3-1-10</td><td>Donald Knuth</td></tr>
<tr><td>1983</td><td>Knuth:TB4-2-64</td><td>Don­ald Knuth</td></tr>
+
<tr><td>1983</td><td>Knuth:TB4-2-64</td><td>Donald Knuth</td></tr>
<tr><td>1986</td><td>Knuth:TB7-2-95</td><td>Don­ald Knuth</td></tr>
+
<tr><td>1986</td><td>Knuth:TB7-2-95</td><td>Donald Knuth</td></tr>
<tr><td>1987</td><td>Knuth:TB8-1-6</td><td>Don­ald Knuth</td></tr>
+
<tr><td>1987</td><td>Knuth:TB8-1-6</td><td>Donald Knuth</td></tr>
<tr><td>1987</td><td>Knuth:TB8-1-73</td><td>Don­ald Knuth</td></tr>
+
<tr><td>1987</td><td>Knuth:TB8-1-73</td><td>Donald Knuth</td></tr>
<tr><td>1987</td><td>Knuth:TB8-2-210</td><td>Don­ald Knuth</td></tr>
+
<tr><td>1987</td><td>Knuth:TB8-2-210</td><td>Donald Knuth</td></tr>
<tr><td>1987</td><td>Knuth:TB8-2-217</td><td>Don­ald Knuth</td></tr>
+
<tr><td>1987</td><td>Knuth:TB8-2-217</td><td>Donald Knuth</td></tr>
<tr><td>1989</td><td>Knuth:TB10-1-8</td><td>Don­ald Knuth</td></tr>
+
<tr><td>1989</td><td>Knuth:TB10-1-8</td><td>Donald Knuth</td></tr>
<tr><td>1989</td><td>Knuth:TB10-1-31</td><td>Don­ald Knuth</td></tr>
+
<tr><td>1989</td><td>Knuth:TB10-1-31</td><td>Donald Knuth</td></tr>
<tr><td>1990</td><td>Knuth:TB11-1-13</td><td>Don­ald Knuth</td></tr>
+
<tr><td>1990</td><td>Knuth:TB11-1-13</td><td>Donald Knuth</td></tr>
<tr><td>1990</td><td>Knuth:TB11-2-165</td><td>Don­ald Knuth</td></tr>
+
<tr><td>1990</td><td>Knuth:TB11-2-165</td><td>Donald Knuth</td></tr>
<tr><td>1990</td><td>Knuth:TB11-4-497</td><td>Don­ald Knuth</td></tr>
+
<tr><td>1990</td><td>Knuth:TB11-4-497</td><td>Donald Knuth</td></tr>
<tr><td>1990</td><td>Knuth:TB11-4-499</td><td>Don­ald Knuth</td></tr>
+
<tr><td>1990</td><td>Knuth:TB11-4-499</td><td>Donald Knuth</td></tr>
<tr><td>1991</td><td>Knuth:TB12-2-313</td><td>Don­ald Knuth</td></tr>
+
<tr><td>1991</td><td>Knuth:TB12-2-313</td><td>Donald Knuth</td></tr>
<tr><td>2010</td><td>Knuth:TB31-2-121</td><td>Don­ald Knuth</td></tr>
+
<tr><td>2010</td><td>Knuth:TB31-2-121</td><td>Donald Knuth</td></tr>
 
</table>
 
</table>
  
 
<p></p>
 
<p></p>
The orig­i­nal data is stored in a <span tag="MKIV" style="font-style:sans;">Lua</span> ta­ble, hashed by tag. Start­ing with <span tag="MKIV" style="font-style:sans;">Lua</span> 5.2 each run of <span tag="MKIV" style="font-style:sans;">Lua</span> gets a dif­fer­ent or­der­ing of such a hash. In older ver­sions, when you looped over a hash, the or­der was un­de­fined, but the same as long as you used the same bi­nary. This had the ad­van­tage that suc­ces­sive runs, some­thing we of­ten have in doc­u­ment pro­cess­ing gave con­sis­tent re­sults. In to­day’s <span tag="MKIV" style="font-style:sans;">Lua</span> we need to do much more sort­ing of hashes be­fore we loop, es­pe­cially when we save multi--pass data. It is for this rea­son that the <span tag="MKIV" style="font-style:sans;">xml</span> tree is sorted by hash key by de­fault. That way lookups (es­pe­cially the first of a set) give con­sis­tent out­comes.
+
The original data is stored in a <span tag="MKIV" style="font-style:sans;">Lua</span> table, hashed by tag. Starting with <span tag="MKIV" style="font-style:sans;">Lua</span> 5.2 each run of <span tag="MKIV" style="font-style:sans;">Lua</span> gets a different ordering of such a hash. In older versions, when you looped over a hash, the order was undefined, but the same as long as you used the same binary. This had the advantage that successive runs, something we often have in document processing gave consistent results. In today’s <span tag="MKIV" style="font-style:sans;">Lua</span> we need to do much more sorting of hashes before we loop, especially when we save multi--pass data. It is for this reason that the <span tag="MKIV" style="font-style:sans;">xml</span> tree is sorted by hash key by default. That way lookups (especially the first of a set) give consistent outcomes.
 
    
 
    
 
    
 
    
<h1>Stan­dards</h1>
+
<h1>Standards</h1>
  
The ren­der­ing of bib­li­o­graphic en­tries is of­ten stan­dard­ized and pre­scribed by the pub­lisher. If you sub­mit an ar­ti­cle to a jour­nal, nor­mally it will be re­for­mat­ted (or even re- keyed) and the ren­der­ing will hap­pen at the pub­lish­ers end. In that case it may not mat­ter how en­tries were ren­dered when writ­ing the pub­li­ca­tion, be­cause the pub­lisher will do it his or her way. This means that most users prob­a­bly will stick to the stan­dard <span tag="MKIV" style="font-style:sans;">apa</span> rules and for them we pro­vide some con­fig­u­ra­tion. Be­cause we use se­tups it is easy to over­load specifics. If you re­ally want to tweak, best look in the files that deal with it.
+
The rendering of bibliographic entries is often standardized and prescribed by the publisher. If you submit an article to a journal, normally it will be reformatted (or even re- keyed) and the rendering will happen at the publishers end. In that case it may not matter how entries were rendered when writing the publication, because the publisher will do it his or her way. This means that most users probably will stick to the standard <span tag="MKIV" style="font-style:sans;">apa</span> rules and for them we provide some configuration. Because we use setups it is easy to overload specifics. If you really want to tweak, best look in the files that deal with it.
 
<p></p>
 
<p></p>
Many stan­dards ex­ist and sup­port for other ren­der­ings may be added to the core. In­ter­ested users are in­vited to de­velop and to test al­ter­nate stan­dard ren­der­ings ac­cord­ing to their needs.
+
Many standards exist and support for other renderings may be added to the core. Interested users are invited to develop and to test alternate standard renderings according to their needs.
 
<p></p>
 
<p></p>
Todo: maybe a list of cat­e­gories and fields.
+
Todo: maybe a list of categories and fields.
 
    
 
    
 
    
 
    
<h1>Clean­ing up</h1>
+
<h1>Cleaning up</h1>
  
Al­though the <span tag="MKIV" style="font-style:sans;">bibTEX</span> for­mat is rea­son­ably well de­fined, in prac­tice there are many ways to or­ga­nize the data. For in­stance, one can use pre­de­fined string con­stants that get used (ei­ther or not com­bined with other strings) later on. A string can be en­closed in curly braces or dou­ble quotes. The strings can con­tain <span tag="MKIV" style="font-style:sans;">TEX</span> com­mands but these are not stan­dard­ized. The data­bases of­ten have some­what com­plex ways to deal with spe­cial char­ac­ters and the use of braces in their de­f­i­n­i­tion is also not nor­mal­ized.
+
Although the <span tag="MKIV" style="font-style:sans;">bibTEX</span> format is reasonably well defined, in practice there are many ways to organize the data. For instance, one can use predefined string constants that get used (either or not combined with other strings) later on. A string can be enclosed in curly braces or double quotes. The strings can contain <span tag="MKIV" style="font-style:sans;">TEX</span> commands but these are not standardized. The databases often have somewhat complex ways to deal with special characters and the use of braces in their definition is also not normalized.
 
<p></p>
 
<p></p>
The most com­plex to deal with are the fields that con­tain names of peo­ple. At some point it might be needed to split a com­bi­na­tion of names into in­di­vid­ual ones that then get split into ti­tle, first name, op­tional in­be­tweens, sur­name(s) and ad­di­tional: <tt style="color:rgb(0,102,102);font-size:100%;" >Prof. Dr. Alfred B. C. von Kwik Kwak Jr. II and P. Q. Olet</tt> is just one ex­am­ple of this. The con­ven­tion seems to be not to use com­mas but <tt style="color:rgb(0,102,102);font-size:100%;" >and</tt> to sep­a­rate names (of­ten each name will be spec­i­fied as last­name, first­name).
+
The most complex to deal with are the fields that contain names of people. At some point it might be needed to split a combination of names into individual ones that then get split into title, first name, optional inbetweens, surname(s) and additional: <tt style="color:rgb(0,102,102);font-size:100%;" >Prof. Dr. Alfred B. C. von Kwik Kwak Jr. II and P. Q. Olet</tt> is just one example of this. The convention seems to be not to use commas but <tt style="color:rgb(0,102,102);font-size:100%;" >and</tt> to separate names (often each name will be specified as lastname, firstname).
 
<p></p>
 
<p></p>
We don’t see it as chal­lenge nor as a duty to sup­port all kinds of messy de­f­i­n­i­tions. Of course we try to be some­what tol­er­ant, but you will be sure to get bet­ter re­sults if you use nicely setup, con­sis­tent data­bases.
+
We don’t see it as challenge nor as a duty to support all kinds of messy definitions. Of course we try to be somewhat tolerant, but you will be sure to get better results if you use nicely setup, consistent databases.
 
<p></p>
 
<p></p>
Todo: maybe some ex­am­ples of bad.
+
Todo: maybe some examples of bad.
 
    
 
    
 
    
 
    
<h1>Tran­si­tion</h1>
+
<h1>Transition</h1>
  
In the orig­i­nal bib­li­og­ra­phy sup­port mod­ule us­age was as fol­lows (ex­am­ple taken from the con­textgar­den wiki):  
+
In the original bibliography support module usage was as follows (example taken from the contextgarden wiki):  
 
      
 
      
 
<pre style="color:rgb(102,0,102);font-size:100%">% engine=pdftex
 
<pre style="color:rgb(102,0,102);font-size:100%">% engine=pdftex
Line 920: Line 920:
 
</pre>
 
</pre>
 
<p></p>
 
<p></p>
For <span tag="MKIV" style="font-style:sans;">MkIV</span> the mod­ules were partly rewrit­ten and ended up in the core so the two com­mands were no longer needed. The over­head as­so­ci­ated with the au­to­matic load­ing of the bib­li­og­ra­phy macros can be ne­glected these days, so stan­dard­ized mod­ules such as <tt style="color:rgb(0,102,102);font-size:100%;" >bib</tt> are all be­ing moved to the core and do not need to be ex­plic­itly loaded.
+
For <span tag="MKIV" style="font-style:sans;">MkIV</span> the modules were partly rewritten and ended up in the core so the two commands were no longer needed. The overhead associated with the automatic loading of the bibliography macros can be neglected these days, so standardized modules such as <tt style="color:rgb(0,102,102);font-size:100%;" >bib</tt> are all being moved to the core and do not need to be explicitly loaded.
 
<p></p>
 
<p></p>
The first <tt style="color:rgb(0,102,102);font-size:100%;" >\setupbibtex</tt> com­mand in this ex­am­ple is needed to boot­strap the process: it tells what data­base has to be processed by <span tag="MKIV" style="font-style:sans;">bibTEX</span> be­tween runs. The sec­ond <tt style="color:rgb(0,102,102);font-size:100%;" >\setuppublications</tt> com­mand is op­tional. Each ci­ta­tion (tagged with <tt style="color:rgb(0,102,102);font-size:100%;" >\cite</tt>) ends up in the list of pub­li­ca­tions.
+
The first <tt style="color:rgb(0,102,102);font-size:100%;" >\setupbibtex</tt> command in this example is needed to bootstrap the process: it tells what database has to be processed by <span tag="MKIV" style="font-style:sans;">bibTEX</span> between runs. The second <tt style="color:rgb(0,102,102);font-size:100%;" >\setuppublications</tt> command is optional. Each citation (tagged with <tt style="color:rgb(0,102,102);font-size:100%;" >\cite</tt>) ends up in the list of publications.
 
<p></p>
 
<p></p>
In the new ap­proach we no longer use <span tag="MKIV" style="font-style:sans;">bibTEX</span>so we don’t need to setup <span tag="MKIV" style="font-style:sans;">bibTEX</span>. In­stead we de­fine dataset(s). We also no longer set up pub­li­ca­tions with one com­mand, but have split that up in ren­der­ing-, list-, and cite-vari­ants. The ba­sic <tt style="color:rgb(0,102,102);font-size:100%;" >\cite</tt> com­mand re­mains. The above ex­am­ple be­comes:  
+
In the new approach we no longer use <span tag="MKIV" style="font-style:sans;">bibTEX</span>so we don’t need to setup <span tag="MKIV" style="font-style:sans;">bibTEX</span>. Instead we define dataset(s). We also no longer set up publications with one command, but have split that up in rendering-, list-, and cite-variants. The basic <tt style="color:rgb(0,102,102);font-size:100%;" >\cite</tt> command remains. The above example becomes:  
 
      
 
      
 
<pre style="color:rgb(102,0,102);font-size:100%">\definebtxdataset
 
<pre style="color:rgb(102,0,102);font-size:100%">\definebtxdataset
Line 943: Line 943:
 
</pre>
 
</pre>
 
<p></p>
 
<p></p>
So, we have a few more com­mands to set up things. If you in­tend to use just a sin­gle dataset and ren­der­ing, the above pre­am­ble can be sim­pli­fied to:  
+
So, we have a few more commands to set up things. If you intend to use just a single dataset and rendering, the above preamble can be simplified to:  
 
      
 
      
 
<pre style="color:rgb(102,0,102);font-size:100%">\usebtxdataset
 
<pre style="color:rgb(102,0,102);font-size:100%">\usebtxdataset
Line 951: Line 951:
 
</pre>
 
</pre>
 
<p></p>
 
<p></p>
But keep in mind that com­pared to the old <span tag="MKIV" style="font-style:sans;">MkII</span> de­rived method we have moved some of the op­tions to the ren­der­ing, list and cite setup vari­ants.
+
But keep in mind that compared to the old <span tag="MKIV" style="font-style:sans;">MkII</span> derived method we have moved some of the options to the rendering, list and cite setup variants.
 
<p></p>
 
<p></p>
An­other dif­fer­ence is now the use of lists. When you de­fine a ren­der­ing, you also de­fine a list. How­ever, all en­tries are col­lected in a com­mon list tagged <tt style="color:rgb(0,102,102);font-size:100%;" >btx</tt>. Al­though you will nor­mally con­fig­ure a ren­der­ing you can still set some prop­er­ties of lists, but in that case you need to pre­fix the list iden­ti­fier. In the case of the above ex­am­ple this is <tt style="color:rgb(0,102,102);font-size:100%;" >btx:document</tt>.
+
Another difference is now the use of lists. When you define a rendering, you also define a list. However, all entries are collected in a common list tagged <tt style="color:rgb(0,102,102);font-size:100%;" >btx</tt>. Although you will normally configure a rendering you can still set some properties of lists, but in that case you need to prefix the list identifier. In the case of the above example this is <tt style="color:rgb(0,102,102);font-size:100%;" >btx:document</tt>.
 
    
 
    
 
    
 
    
<h1>ML­BIBTEX</h1>
+
<h1>MLBIBTEX</h1>
  
Todo: how to plug in <span tag="MKIV" style="font-style:sans;">ML­bibTEX</span> for sort­ing and other ad­vanced op­er­a­tions.
+
Todo: how to plug in <span tag="MKIV" style="font-style:sans;">MLbibTEX</span> for sorting and other advanced operations.
 
    
 
    
 
    
 
    
<h1>Ex­ten­sions</h1>
+
<h1>Extensions</h1>
  
As <span tag="MKIV" style="font-style:sans;">TEX</span> and <span tag="MKIV" style="font-style:sans;">Lua</span> are both open and ac­ces­si­ble in <span tag="MKIV" style="font-style:sans;">ConTEXt</span> it is pos­si­ble to ex­tend the func­tion­al­ity of the bib­li­og­ra­phy re­lated code. For in­stance, you can add ex­tra load­ers.  
+
As <span tag="MKIV" style="font-style:sans;">TEX</span> and <span tag="MKIV" style="font-style:sans;">Lua</span> are both open and accessible in <span tag="MKIV" style="font-style:sans;">ConTEXt</span> it is possible to extend the functionality of the bibliography related code. For instance, you can add extra loaders.  
 
      
 
      
 
<pre style="color:rgb(102,0,102);font-size:100%">function publications.loaders.myformat(dataset,filename)
 
<pre style="color:rgb(102,0,102);font-size:100%">function publications.loaders.myformat(dataset,filename)
Line 974: Line 974:
 
</pre>
 
</pre>
 
<p></p>
 
<p></p>
This then per­mits load­ing a data­base (into a dataset) with the com­mand:  
+
This then permits loading a database (into a dataset) with the command:  
 
      
 
      
 
<pre style="color:rgb(102,0,102);font-size:100%">\usebtxdataset[standard][myfile.myformat]
 
<pre style="color:rgb(102,0,102);font-size:100%">\usebtxdataset[standard][myfile.myformat]
 
</pre>
 
</pre>
 
<p></p>
 
<p></p>
The <tt style="color:rgb(0,102,102);font-size:100%;" >myformat</tt> suf­fix is rec­og­nized au­to­mat­i­cally. If you want to use an­other suf­fix, you can do this:  
+
The <tt style="color:rgb(0,102,102);font-size:100%;" >myformat</tt> suffix is recognized automatically. If you want to use another suffix, you can do this:  
 
      
 
      
 
<pre style="color:rgb(102,0,102);font-size:100%">\usebtxdataset[standard][myformat::myfile.txt]
 
<pre style="color:rgb(102,0,102);font-size:100%">\usebtxdataset[standard][myformat::myfile.txt]
 
</pre>
 
</pre>
 +
 
 +
 
 +
<h1>Notes</h1>
 +
 +
The move from external <span tag="MKIV" style="font-style:sans;">bibTEX</span> processing to internal processing has the advantage that we stay within the same run. In the traditional approach we had roughly the following steps:
 +
   
 +
<div >
 +
 +
       
 +
<div >
 +
 +
          <span>• </span>
 +
      <span>the first run information is collected and written to file</span>
 +
   
 +
</div>
 +
 +
       
 +
<div >
 +
 +
          <span>• </span>
 +
      <span>after that run the <span tag="MKIV" style="font-style:sans;">bibTEX</span> program converts that file to another one</span>
 +
   
 +
</div>
 +
 +
       
 +
<div >
 +
 +
          <span>• </span>
 +
      <span>successive runs use that data for references and producing lists</span>
 +
   
 +
</div>
 +
 +
   
 +
</div>
 +
 +
<p></p>
 +
In the <span tag="MKIV" style="font-style:sans;">MkIV</span> approach the bibliographic database is loaded in memory each run and processing also happens each run. On paper this looks less efficient but as <span tag="MKIV" style="font-style:sans;">Lua</span> is quite fast, in practice performance is much better.
 +
<p></p>
 +
Probably most demanding is the treatment of authors as we have to analyze names, split multiple authors and reassemble firstnames, vons, surnames and juniors. When we sort by author sorting vectors have to be made which also has a penalty. However, in practice the user will not notice a performance degradation. We did some tests with a list of 500.000 authors, sorted them and typeset them as list (producing some 5400 dense pages in a small font and with small margins). This is typical one of these cases where using <span tag="MKIV" style="font-style:sans;">LuajitTEX</span> saves quite time. On my machine it took just over 100 seconds to get this done. Unfortunately not all operating systems performed equally well: 32 bit versions worked fine, but 64 bit <span tag="MKIV" style="font-style:sans;">linux</span> either crashed (stalled) the machine or ran out of memory rather fast, while <span tag="MKIV" style="font-style:sans;">MacOSX</span> and <span tag="MKIV" style="font-style:sans;">Windows</span> performed fine. In practice you will never run into this, unless you produce massive amounts of bibliographic entries. <span tag="MKIV" style="font-style:sans;">LuaJIT</span> has some benefits but also some drawbacks.

Revision as of 09:05, 21 January 2014

The database

The bibTEX format is rather popular in the TEX community and even with its shortcomings it will stay around for a while. Many publication websites can export and many tools are available to work with this database format. It is rather simple and looks a bit like Lua tables. Unfortunately the content can be polluted with non-standardized TEX commands which complicates pre- or postprocessing outside TEX. In that sense a bibTEX database is often not coded neutrally. Some limitations, like the use of commands to encode accented characters root in the ascii world and can be bypassed by using utf instead (as handled somewhat in LATEX through extensions such as bibtex8).

The normal way to deal with a bibliography is to refer to entries using a unique tag or key. When a list of entries is typeset, this reference can be used for linking purposes. The typeset list can be processed and sorted using the bibtex program that converts the database into something more TEX friendly (a .bbl file). I never used the program myself (nor bibliographies) so I will not go into too much detail here, if only because all I say can be wrong.

In ConTEXt we no longer use the bibtex program: we just use database files and deal with the necessary manipulations directly in ConTEXt. One or more such databases can be used and combined with additional entries defined within the document. We can have several such datasets active at the same time.

A bibTEX file looks like this:

@Article{sometag,
    author  = "An Author and Another One",
    title   = "A hopefully meaningful title",
    journal = maps,
    volume  = "25",
    number  = "2",
    pages   = "5--9",
    month   = mar,
    year    = "2013",
    ISSN    = "1234-5678",
}

Normally a value is given between quotes (or curly brackets) but single words are also OK (there is no real benefit in not using quotes, so we advise to always use them). There can be many more fields and instead of strings one can use predefined shortcuts. The title for example quite often contains TEX macros. Some fields, like pages have funny characters such as the endash (typically as --) so we have a mixture of data and typesetting directives. If you are covering non--english references, you often need characters that are not in the ascii subset but ConTEXt is quite happy with utf. If your database file uses old-fashioned TEX accent commands then these will be internally converted automatically to utf. Commands (macros) are converted to an indirect call, which is quite robust.

The bibTEX files are loaded in memory as Lua table but can be converted to xml so that we can access them in a more flexible way, but that is a subject for specialists.

In the old MkII setup we have two kinds of entries: the ones that come from the bibTEX run and user supplied ones. We no longer rely on bibTEX output but we do still support the user supplied definitions. These were in fact prepared in a way that suits the processing of bibTEX generated entries. The next variant reflects the ConTEXt recoding of the old bibTEX output.

\startpublication[k=Hagen:Second,t=article,a={Hans Hagen},y=2013,s=HH01]
    \artauthor[]{Hans}[H.]{}{Hagen}
    \arttitle{Who knows more?}
    \journal{MyJournal}
    \pubyear{2013}
    \month{8}
    \volume{1}
    \issue{3}
    \issn{1234-5678}
    \pages{123--126}
\stoppublication

The split \artauthor fields are collapsed into a single author field as we deal with the splitting later when it gets parsed in Lua. The \artauthor syntax is only kept around for backward compatibility with the previous use of bibTEX.

In the new setup we support these variants as well:

\startpublication[k=Hagen:Third,t=article]
    \author{Hans Hagen}
    \title{Who knows who?}
    ...
\stoppublication

and

\startpublication[tag=Hagen:Third,category=article]
    \author{Hans Hagen}
    \title{Who knows who?}
    ...
\stoppublication

and

\startpublication
    \tag{Hagen:Third}
    \category{article}
    \author{Hans Hagen}
    \title{Who knows who?}
    ...
\stoppublication

Because internally the entries are Lua tables, we also support loading of Lua based definitions:

return {
    ["Hagen:First"] = {
        author   = "Hans Hagen",
        category = "article",
        issn     = "1234-5678",
        issue    = "3",
        journal  = "MyJournal",
        month    = "8",
        pages    = "123--126",
        tag      = "Hagen:First",
        title    = "Who knows nothing?",
        volume   = "1",
        year     = "2013",
    },
}

Files set up like this can be loaded too. The following xml input is rather close to this, and is also accepted as input.

<?xml version="2.0" standalone="yes" ?>
<bibtex>
    <entry tag="Hagen:First" category="article">
        <field name="author">Hans Hagen</field>
        <field name="category">article</field>
        <field name="issn">1234-5678</field>
        <field name="issue">3</field>
        <field name="journal">MyJournal</field>
        <field name="month">8</field>
        <field name="pages">123--126</field>
        <field name="tag">Hagen:First</field>
        <field name="title">Who knows nothing?</field>
        <field name="volume">1</field>
        <field name="year">2013</field>
    </entry>
</bibtex>

Todo: Add some remarks about loading EndNote and RIS formats, but first we need to complete the tag mapping (on Alan’s plate).

So the user has a rather wide choice of formatting style for bibliography database files.

You can load more data than you actually need. Only entries that are referred to explicitly through the \cite and \nocite commands will be shown in lists. We will cover these details later.

Commands in entries

One unfortunate aspect commonly found in bibTEX files is that they often contain TEX commands. Even worse is that there is no standard on what these commands can be and what they mean, at least not formally, as bibTEX is a program intended to be used with many variants of TEX style: plain, LATEX, and others. This means that we need to define our use of these typesetting commands. However, in most cases, they are just abbreviations or font switches and these are often known. Therefore, ConTEXt will try to resolve them before reporting an issue. In the log file there is a list of commands that has been seen in the loaded databases. For instance, loading tugboat.bib gives a long list of commands of which we show a small set here:

publications > start used btx commands
publications > standard CONTEXT 1 known
publications > standard ConTeXt 4 known
publications > standard TeXLive 3 KNOWN
publications > standard eTeX    1 known
publications > standard hbox    6 known
publications > standard sltt    1 unknown
publications > stop used btxcommands

You can define unknown commands, or overload existing definitions in the following way:

\definebtxcommand\TUB {TUGboat}
\definebtxcommand\sltt{\tt}
\definebtxcommand\<#1>{\type{#1}}

Unknown commands do not stall processing, but their names are then typeset in a mono- spaced font so they probably stand out for proofreading. You can access the commands with \btxcommand{...}, as in:

commands like \btxcommand{MySpecialCommand} are handled in an indirect way

As this is an undefined command we get: “commands like MySpecialCommand are handled in an indirect way”.

??


Datasets

Normally in a document you will use only one bibliographic database, whether or not distributed over multiple files. Nevertheless we support multiple databases as well which is why we talk of datasets instead. A dataset is loaded with the \usebtxdataset command. Although currently it is not necessary to define a (default) dataset you can best do this because in the future we might provide more options. Here are some examples:

\definebtxdataset[standard]
\usebtxdataset[standard][tugboat.bib]
\usebtxdataset[standard][mtx-bibtex-output.xml]
\usebtxdataset[standard][test-001-btx-standard.lua]

These three suffixes are understood by the loader. Here the dataset has the name standard and the three database files are merged, where later entries having the same tag overload previous ones. Definitions in the document source (coded in TEX speak) are also added, and they are saved for successive runs. This means that if you load and define entries, they will be known at a next run beforehand, so that references to them are independent of when loading and definitions take place.

setup definition setupbtxdataset

setup definition definebtxdataset

setup definition usebtxdataset

In this document we use some example databases, so let’s load one of them now:

\definebtxdataset[example]
\usebtxdataset[example][mkiv-publications.bib]

You can ask for an overview of entries in a dataset with:

\showbtxdatasetfields[example]

this gives:

tagcategoryfields
demo-001bookauthor index title year
demo-002bookcrossref index year
demo-003bookauthor comment index title year
demo-004bookauthor comment index title year
demo-005bookauthor doi index pages serial title url year

You can set the current active dataset with

\setbtxdataset[standard]

but most publication-related commands accept optional arguments that denote the dataset and references to entries can be prefixed with a dataset identifier.. More about that later.

Sometimes you want to check a database. One way of doing that is the following:

\definebtxdataset[check]
\usebtxdataset[check][mkiv-publications-check.bib]
\showbtxdatasetcompleteness[check]

The database like like this:

The completeness check shows (with green field names) the required fields and when one is missing this is indicated in red. In blue we show what gets inherited.


Renderings

A list of publications can be rendered at any place in the document. A database can be much larger than needed for a document. The same is true for the fields that make up an entry. Here is the list of fields that are currently handled, but of course there can be additional ones:

abstract, address, annotate, assignee, author, bibnumber, booktitle, chapter, comment, country, day, dayfiled, doi, edition, editor, eprint, howpublished, institution, isbn, issn, journal, key, keyword, keywords, language, lastchecked, month, monthfiled, names, nationality, note, notes, number, organization, pages, publisher, revision, school, series, size, title, type, url, volume, year, yearfiled

If you want to see what publications are in the database, the easiest way is to ask for a complete list:

\definebtxrendering
  [example]
  [dataset=example,
   method=local,
   alternative=apa]
\placelistofpublications % \placebtxrendering
  [example]
  [criterium=all]

This gives:1 Hagen, H. and Otten, T. (1996). Typesetting education documents.2 Scarso, L. (2021). Designing high speed trains.3 author (year). title. pages p.

The rendering itself is somewhat complex to set up because we have not only many different standards but also many fields that can be set up. This means that there are several commands involved. Often there is a prescribed style to render bibliographic descriptions, for example apa. A rendering is setup and defined with:

setup definition setupbtxrendering

setup definition definebtxrendering

And a list of such descriptions is generated with:

setup definition placebtxrendering

A dataset can have all kind of entries:

Each has its own rendering variant. To keep things simple we have their settings separated. However, these settings are shared for all rendering alternatives. In practice this is seldom a problem in a publication as only one rendering alternative will be active. If this be not sufficient, you can always group local settings in a setup and hook that into the specific rendering.

setup definition setupbtxlistvariant

setup definition definebtxlistvariant

Examples of list variants are:

setupbtxlistvariant : artauthor

no specific settings

setupbtxlistvariant : author

no specific settings

setupbtxlistvariant : editor

no specific settings

The exact rendering of list entries is determined by the alternative key and defaults to apa which uses definitions from publ-imp-apa.mkiv. If you look at that file you will see that each category has its own setup. You may also notice that additional tests are needed to make sure that empty fields don’t trigger separators and such.

There are a couple of accessors and helpers to get the job done. When you want to fetch a field from the current entry you use \btxfield. In most cases you want to make sure this field has a value, for instance because you don’t want fences or punctuation that belongs to a field.

\btxdoif {title} {
    \bold{\btxfield{title}},
}

There are three test macros:

\btxdoifelse{fieldname}{action when found}{action when not found}
\btxdoif    {fieldname}{action when found}
\btxdoifnot {fieldname}                   {action when not found}

An extra conditional is available for testing interactivity:

\btxdoifelseinteraction{action when true}{action when false}

In addition there is also a conditional \btxinteractive which is more efficient, although in practice efficiency is not so important here.

There are three commands to flush data:

\btxfieldfetch a explicit field (e.g. year)
\btxdetailfetch a derived field (e.g. short)
\btxflushfetch a derived or explicit field

Normally you can use \btxfield or \btxflush as derived fields just like analyzed author fields are flushed in a special way.

You can improve readability by using setups, for instance:

\btxdoifelse {author} {
    \btxsetup{btx:apa:author:yes}
} {
    \btxsetup{btx:apa:author:nop}
}

Keep in mind that normally you don’t need to mess with definitions like this because standard rendering styles are provided. These styles use a few helpers that inject symbols but also take care of leading and trailing spaces:

\btxspacebefore after
\btxperiodbefore. after
\btxcommabefore, after
\btxlparentbefore (after
\btxrparentbefore) after
\btxlbracketbefore [after
\btxrbracketbefore] after

So, the previous example setup can be rewritten as:

\btxdoif {title} {
    \bold{\btxfield{title}}
    \btxcomma
}

There is a special command for rendering a (combination) of authors:

\btxflushauthor{author}
\btxflushauthor{editor}
\btxflushauthor[inverted]{editor}

Instead of the last one you can also use:

\btxflushauthorinverted{editor}

You can use a (configurable) default or pass directives: Valid directives are

conversionrendering
invertedthe Frog jr, Kermit
invertedshortthe Frog jr, K
normalKermit, the Frog, jr
normalshortK, the Frog, jr


Citations

Citations are references to bibliographic entries that normally show up in lists someplace in the document: at the end of a chapter, in an appendix, at the end of an article, etc. We discussed the rendering of these lists in the previous chapter. A citation is normally pretty short as its main purpose is to refer uniquely to a more detailed description. But, there are several ways to refer, which is why the citation subsystem is configurable and extensible. Just look at the following commands:

\cite[author][example::demo-003]
\cite[authoryear][example::demo-003]
\cite[authoryears][example::demo-003]
\cite[author][example::demo-003,demo-004]
\cite[authoryear][example::demo-003,demo-004]
\cite[authoryears][example::demo-003,demo-004]
\cite[author][example::demo-004,demo-003]
\cite[authoryear][example::demo-004,demo-003]
\cite[authoryears][example::demo-004,demo-003]
    • (Hans Hagen and Ton Otten)
      (Hans Hagen and Ton Otten (1996))
      (Hans Hagen and Ton Otten, 1996)
      (Hans Hagen and Ton Otten, Luigi Scarso)
      (Hans Hagen and Ton Otten (1996), Luigi Scarso (2021))
      (Hans Hagen and Ton Otten, 1996, Luigi Scarso, 2021)
      (Luigi Scarso, Hans Hagen and Ton Otten)
      (Luigi Scarso (2021), Hans Hagen and Ton Otten (1996))
      (Luigi Scarso, 2021, Hans Hagen and Ton Otten, 1996)
  • The first argument is optional.

    setup definition cite

    You can tune the way a citation shows up:

    \setupbtxcitevariant[author]     [sorttype=author,color=darkyellow]
    \setupbtxcitevariant[authoryear] [sorttype=author,color=darkyellow]
    \setupbtxcitevariant[authoryears][sorttype=author,color=darkyellow]
    \cite[author][example::demo-004,demo-003]
    \cite[authoryear][example::demo-004,demo-003]
    \cite[authoryears][example::demo-004,demo-003]
    

    Here we sort the authors and color the citation:

    • (Hans Hagen and Ton Otten, Luigi Scarso)
      (Hans Hagen and Ton Otten (1996), Luigi Scarso (2021))
      (Hans Hagen and Ton Otten, 1996, Luigi Scarso, 2021)
  • For reasons of backward compatibility the \cite command is a bit picky about spaces between the two arguments, of which the first is optional. This is a consequence of allowing its use with the key specified between curly brackets as is the traditional practice. (We do encourage users to adopt the more coherent ConTEXt syntax by using square brackets for keywords and reserving curly brackets to regroup text to be typeset.)

    The \citation command is synonymous but is more flexible with respect to spacing of its arguments:

    \citation[author]     [example::demo-004,demo-003]
    \citation[authoryear] [example::demo-004,demo-003]
    \citation[authoryears][example::demo-004,demo-003]
    

    There is a whole bunch of cite options and more can be easily defined.

    keyrendering
    author(author)
    authornum[author [btx error 1]]
    authoryear(author (year))
    authoryears(author, year)
    doi[todo: doi]
    key[demo-005]
    none
    numbtx error 1
    pagepages
    serial[5]
    short[aut00]
    type[book]
    url[todo: url]
    year(year)

    Because we are dealing with database input and because we generally need to manipulate entries, much of the work is delegated to Lua. This makes it easier to maintain and extend the code. Of course TEX still does the rendering. The typographic details are controlled by parameters but not all are used in all variants. As with most ConTEXt commands, it starts out with a general setup command:

    setup definition setupbtxcitevariant

    On top of that we can define instances that inherit either from a given parent or from the topmost setup.

    setup definition definebtxcitevariant

    But, specific variants can have them overloaded:

    setupbtxcitevariant : author

    right)
    middle,
    left(

    setupbtxcitevariant : authornum

    right]
    middle,
    left[

    setupbtxcitevariant : authoryear

    compressyes
    inbetween,
    right)
    middle,
    left(

    setupbtxcitevariant : authoryears

    compressyes
    inbetween,
    right)
    middle,
    left(

    setupbtxcitevariant : doi

    right]
    left[

    setupbtxcitevariant : key

    right]
    left[

    setupbtxcitevariant : none

    no specific settings

    setupbtxcitevariant : num

    compressyes
    inbetween--
    right]
    left[

    setupbtxcitevariant : page

    inbetween

    setupbtxcitevariant : serial

    right]
    left[

    setupbtxcitevariant : short

    right]
    left[

    setupbtxcitevariant : type

    right]
    left[

    setupbtxcitevariant : url

    right]
    left[

    setupbtxcitevariant : year

    right)
    left(

    A citation variant is defined in several steps and if you really want to know the dirty details, you should look into the publ-imp-*.mkiv files. Here we stick to the concept.

    \startsetups btx:cite:author
        \btxcitevariant{author}
    \stopsetups
    

    You can overload such setups if needed, but that only makes sense when you cannot configure the rendering with parameters. The \btxcitevariant command is one of the build in accessors and it calls out to Lua where more complex manipulation takes place if needed. If no manipulation is known, the field with the same name (if found) will be flushed. A command like \btxcitevariant assumes that a dataset and specific tag has been set. This is normally done in the wrapper macros, like \cite. For special purposes you can use these commands

    \setbtxdataset[example]
    \setbtxentry[hh2013]
    

    But don’t expect too much support for such low level rendering control.

    Unless you use criterium=all only publications that are cited will end up in the lists. You can force a citation into a list using \usecitation, for example:

    \usecitation[example::demo-004,demo-003]
    

    This command has two synonyms: \nocite and \nocitation so you can choose whatever fits you best.

    setup definition nocite


    The LUA view

    Because we manage data at the Lua end it is tempting to access it there for other purposes. This is fine as long as you keep in mind that aspects of the implementation may change over time, although this is unlikely once the modules become stable.

    The entries are collected in datasets and each set has a unique name. In this document we have the set named example. A dataset table has several fields, and probably the one of most interest is the luadata field. Each entry in this table describes a publication:

    t={ 
     ["author"]="Hans Hagen", 
     ["category"]="book", 
     ["index"]=1, 
     ["tag"]="demo-001", 
     ["title"]="\\btxcmd{BIBTEX}, the \\btxcmd{CONTEXT}\\ way", 
     ["year"]="2013", 
    } 
    

    This is publications.datasets.example.luadata["demo-001"]. There can be a companion entry in the parallel details table.

    t={ 
     ["author"]={ 
      { 
       ["firstnames"]={ "Hans" }, 
       ["initials"]={ "H" }, 
       ["original"]="Hans Hagen", 
       ["surnames"]={ "Hagen" }, 
       ["vons"]={}, 
      }, 
     }, 
     ["short"]="Hag13", 
    } 
    

    These details are accessed as publications.datasets.example.details["demo-001"] and by using a separate table we can overload fields in the original entry without losing the original.

    You can loop over the entries using regular Lua code combined with MkIV helpers:

    local dataset = publications.datasets.example
    context.starttabulate { "|l|l|l|" }
    for tag, entry in table.sortedhash(dataset.luadata) do
        local detail = dataset.details[tag] or { }
        context.NC() context.type(tag)
        context.NC() context(detail.short)
        context.NC() context(entry.title)
        context.NC() context.NR()
    end
    context.stoptabulate()
    

    This results in:

    demo-001Hag13bibTEX, the ConTEXt way
    demo-002Hag14bibTEX, the ConTEXt way
    demo-003HO96Typesetting education documents
    demo-004Sca21Designing high speed trains
    demo-005aut00title

    You can manipulate a dataset after loading. Of course this assumes that you know what kind of content you have and what you need for rendering. As example we load a small dataset.

    \definebtxdataset[drumming]
    \usebtxdataset[drumming][mkiv-publications.lua]
    

    Because we’re going to do some Lua, we could also have loaded the dataset with:

    publications.load("drumming","mkiv-publications.lua","lua")
    

    The dataset has three entries:

    As you can see, we can have a subtitle. We will combine the title and subtitle into one:

    \startluacode
    for tag, entry in next, publications.datasets.drumming.luadata do
        if entry.subtitle then
            if entry.title then
                entry.title = entry.title .. ", " .. entry.subtitle
            else
                entry.title = entry.subtitle
            end
            entry.subtitle = nil
            logs.report("btx","combining title and subtitle of entry tagged %a",tag)
        end
    end
    \stopluacode
    

    We can now typeset the entries with:

    \definebtxrendering[drumming][dataset=drumming,method=dataset]
    \placebtxrendering[drumming]
    

    Because we just want to show the entries, and have no citations that force them to be shown, we have to the method to dataset.1


    The XML view

    The luadata table can be converted into an xml representation. This is a follow up on earlier experiments with an xml-only approach. I decided in the end to stick to a Lua approach and provide some simple xml support in addition.

    Once a dataset is accessible as xml tree, you can use the regular \xml... commands. We start with loading a dataset, in this case from just one file.

    \usebtxdataset[tugboat][tugboat.bib]
    

    The dataset has to be converted to xml:

    \convertbtxdatasettoxml[tugboat]
    

    The tree is now accessible by its root reference btx:tugboat. If we want simple field access we can use a few setups:

    \startxmlsetups btx:initialize
        \xmlsetsetup{#1}{bibtex|entry|field}{btx:*}
        \xmlmain{#1}
    \stopxmlsetups
    \startxmlsetups btx:field
        \xmlflushcontext{#1}
    \stopxmlsetups
    \xmlsetup{btx:tugboat}{btx:initialize}
    

    The two setups are predefined in the core already, but you might want to change them. They are applied in for instance:

    \starttabulate[|||]
        \NC \type {tag}   \NC \xmlfirst {btx:tugboat}
            {/bibtex/entry[string.find(@tag,'Hagen')]/attribute('tag')}
        \NC \NR
        \NC \type {title} \NC \xmlfirst {btx:tugboat}
            {/bibtex/entry[string.find(@tag,'Hagen')]/field[@name='title']}
        \NC \NR
    \stoptabulate
    
    tagHagen:TB17-1-54
    titlePPCHTEX: typesetting chemical formulas in TEX


    \startxmlsetups btx:demo
        \xmlcommand
            {#1}
            {/bibtex/entry[string.find(@tag,'Hagen')][1]}{btx:table}
    \stopxmlsetups
    \startxmlsetups btx:table
    \starttabulate[|||]
        \NC \type {tag}   \NC \xmlatt{#1}{tag} \NC \NR
        \NC \type {title} \NC \xmlfirst{#1}{/field[@name='title']} \NC \NR
    \stoptabulate
    \stopxmlsetups
    \xmlsetup{btx:tugboat}{btx:demo}
    
    tagHagen:TB17-1-54
    titlePPCHTEX: typesetting chemical formulas in TEX

    Here is another example:

    \startxmlsetups btx:row
        \NC \xmlatt{#1}{tag}
        \NC \xmlfirst{#1}{/field[@name='title']}
        \NC \NR
    \stopxmlsetups
    \startxmlsetups btx:demo
        \xmlfilter {#1} {
            /bibtex
            /entry[@category='article']
            /field[@name='author' and (find(text(),'Knuth') or find(text(),'DEK'))]
            /../command(btx:row)
        }
    \stopxmlsetups
    \starttabulate[|||]
        \xmlsetup{btx:tugboat}{btx:demo}
    \stoptabulate
    
    Knuth:TB10-1-31Typesetting Concrete Mathematics
    Knuth:TB10-1-8TEX would find it difficult …
    Knuth:TB10-3-325The new versions of TEX and MF
    Knuth:TB10-4-529The errors of TEX
    Knuth:TB11-1-13Virtual Fonts: More Fun for Grand Wizards
    Knuth:TB11-2-165Exercises for TEX: The Program
    Knuth:TB11-4-489The future of TEX and MF
    Knuth:TB11-4-497Arthur Lee Samuel, 1901--1990
    Knuth:TB11-4-499Answers to Exercises for TEX: The Program
    Knuth:TB12-2-313Fixed-point glue setting: Errata
    Knuth:TB14-4-387Icons for TEX and MF
    Knuth:TB17-1-29Important message regarding CM fonts
    Knuth:TB2-3-5The current state of things
    Knuth:TB3-1-10Fixed-point glue settingDashan example of WEB
    Knuth:TB31-2-121An Earthshaking Announcement
    Knuth:TB4-2-64A note on hyphenation
    Knuth:TB5-1-4TEX incunabula
    Knuth:TB5-1-67Comments on quality in publishing
    Knuth:TB5-2-105A course on MF programming
    Knuth:TB6-1-36Recipes and fractions
    Knuth:TB7-2-101The TEX logo in various fonts
    Knuth:TB7-2-95Remarks to celebrate the publication of Computers & Typesetting
    Knuth:TB8-1-14Mixing right-to-left texts with left-to-right texts
    Knuth:TB8-1-6It happened: announcement of TEX 2.1
    Knuth:TB8-1-73Problem for a Saturday afternoon
    Knuth:TB8-2-135Fonts for digital halftones
    Knuth:TB8-2-210Saturday morning problemDashsolution
    Knuth:TB8-2-217Reply: Printing out selected pages
    Knuth:TB8-3-309Macros for Jill
    Knuth:TB9-2-152A Punk Meta-Font

    A more extensive example is the following. Of course this assumes that you know what xml support mechanisms and macros are available.

    \startxmlsetups btx:getkeys
        \xmladdsortentry{btx}{#1}{\xmlfilter{#1}{/field[@name='author']/text()}}
        \xmladdsortentry{btx}{#1}{\xmlfilter{#1}{/field[@name='year'  ]/text()}}
        \xmladdsortentry{btx}{#1}{\xmlatt{#1}{tag}}
    \stopxmlsetups
    \startxmlsetups btx:sorter
        \xmlresetsorter{btx}
      % \xmlfilter{#1}{entry/command(btx:getkeys)}
        \xmlfilter{#1}{
            /bibtex
            /entry[@category='article']
            /field[@name='author' and find(text(),'Knuth')]
            /../command(btx:getkeys)}
        \xmlsortentries{btx}
        \starttabulate[||||]
            \xmlflushsorter{btx}{btx:entry:flush}
        \stoptabulate
    \stopxmlsetups
    \startxmlsetups btx:entry:flush
        \NC \xmlfilter{#1}{/field[@name='year'  ]/context()}
        \NC \xmlatt{#1}{tag}
        \NC \xmlfilter{#1}{/field[@name='author']/context()}
        \NC \NR
    \stopxmlsetups
    \xmlsetup{btx:tugboat}{btx:sorter}
    
    1984Knuth:TB5-1-67Don Knuth
    1984Knuth:TB5-1-4Donald E. Knuth
    1984Knuth:TB5-2-105Donald E. Knuth
    1985Knuth:TB6-1-36Donald E. Knuth
    1986Knuth:TB7-2-101Donald E. Knuth
    1987Knuth:TB8-2-135Donald E. Knuth
    1987Knuth:TB8-3-309Donald E. Knuth
    1988Knuth:TB9-2-152Donald E. Knuth
    1989Knuth:TB10-3-325Donald E. Knuth
    1989Knuth:TB10-4-529Donald E. Knuth
    1990Knuth:TB11-4-489Donald E. Knuth
    1993Knuth:TB14-4-387Donald E. Knuth
    1996Knuth:TB17-1-29Donald E. Knuth
    1987Knuth:TB8-1-14Donald Knuth and Pierre MacKay
    1981Knuth:TB2-3-5Donald Knuth
    1982Knuth:TB3-1-10Donald Knuth
    1983Knuth:TB4-2-64Donald Knuth
    1986Knuth:TB7-2-95Donald Knuth
    1987Knuth:TB8-1-6Donald Knuth
    1987Knuth:TB8-1-73Donald Knuth
    1987Knuth:TB8-2-210Donald Knuth
    1987Knuth:TB8-2-217Donald Knuth
    1989Knuth:TB10-1-8Donald Knuth
    1989Knuth:TB10-1-31Donald Knuth
    1990Knuth:TB11-1-13Donald Knuth
    1990Knuth:TB11-2-165Donald Knuth
    1990Knuth:TB11-4-497Donald Knuth
    1990Knuth:TB11-4-499Donald Knuth
    1991Knuth:TB12-2-313Donald Knuth
    2010Knuth:TB31-2-121Donald Knuth

    The original data is stored in a Lua table, hashed by tag. Starting with Lua 5.2 each run of Lua gets a different ordering of such a hash. In older versions, when you looped over a hash, the order was undefined, but the same as long as you used the same binary. This had the advantage that successive runs, something we often have in document processing gave consistent results. In today’s Lua we need to do much more sorting of hashes before we loop, especially when we save multi--pass data. It is for this reason that the xml tree is sorted by hash key by default. That way lookups (especially the first of a set) give consistent outcomes.


    Standards

    The rendering of bibliographic entries is often standardized and prescribed by the publisher. If you submit an article to a journal, normally it will be reformatted (or even re- keyed) and the rendering will happen at the publishers end. In that case it may not matter how entries were rendered when writing the publication, because the publisher will do it his or her way. This means that most users probably will stick to the standard apa rules and for them we provide some configuration. Because we use setups it is easy to overload specifics. If you really want to tweak, best look in the files that deal with it.

    Many standards exist and support for other renderings may be added to the core. Interested users are invited to develop and to test alternate standard renderings according to their needs.

    Todo: maybe a list of categories and fields.


    Cleaning up

    Although the bibTEX format is reasonably well defined, in practice there are many ways to organize the data. For instance, one can use predefined string constants that get used (either or not combined with other strings) later on. A string can be enclosed in curly braces or double quotes. The strings can contain TEX commands but these are not standardized. The databases often have somewhat complex ways to deal with special characters and the use of braces in their definition is also not normalized.

    The most complex to deal with are the fields that contain names of people. At some point it might be needed to split a combination of names into individual ones that then get split into title, first name, optional inbetweens, surname(s) and additional: Prof. Dr. Alfred B. C. von Kwik Kwak Jr. II and P. Q. Olet is just one example of this. The convention seems to be not to use commas but and to separate names (often each name will be specified as lastname, firstname).

    We don’t see it as challenge nor as a duty to support all kinds of messy definitions. Of course we try to be somewhat tolerant, but you will be sure to get better results if you use nicely setup, consistent databases.

    Todo: maybe some examples of bad.


    Transition

    In the original bibliography support module usage was as follows (example taken from the contextgarden wiki):

    % engine=pdftex
    \usemodule[bib]
    \usemodule[bibltx]
    \setupbibtex
      [database=xampl]
    \setuppublications
      [numbering=yes]
    \starttext
        As \cite [article-full] already indicated, bibtex is a \LATEX||centric
        program.
        \completepublications
    \stoptext
    

    For MkIV the modules were partly rewritten and ended up in the core so the two commands were no longer needed. The overhead associated with the automatic loading of the bibliography macros can be neglected these days, so standardized modules such as bib are all being moved to the core and do not need to be explicitly loaded.

    The first \setupbibtex command in this example is needed to bootstrap the process: it tells what database has to be processed by bibTEX between runs. The second \setuppublications command is optional. Each citation (tagged with \cite) ends up in the list of publications.

    In the new approach we no longer use bibTEXso we don’t need to setup bibTEX. Instead we define dataset(s). We also no longer set up publications with one command, but have split that up in rendering-, list-, and cite-variants. The basic \cite command remains. The above example becomes:

    \definebtxdataset
      [document]
    \usebtxdataset
      [document]
      [mybibfile.bib]
    \definebtxrendering
      [document]
    \setupbtxrendering
      [document]
      [numbering=yes]
    \starttext
        As \cite [article-full] already indicated, bibtex is a \LATEX||centric
        program.
        \completebtxrendering[document]
    \stoptext
    

    So, we have a few more commands to set up things. If you intend to use just a single dataset and rendering, the above preamble can be simplified to:

    \usebtxdataset
      [mybibfile.bib]
    \setupbtxrendering
      [numbering=yes]
    

    But keep in mind that compared to the old MkII derived method we have moved some of the options to the rendering, list and cite setup variants.

    Another difference is now the use of lists. When you define a rendering, you also define a list. However, all entries are collected in a common list tagged btx. Although you will normally configure a rendering you can still set some properties of lists, but in that case you need to prefix the list identifier. In the case of the above example this is btx:document.


    MLBIBTEX

    Todo: how to plug in MLbibTEX for sorting and other advanced operations.


    Extensions

    As TEX and Lua are both open and accessible in ConTEXt it is possible to extend the functionality of the bibliography related code. For instance, you can add extra loaders.

    function publications.loaders.myformat(dataset,filename)
        local t = { }
        -- Load data from 'filename' and convert it to a Lua table 't' with
        -- the key as hash entry and fields conforming the luadata table
        -- format.
        loaders.lua(dataset,t)
    end
    

    This then permits loading a database (into a dataset) with the command:

    \usebtxdataset[standard][myfile.myformat]
    

    The myformat suffix is recognized automatically. If you want to use another suffix, you can do this:

    \usebtxdataset[standard][myformat::myfile.txt]
    


    Notes

    The move from external bibTEX processing to internal processing has the advantage that we stay within the same run. In the traditional approach we had roughly the following steps:


             
         the first run information is collected and written to file
        
    


             
         after that run the bibTEX program converts that file to another one
        
    


             
         successive runs use that data for references and producing lists
        
    


    In the MkIV approach the bibliographic database is loaded in memory each run and processing also happens each run. On paper this looks less efficient but as Lua is quite fast, in practice performance is much better.

    Probably most demanding is the treatment of authors as we have to analyze names, split multiple authors and reassemble firstnames, vons, surnames and juniors. When we sort by author sorting vectors have to be made which also has a penalty. However, in practice the user will not notice a performance degradation. We did some tests with a list of 500.000 authors, sorted them and typeset them as list (producing some 5400 dense pages in a small font and with small margins). This is typical one of these cases where using LuajitTEX saves quite time. On my machine it took just over 100 seconds to get this done. Unfortunately not all operating systems performed equally well: 32 bit versions worked fine, but 64 bit linux either crashed (stalled) the machine or ran out of memory rather fast, while MacOSX and Windows performed fine. In practice you will never run into this, unless you produce massive amounts of bibliographic entries. LuaJIT has some benefits but also some drawbacks.