John MacFarlane’s excellent command line utility Pandoc is a Haskell program that converts to and from various text markup languages. Pandoc’s help option lists its supported input and output formats.
The following examples are Linux bash shell commands. Windows shell commands are identical.
$ pandoc --help
pandoc [OPTIONS] [FILES]
Input formats: native, json, markdown, markdown+lhs, rst, rst+lhs, docbook,
textile, html, latex, latex+lhs
Output formats: native, json, html, html5, html+lhs, html5+lhs, s5, slidy,
slideous, dzslides, docbook, opendocument, latex, latex+lhs,
beamer, beamer+lhs, context, texinfo, man, markdown,
markdown+lhs, plain, rst, rst+lhs, mediawiki, textile, rtf, org,
asciidoc, odt, docx, epubSome Pandoc conversions are better than others. Pandoc does a better job of turning markdown into LaTeX than LaTeX into markdown. It’s also better at converting HTML into LaTeX than LaTeX into HTML. Pandoc works best when converting markdown, the simplest of its inputs, to other formats. In fact Pandoc does such a good job of converting markdown to HTML, HTML+MathJax, LaTeX or PDF that many writers are now saving their source documents as markdown text knowing they can easily produce other formats as needed.
As handy as Pandoc’s markup conversions are this nifty tool also supports syntax highlighting for over a hundred programming languages. Unfortunately, my favorite language J is not on Pandoc’s list of highlighted languages. [1] Where have I run into this problem before? Luckily for me Pandoc is an open source tool and Pandoc’s author has made it easy to add new highlight languages.
Pandoc is a Haskell program. I’ve been aware of Haskell’s existence for years but until I decided to take on this specialized Pandoc hack I had never studied or used the language. Usually when you set out to modify a large program in an unfamiliar programming language you’re in for what can only be described as an f’ing educational experience. It’s a testament to the quality of the Haskell’s global libraries and standard tools that a complete Haskell novice can effectively tweak large Haskell programs. Here’s what you have to do.
- Install the Haskell Platform. The Haskell Platform is available for all the usual suspects. I’ve used both the Windows and Linux versions. I almost installed the Mac version on my wife’s Mac but resisted the urge.
- Get with the Cabal. Cabal is the main Haskell package distribution and build utility. Cabal comes with the Haskell Platform and is easily accessed from the command line. Type cabal –help in your favorite shell to view the program’s options.
- Spend sometime playing with Hackage. Hackage contains a large set of Haskell packages including all the source code required to build Pandoc.
After installing the Haskell Platform and familiarizing yourself with Cabal try building Pandoc. This will thoroughly exercise your Haskell system. Instructions for building Haskell packages are here. After reading the package build instructions run the following in your command shell:
$ cabal update
$ cabal install pandocThis will download, compile and install a number of Haskell packages. Where Cabal puts the packages depends on your operating system. Cabal saves Linux packages in a hidden local directory. On my machine they ended up in:
/home/john/.cabal/libIf you managed to build Pandoc you’re now ready to add a new highlighting language. Pandoc uses the highlighting-kate package for highlighting. highlighting-kate works by reading a directory of Kate editor xml language regex based definition files and generating custom language parsers. We want to generate a custom J parser so we need to download highlighting-kate source and add a Kate xml definition file for J.
You can find such a J Kate file on the J Wiki here. Download this file by cutting and pasting and save it as j.xml. Now do the following.
- Run the Pandoc version command
pandoc --versionof the Pandoc you just built to determine the version of the highlighting-kate package you need. - Use Cabal to unpack the required highlighting-kate package. This downloads the required package and creates a temporary subdirectory in your current directory that contains package source code.
$ cabal unpack highlighting-kate-0.5.3.2 Unpacking to highlighting-kate-0.5.3.2/ - Move into the temporary subdirectory and copy the Kate j.xml file to the package’s xml subdirectory.
$ cd highlighting-kate-0.5.3.2 $ cp ~/pd/blog/j.xml ~/temp/highlighting-kate-0.5.3.2/xml/j.xml - Configure the package.
$ cabal configure Resolving dependencies... Configuring highlighting-kate-0.5.3.2... - Build the highlighting-kate package.
$ cabal build Resolving dependencies... ... (omitted) ... - If highlighting-kate builds without problems run the command.
$ runhaskell ParseSyntaxFiles.hs xml Writing Text/Highlighting/Kate/Syntax/SqlPostgresql.hs Writing Text/Highlighting/Kate/Syntax/Scala.hs ... (omitted) ...ParseSyntaxFiles scans the package’s xml subdirectory and generates language specific parsers. If all goes well you will find J.hs in this directory.
~/temp/highlighting-kate-0.5.3.2/Text/Highlighting/Kate/SyntaxJ.hs, like all the files referred to in this post, are available on GitHub in this directory.
- Now rebuild the highlighting-kate package. This compiles your new J.hs parser file.
$ cabal build Resolving dependencies... ... (omitted) ... - After rebuilding the package run the Cabal copy command to put the modified package in the expected library location.
$ cabal copy Installing library in /home/john/.cabal/lib/highlighting-kate-0.5.3.2/ghc-7.4.1
Now that the highlighting library is up to date we have to rebuild Pandoc. To do this mirror the steps taken to download and build the highlighting package.
- Use Cabal to unpack the Pandoc package.
$ cd ~/temp $ cabal unpack pandoc-1.9.4.2 Unpacking to pandoc-1.9.4.2/ - Switch to the Pandoc subdirectory and configure the package.
$ cabal configure Resolving dependencies... [1 of 1] Compiling Main ( Setup.hs, dist/setup/Main.o ) ... (omitted) ... - Rebuild Pandoc.
$ cabal build Building pandoc-1.9.4.2... Preprocessing executable 'pandoc' for pandoc-1.9.4.2... ... (omitted) ...If all goes well a Pandoc executable will be written to this subdirectory.
~/temp/pandoc-1.9.4.2/dist/build/pandoc - You can check the new executable by running
pandoc --version. The result should display J in the list of supported languages.
Now that we have a Pandoc that can highlight J we’re almost ready to blog gaudy J code. However before doing this we need to install some custom CSS. Custom CSS is not available on free WordPress.com blogs. To apply custom coloring schemes get the custom package and learn how to use WordPress’s custom CSS editor. As daunting as this sounds it’s no problemo for my limited purposes. To enable tango style Pandoc syntax highlighting on your WordPress blog paste tango.css into the custom CSS editor, check the “Add my CSS to CSS stylesheet” button and then press the “Save Stylesheet” button. Now your WordPress blog will be sensitive to the HTML span tags generated by Pandoc.
To show that all this hacking works as intended you can check out the Pandoc generated versions of this blog post. I’ve posted the original markdown source with PDF, LaTeX and HTML versions. All these files are available via the files sidebar. You can generate the HTML version with the command:
$ pandoc -s --highlight-style=tango PJHighlight.markdown -o PJHighlight.htmlTo get other versions simply change the file extension of the output -o file.
Click for “Haven Age” Walkthrough
Finally we are ready to post syntax highlighted J code. The following J verb bonebridge generates all “likely” lock combinations for the MYST IV Bonebridge puzzle in Pandoc’s tango style. At one time I was a big fan of MYST computer games. I always enjoyed being lost in a beautiful puzzle which, if you discard the beautiful bit, is a pretty accurate description of my programmer day job.
[1] J has its own syntax highlighting tools but they are not part of a document generation system. Pandoc’s highlighters elegantly feed into many output formats making them far more useful.
Analyze the Data not the Drivel 
My family members all the time say that I am wasting my time
here at web, however I know I am getting experience all the time by reading thes nice articles.
Great article. I am experiencing a few of these issues as
well..