Wikitext

= Wikitext

The Wikitext extension is a fast wikitext-to-HTML translator written in C and packaged as a Ruby extension.

Usage is straightforward:

!!!plain
$ irb -r wikitext
>> Wikitext::Parser.new.parse("hello world!")
=> "<p>hello world!</p>\n"

= Design goals

I needed a wikitext-to-HTML translator for a Rails application; a number of design goals flowed on from this:

• fast: Rails has a reputation for being slow, so the translator had to be part of the solution, not part of the problem
• efficient: again, given how much memory Rails likes to use, the translator had to be very memory-efficient
• robust: on a public-facing web application that had to be up for long periods, the translator had to be stable (no crashes, no resource leaks)
• secure: again, accepting input from untrusted sources meant that the translator had to sanitize or reject unsafe input
• <em>easy to use</em>: for end users, the translator should provide a simple, familiar markup as close as possible to what they already know from other applications (such as MediaWiki, the wiki software that powers Wikipedia)
• forgiving: wikitext is presentation markup, not source code, so the translator should do a reasonable job of formatting even the most invalid markup rather than giving up
• informative: when provided invalid markup the translator should fail gracefully and emit HTML that provides useful visual feedback about where the errors are in the input
• <em>multilingual-friendly</em>: the translator should handle input beyond printable ASCII in a compatible fashion
• attractive: the emitted HTML source should be consistent and attractive
• <em>valid output</em>: regardless of the input, the translator should always produce valid HTML5 output
• <em>well-tested</em>: the translator should have a comprehensive test suite to ensure that its behaviour is not only correct but also stable over time
• <em>cross-platform</em>: should work identically on Mac OS X, Linux (explicitly tested platforms) and perhaps others as well

Some notable things that were not design goals:

• implement all of the MediaWiki syntax (tables etc)

= Markup

The markup is very close to that used by MediaWiki, the most popular wiki software and the one that powers Wikipedia.

== Headings

!!!wikitext
= Heading 1 =
== Heading 2 ==
=== Heading 3 ===
==== Heading 4 ====
===== Heading 5 =====
====== Heading 6 ======

Are marked up as:

!!!html
<h1>Heading 1</h1>
<h2>Heading 2</h2>
<h3>Heading 3</h3>
<h4>Heading 4</h4>
<h5>Heading 5</h5>
<h6>Heading 6</h6>

== Paragraphs

Consecutive linebreaks are converted into paragraph breaks.

!!!wikitext
This is one paragraph.
Another line.

And this is another.

Would be marked up as:

!!!html
<p>This is one paragraph. Another line.</p>
<p>And this is another.</p>

== Emphasis, Strong

Emphasis is marked up as follows:

!!!wikitext
''emphasized''

Which gets translated into:

!!!html
<em>emphasized</em>

Strong is marked up like this:

!!!wikitext
'''strong text'''

And transformed into:

!!!html
<strong>strong text</strong>

You can nest spans inside one another, provided you don't try to produce invalid HTML (for example, nesting strong inside strong). Here is a valid example:

!!!wikitext
'''''foo'' bar''' baz

This would become:

!!!html
<strong><em>foo</em> bar</strong> baz

Note that the translator emits HTML on the fly, so when it sees the first run of five apostrophes it has no way of knowing what will come afterwards and so doesn't know whether you mean to say "strong em" or "em strong"; it therefore always assumes "strong em". If you wish to force the alternative interpretation you can do one of the following:

!!!wikitext
'' '''foo''' bar'' baz (ie. use whitespace)
''<nowiki></nowiki>'''foo''' bar'' baz (ie. insert an empty nowiki span)
<em><strong>foo</strong> bar</em> baz (ie. use explicit HTML tags instead)
<em>'''foo''' bar</em> baz (ie. use explicit HTML tags instead)

Note that to avoid ambiguity, the translator will not let you intermix the shorthand style with the literal HTML tag style.

!!!wikitext
<em>foo'' (ie. intermixed, invalid)

== Teletype

The translator recognizes both standard HTML +tt+ tags and the backtick (`) as a shorthand. These two are equivalent:

!!!wikitext
<tt>fixed</tt>
`fixed`

As of version 2.0, this markup is actually translated to +code+ tags in the output because the +tt+ tag was removed from the HTML5 standard.

If you need to insert a literal backtick in your text you use a +nowiki+ span:

!!!wikitext
here follows a literal <nowiki>`</nowiki> backtick

To avoid ambiguity, the translator will not let you intermix the two styles.

== +nowiki+ spans

Already mentioned above, you can use +nowiki+ tags to temporarily disable wikitext markup. As soon as the translator sees the opening +nowiki+ tag it starts emitting a literal copy of everything it sees up until the closing +nowiki+ tag:

!!!wikitext
Hello <nowiki>''world''</nowiki>

Would be emitted as:

!!!html
Hello ''world''

== Blockquotes

!!!wikitext
> Hello world!
> Bye for now.

Would be emitted as:

!!!html
<blockquote><p>Hello world! Bye for now.</p></blockquote>

You can nest blockquotes or any other kind of block or span inside blockquotes. For example:

!!!wikitext
> first quote
>> quote inside a quote

== Preformatted text

Any line indented with whitespace will be interpreted as part of a +pre+ block. Wikitext markup inside +pre+ blocks has no special meaning. For example, consider the following block indented by a single space:

!!!wikitext
 // source code listing
 void foo(void)
 {
     x++;
 }

Would be translated into:

!!!html
<pre>// source code listing
void foo(void)
{
    x++;
}</pre>

+pre+ blocks may be nested inside +blockquote+ blocks.

== Internal links

!!!wikitext
[[article title]]

Would become:

!!!html
<a href="/wiki/article_title">article title</a>

And:

!!!wikitext
[[title|link text]]

Would become:

!!!html
<a href="/wiki/article">link text</a>

See the Wikitext::Parser attributes documentation for how you can override the default link prefix (<em>/wiki/</em> as shown in the example), and how "red links" can be implemented by applying custom CSS depending on the link target (this can be used to make links to non-existent pages appear in a different color).

== Alternative blockquote and preformatted block syntax

For +blockquote+ and +pre+ blocks that go on for many lines it may be more convenient to use the alternative syntax which uses standard HTML tags rather than special prefixes at the beginning of each line.

!!!wikitext
<blockquote>This is
a blockquote!</blockquote>

<pre>And this is
preformatted text</pre>

+blockquote+ and +pre+ blocks may nest inside other +blockquote+ blocks.

Note that to avoid ambiguity, the translator will not let you intermix the two styles (HTML markup and wikitext markup).

+pre+ blocks may also contain a custom +lang+ attribute for the purposes of marking up a block for syntax-highlighting (note that the highlighting itself would be provided by JavaScript in the browser and is not actually part of the Wikitext extension). For example:

!!!wikitext
<pre lang="ruby">puts @person.name</pre>

Would be translated into:

!!!html
<pre class="ruby-syntax">puts @person.name</pre>

The +lang+ attribute may only contain letters, so "Objective-C", for example would need to be written as "objc" or similar.

== External links

!!!wikitext
[http://example.com/ this site]

Would become:

!!!html
<a href="http://example.com/" class="external">this site</a>

See the {Wikitext::Parser} attributes documentation for information on overriding the default external link class (+external+ in this example), or including a +rel+ attribute of "nofollow" (which may be useful for search-engine optimization).

Note that in addition to providing a fully-qualified URL including a protocol (such as "http://" or "ftp://") you also have the option of using an unqualified "path"-style URL. This is useful for making links to other pages still on the same site, but outside of the wiki:

!!!wikitext
[/issues/1024 ticket #1024]

Would become:

!!!html
<a href="/issues/1024">ticket #1024</a>

Note that no "external" class is included in the generated link.

To avoid false positives, what constitutes a "path" is narrowly-defined as a string that begins with a slash, optionally followed by zero or more "path components" consisting of upper and lowercase letters, numbers, underscores, hyphens or periods. Path components are separated by a slash, and the trailing slash after the last path component is optional.

== Images

!!!wikitext
{{foo.png}}

When outputting using HTML syntax (the default), this would become:

!!!html
<img src="/images/foo.png" alt="foo.png">

When outputting using XML syntax, this would become a self-closing tag:

!!!html
<img src="/images/foo.png" alt="foo.png" />

See the Wikitext::Parser documentation for information on setting the output syntax.

You can override the "/images/" prefix using the +img_prefix+ attribute of the Parser.

You can also specify "absolute" image "src" attributes regardless of the current prefix setting by starting the image path with a forward slash; that is:

!!!wikitext
{{/foo.png}}

Would become:

!!!html
<img src="/foo.png" alt="/foo.png">

== Lists

Lists come in both unordered ("ul"):

!!!wikitext
* item
* item
* item

And ordered ("ol") forms:

!!!wikitext
# first
# second
# third

These would produce, respectively:

!!!html
<ul>
  <li