You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

README.md 21KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321322323324325326327328329330331332333334335336337338339340341342343344345346347348349350351352353354355356357358359360361362363364365366367368369370371372373374375376377378379380381382383384385386387388389390391392393394395396397398399400401402403404405406407408409410411412413414415416417418419420421422423424425426427428429430431432433434435436437438439440441442443444445446447448449450451452453454455456457458459460461462463464465466467468469470471472473474475476477478479480481482483484485486487488489490491492493494495496497498499500501502503504505506507508509510511512513
  1. A super fast, highly extensible markdown parser for PHP
  2. =======================================================
  3. [![Latest Stable Version](https://poser.pugx.org/cebe/markdown/v/stable.png)](https://packagist.org/packages/cebe/markdown)
  4. [![Total Downloads](https://poser.pugx.org/cebe/markdown/downloads.png)](https://packagist.org/packages/cebe/markdown)
  5. [![Build Status](https://travis-ci.org/cebe/markdown.svg?branch=master)](http://travis-ci.org/cebe/markdown)
  6. [![Tested against HHVM](http://hhvm.h4cc.de/badge/cebe/markdown.png)](http://hhvm.h4cc.de/package/cebe/markdown)
  7. [![Code Coverage](https://scrutinizer-ci.com/g/cebe/markdown/badges/coverage.png?s=db6af342d55bea649307ef311fbd536abb9bab76)](https://scrutinizer-ci.com/g/cebe/markdown/)
  8. [![Scrutinizer Quality Score](https://scrutinizer-ci.com/g/cebe/markdown/badges/quality-score.png?s=17448ca4d140429fd687c58ff747baeb6568d528)](https://scrutinizer-ci.com/g/cebe/markdown/)
  9. What is this? <a name="what"></a>
  10. -------------
  11. A set of [PHP][] classes, each representing a [Markdown][] flavor, and a command line tool
  12. for converting markdown files to HTML files.
  13. The implementation focus is to be **fast** (see [benchmark][]) and **extensible**.
  14. Parsing Markdown to HTML is as simple as calling a single method (see [Usage](#usage)) providing a solid implementation
  15. that gives most expected results even in non-trivial edge cases.
  16. Extending the Markdown language with new elements is as simple as adding a new method to the class that converts the
  17. markdown text to the expected output in HTML. This is possible without dealing with complex and error prone regular expressions.
  18. It is also possible to hook into the markdown structure and add elements or read meta information using the internal representation
  19. of the Markdown text as an abstract syntax tree (see [Extending the language](#extend)).
  20. Currently the following markdown flavors are supported:
  21. - **Traditional Markdown** according to <http://daringfireball.net/projects/markdown/syntax> ([try it!](http://markdown.cebe.cc/try?flavor=default)).
  22. - **Github flavored Markdown** according to <https://help.github.com/articles/github-flavored-markdown> ([try it!](http://markdown.cebe.cc/try?flavor=gfm)).
  23. - **Markdown Extra** according to <http://michelf.ca/projects/php-markdown/extra/> (currently not fully supported WIP see [#25][], [try it!](http://markdown.cebe.cc/try?flavor=extra))
  24. - Any mixed Markdown flavor you like because of its highly extensible structure (See documentation below).
  25. Future plans are to support:
  26. - Smarty Pants <http://daringfireball.net/projects/smartypants/>
  27. - ... (Feel free to [suggest](https://github.com/cebe/markdown/issues/new) further additions!)
  28. [PHP]: http://php.net/ "PHP is a popular general-purpose scripting language that is especially suited to web development."
  29. [Markdown]: http://en.wikipedia.org/wiki/Markdown "Markdown on Wikipedia"
  30. [#25]: https://github.com/cebe/markdown/issues/25 "issue #25"
  31. [benchmark]: https://github.com/kzykhys/Markbench#readme "kzykhys/Markbench on github"
  32. ### Who is using it?
  33. - It powers the [API-docs and the definitive guide](http://www.yiiframework.com/doc-2.0/) for the [Yii Framework][] [2.0](https://github.com/yiisoft/yii2).
  34. [Yii Framework]: http://www.yiiframework.com/ "The Yii PHP Framework"
  35. Installation <a name="installation"></a>
  36. ------------
  37. [PHP 5.4 or higher](http://www.php.net/downloads.php) is required to use it.
  38. It will also run on facebook's [hhvm](http://hhvm.com/).
  39. Installation is recommended to be done via [composer][] by running:
  40. composer require cebe/markdown "~1.0.1"
  41. Alternatively you can add the following to the `require` section in your `composer.json` manually:
  42. ```json
  43. "cebe/markdown": "~1.0.1"
  44. ```
  45. Run `composer update` afterwards.
  46. [composer]: https://getcomposer.org/ "The PHP package manager"
  47. Usage <a name="usage"></a>
  48. -----
  49. ### In your PHP project
  50. To parse your markdown you need only two lines of code. The first one is to choose the markdown flavor as
  51. one of the following:
  52. - Traditional Markdown: `$parser = new \cebe\markdown\Markdown();`
  53. - Github Flavored Markdown: `$parser = new \cebe\markdown\GithubMarkdown();`
  54. - Markdown Extra: `$parser = new \cebe\markdown\MarkdownExtra();`
  55. The next step is to call the `parse()`-method for parsing the text using the full markdown language
  56. or calling the `parseParagraph()`-method to parse only inline elements.
  57. Here are some examples:
  58. ```php
  59. // traditional markdown and parse full text
  60. $parser = new \cebe\markdown\Markdown();
  61. echo $parser->parse($markdown);
  62. // use github markdown
  63. $parser = new \cebe\markdown\GithubMarkdown();
  64. echo $parser->parse($markdown);
  65. // use markdown extra
  66. $parser = new \cebe\markdown\MarkdownExtra();
  67. echo $parser->parse($markdown);
  68. // parse only inline elements (useful for one-line descriptions)
  69. $parser = new \cebe\markdown\GithubMarkdown();
  70. echo $parser->parseParagraph($markdown);
  71. ```
  72. You may optionally set one of the following options on the parser object:
  73. For all Markdown Flavors:
  74. - `$parser->html5 = true` to enable HTML5 output instead of HTML4.
  75. - `$parser->keepListStartNumber = true` to enable keeping the numbers of ordered lists as specified in the markdown.
  76. The default behavior is to always start from 1 and increment by one regardless of the number in markdown.
  77. For GithubMarkdown:
  78. - `$parser->enableNewlines = true` to convert all newlines to `<br/>`-tags. By default only newlines with two preceding spaces are converted to `<br/>`-tags.
  79. It is recommended to use UTF-8 encoding for the input strings. Other encodings are currently not tested.
  80. ### The command line script
  81. You can use it to render this readme:
  82. bin/markdown README.md > README.html
  83. Using github flavored markdown:
  84. bin/markdown --flavor=gfm README.md > README.html
  85. or convert the original markdown description to html using the unix pipe:
  86. curl http://daringfireball.net/projects/markdown/syntax.text | bin/markdown > md.html
  87. Here is the full Help output you will see when running `bin/markdown --help`:
  88. PHP Markdown to HTML converter
  89. ------------------------------
  90. by Carsten Brandt <mail@cebe.cc>
  91. Usage:
  92. bin/markdown [--flavor=<flavor>] [--full] [file.md]
  93. --flavor specifies the markdown flavor to use. If omitted the original markdown by John Gruber [1] will be used.
  94. Available flavors:
  95. gfm - Github flavored markdown [2]
  96. extra - Markdown Extra [3]
  97. --full ouput a full HTML page with head and body. If not given, only the parsed markdown will be output.
  98. --help shows this usage information.
  99. If no file is specified input will be read from STDIN.
  100. Examples:
  101. Render a file with original markdown:
  102. bin/markdown README.md > README.html
  103. Render a file using gihtub flavored markdown:
  104. bin/markdown --flavor=gfm README.md > README.html
  105. Convert the original markdown description to html using STDIN:
  106. curl http://daringfireball.net/projects/markdown/syntax.text | bin/markdown > md.html
  107. [1] http://daringfireball.net/projects/markdown/syntax
  108. [2] https://help.github.com/articles/github-flavored-markdown
  109. [3] http://michelf.ca/projects/php-markdown/extra/
  110. Extensions
  111. ----------
  112. Here are some extensions to this library:
  113. - [Bogardo/markdown-codepen](https://github.com/Bogardo/markdown-codepen) - shortcode to embed codepens from http://codepen.io/ in markdown.
  114. - [kartik-v/yii2-markdown](https://github.com/kartik-v/yii2-markdown) - Advanced Markdown editing and conversion utilities for Yii Framework 2.0.
  115. - [cebe/markdown-latex](https://github.com/cebe/markdown-latex) - Convert Markdown to LaTeX and PDF
  116. - [softark/creole](https://github.com/softark/creole) - A creole markup parser
  117. - ... [add yours!](https://github.com/cebe/markdown/edit/master/README.md#L98)
  118. Extending the language <a name="extend"></a>
  119. ----------------------
  120. Markdown consists of two types of language elements, I'll call them block and inline elements simlar to what you have in
  121. HTML with `<div>` and `<span>`. Block elements are normally spreads over several lines and are separated by blank lines.
  122. The most basic block element is a paragraph (`<p>`).
  123. Inline elements are elements that are added inside of block elements i.e. inside of text.
  124. This markdown parser allows you to extend the markdown language by changing existing elements behavior and also adding
  125. new block and inline elements. You do this by extending from the parser class and adding/overriding class methods and
  126. properties. For the different element types there are different ways to extend them as you will see in the following sections.
  127. ### Adding block elements
  128. The markdown is parsed line by line to identify each non-empty line as one of the block element types.
  129. To identify a line as the beginning of a block element it calls all protected class methods who's name begins with `identify`.
  130. An identify function returns true if it has identified the block element it is responsible for or false if not.
  131. In the following example we will implement support for [fenced code blocks][] which are part of the github flavored markdown.
  132. [fenced code blocks]: https://help.github.com/articles/github-flavored-markdown#fenced-code-blocks
  133. "Fenced code block feature of github flavored markdown"
  134. ```php
  135. <?php
  136. class MyMarkdown extends \cebe\markdown\Markdown
  137. {
  138. protected function identifyLine($line, $lines, $current)
  139. {
  140. // if a line starts with at least 3 backticks it is identified as a fenced code block
  141. if (strncmp($line, '```', 3) === 0) {
  142. return 'fencedCode';
  143. }
  144. return parent::identifyLine($lines, $current);
  145. }
  146. // ...
  147. }
  148. ```
  149. In the above, `$line` is a string containing the content of the current line and is equal to `$lines[$current]`.
  150. You may use `$lines` and `$current` to check other lines than the current line. In most cases you can ignore these parameters.
  151. Parsing of a block element is done in two steps:
  152. 1. "consuming" all the lines belonging to it. In most cases this is iterating over the lines starting from the identified
  153. line until a blank line occurs. This step is implemented by a method named `consume{blockName}()` where `{blockName}`
  154. is the same name as used for the identify function above. The consume method also takes the lines array
  155. and the number of the current line. It will return two arguments: an array representing the block element in the abstract syntax tree
  156. of the markdown document and the line number to parse next. In the abstract syntax array the first element refers to the name of
  157. the element, all other array elements can be freely defined by yourself.
  158. In our example we will implement it like this:
  159. ```php
  160. protected function consumeFencedCode($lines, $current)
  161. {
  162. // create block array
  163. $block = [
  164. 'fencedCode',
  165. 'content' => [],
  166. ];
  167. $line = rtrim($lines[$current]);
  168. // detect language and fence length (can be more than 3 backticks)
  169. $fence = substr($line, 0, $pos = strrpos($line, '`') + 1);
  170. $language = substr($line, $pos);
  171. if (!empty($language)) {
  172. $block['language'] = $language;
  173. }
  174. // consume all lines until ```
  175. for($i = $current + 1, $count = count($lines); $i < $count; $i++) {
  176. if (rtrim($line = $lines[$i]) !== $fence) {
  177. $block['content'][] = $line;
  178. } else {
  179. // stop consuming when code block is over
  180. break;
  181. }
  182. }
  183. return [$block, $i];
  184. }
  185. ```
  186. 2. "rendering" the element. After all blocks have been consumed, they are being rendered using the
  187. `render{elementName}()`-method where `elementName` refers to the name of the element in the abstract syntax tree:
  188. ```php
  189. protected function renderFencedCode($block)
  190. {
  191. $class = isset($block['language']) ? ' class="language-' . $block['language'] . '"' : '';
  192. return "<pre><code$class>" . htmlspecialchars(implode("\n", $block['content']) . "\n", ENT_NOQUOTES, 'UTF-8') . '</code></pre>';
  193. }
  194. ```
  195. You may also add code highlighting here. In general it would also be possible to render ouput in a different language than
  196. HTML for example LaTeX.
  197. ### Adding inline elements
  198. Adding inline elements is different from block elements as they are parsed using markers in the text.
  199. An inline element is identified by a marker that marks the beginning of an inline element (e.g. `[` will mark a possible
  200. beginning of a link or `` ` `` will mark inline code).
  201. Parsing methods for inline elements are also protected and identified by the prefix `parse`. Additionally a `@marker` annotation
  202. in PHPDoc is needed to register the parse function for one or multiple markers.
  203. The method will then be called when a marker is found in the text. As an argument it takes the text starting at the position of the marker.
  204. The parser method will return an array containing the element of the abstract sytnax tree and an offset of text it has
  205. parsed from the input markdown. All text up to this offset will be removed from the markdown before the next marker will be searched.
  206. As an example, we will add support for the [strikethrough][] feature of github flavored markdown:
  207. [strikethrough]: https://help.github.com/articles/github-flavored-markdown#strikethrough "Strikethrough feature of github flavored markdown"
  208. ```php
  209. <?php
  210. class MyMarkdown extends \cebe\markdown\Markdown
  211. {
  212. /**
  213. * @marker ~~
  214. */
  215. protected function parseStrike($markdown)
  216. {
  217. // check whether the marker really represents a strikethrough (i.e. there is a closing ~~)
  218. if (preg_match('/^~~(.+?)~~/', $markdown, $matches)) {
  219. return [
  220. // return the parsed tag as an element of the abstract syntax tree and call `parseInline()` to allow
  221. // other inline markdown elements inside this tag
  222. ['strike', $this->parseInline($matches[1])],
  223. // return the offset of the parsed text
  224. strlen($matches[0])
  225. ];
  226. }
  227. // in case we did not find a closing ~~ we just return the marker and skip 2 characters
  228. return [['text', '~~'], 2];
  229. }
  230. // rendering is the same as for block elements, we turn the abstract syntax array into a string.
  231. protected function renderStrike($element)
  232. {
  233. return '<del>' . $this->renderAbsy($element[1]) . '</del>';
  234. }
  235. }
  236. ```
  237. ### Composing your own Markdown flavor
  238. This markdown library is composed of traits so it is very easy to create your own markdown flavor by adding and/or removing
  239. the single feature traits.
  240. Designing your Markdown flavor consists of four steps:
  241. 1. Select a base class
  242. 2. Select language feature traits
  243. 3. Define escapeable characters
  244. 4. Optionally add custom rendering behavior
  245. #### Select a base class
  246. If you want to extend from a flavor and only add features you can use one of the existing classes
  247. (`Markdown`, `GithubMarkdown` or `MarkdownExtra`) as your flavors base class.
  248. If you want to define a subset of the markdown language, i.e. remove some of the features, you have to
  249. extend your class from `Parser`.
  250. #### Select language feature traits
  251. The following shows the trait selection for traditional Markdown.
  252. ```php
  253. class MyMarkdown extends Parser
  254. {
  255. // include block element parsing using traits
  256. use block\CodeTrait;
  257. use block\HeadlineTrait;
  258. use block\HtmlTrait {
  259. parseInlineHtml as private;
  260. }
  261. use block\ListTrait {
  262. // Check Ul List before headline
  263. identifyUl as protected identifyBUl;
  264. consumeUl as protected consumeBUl;
  265. }
  266. use block\QuoteTrait;
  267. use block\RuleTrait {
  268. // Check Hr before checking lists
  269. identifyHr as protected identifyAHr;
  270. consumeHr as protected consumeAHr;
  271. }
  272. // include inline element parsing using traits
  273. use inline\CodeTrait;
  274. use inline\EmphStrongTrait;
  275. use inline\LinkTrait;
  276. /**
  277. * @var boolean whether to format markup according to HTML5 spec.
  278. * Defaults to `false` which means that markup is formatted as HTML4.
  279. */
  280. public $html5 = false;
  281. protected function prepare()
  282. {
  283. // reset references
  284. $this->references = [];
  285. }
  286. // ...
  287. }
  288. ```
  289. In general, just adding the trait with `use` is enough, however in some cases some fine tuning is desired
  290. to get most expected parsing results. Elements are detected in alphabetical order of their identification
  291. function. This means that if a line starting with `-` could be a list or a horizontal rule, the preference has to be set
  292. by renaming the identification function. This is what is done with renaming `identifyHr` to `identifyAHr`
  293. and `identifyBUl` to `identifyBUl`. The consume function always has to have the same name as the identification function
  294. so this has to be renamed too.
  295. There is also a conflict for parsing of the `<` character. This could either be a link/email enclosed in `<` and `>`
  296. or an inline HTML tag. In order to resolve this conflict when adding the `LinkTrait`, we need to hide the `parseInlineHtml`
  297. method of the `HtmlTrait`.
  298. If you use any trait that uses the `$html5` property to adjust its output you also need to define this property.
  299. If you use the link trait it may be useful to implement `prepare()` as shown above to reset references before
  300. parsing to ensure you get a reusable object.
  301. #### Define escapeable characters
  302. Depenedend on the language features you have chosen there is a different set of characters that can be escaped
  303. using `\`. The following is the set of escapeable characters for traditional markdown, you can copy it to your class
  304. as is.
  305. ```php
  306. /**
  307. * @var array these are "escapeable" characters. When using one of these prefixed with a
  308. * backslash, the character will be outputted without the backslash and is not interpreted
  309. * as markdown.
  310. */
  311. protected $escapeCharacters = [
  312. '\\', // backslash
  313. '`', // backtick
  314. '*', // asterisk
  315. '_', // underscore
  316. '{', '}', // curly braces
  317. '[', ']', // square brackets
  318. '(', ')', // parentheses
  319. '#', // hash mark
  320. '+', // plus sign
  321. '-', // minus sign (hyphen)
  322. '.', // dot
  323. '!', // exclamation mark
  324. '<', '>',
  325. ];
  326. ```
  327. #### Add custom rendering behavior
  328. Optionally you may also want to adjust rendering behavior by overriding some methods.
  329. You may refer to the `consumeParagraph()` method of the `Markdown` and `GithubMarkdown` classes for some inspiration
  330. which define different rules for which elements are allowed to interrupt a paragraph.
  331. Acknowledgements <a name="ack"></a>
  332. ----------------
  333. I'd like to thank [@erusev][] for creating [Parsedown][] which heavily influenced this work and provided
  334. the idea of the line based parsing approach.
  335. [@erusev]: https://github.com/erusev "Emanuil Rusev"
  336. [Parsedown]: http://parsedown.org/ "The Parsedown PHP Markdown parser"
  337. FAQ <a name="faq"></a>
  338. ---
  339. ### Why another markdown parser?
  340. While reviewing PHP markdown parsers for choosing one to use bundled with the [Yii framework 2.0][]
  341. I found that most of the implementations use regex to replace patterns instead
  342. of doing real parsing. This way extending them with new language elements is quite hard
  343. as you have to come up with a complex regex, that matches your addition but does not mess
  344. with other elements. Such additions are very common as you see on github which supports referencing
  345. issues, users and commits in the comments.
  346. A [real parser][] should use context aware methods that walk trough the text and
  347. parse the tokens as they find them. The only implentation that I have found that uses
  348. this approach is [Parsedown][] which also shows that this implementation is [much faster][benchmark]
  349. than the regex way. Parsedown however is an implementation that focuses on speed and implements
  350. its own flavor (mainly github flavored markdown) in one class and at the time of this writing was
  351. not easily extensible.
  352. Given the situation above I decided to start my own implementation using the parsing approach
  353. from Parsedown and making it extensible creating a class for each markdown flavor that extend each
  354. other in the way that also the markdown languages extend each other.
  355. This allows you to choose between markdown language flavors and also provides a way to compose your
  356. own flavor picking the best things from all.
  357. I chose this approach as it is easier to implement and also more intuitive approach compared
  358. to using callbacks to inject functionallity into the parser.
  359. [real parser]: http://en.wikipedia.org/wiki/Parsing#Types_of_parser
  360. [Parsedown]: http://parsedown.org/ "The Parsedown PHP Markdown parser"
  361. ### Where do I report bugs or rendering issues?
  362. Just [open an issue][] on github, post your markdown code and describe the problem. You may also attach screenshots of the rendered HTML result to describe your problem.
  363. [open an issue]: https://github.com/cebe/markdown/issues/new
  364. ### How can I contribute to this library?
  365. Check the [CONTRIBUTING.md](CONTRIBUTING.md) file for more info.
  366. ### Am I free to use this?
  367. This library is open source and licensed under the [MIT License][]. This means that you can do whatever you want
  368. with it as long as you mention my name and include the [license file][license]. Check the [license][] for details.
  369. [MIT License]: http://opensource.org/licenses/MIT
  370. [license]: https://github.com/cebe/markdown/blob/master/LICENSE
  371. Contact
  372. -------
  373. Feel free to contact me using [email](mailto:mail@cebe.cc) or [twitter](https://twitter.com/cebe_cc).