Large update: Scanners for HTML, RHTML and Nitro-XHTML added.

CSS style changes/enhancments (mainly the new background color for inline code, affects all Ruby code.)
Demos and tests adjusted.

Plugin: new PluginHost::default method.

Scanner:
- New setup method
- ability to re-use a scanner
- ability to keep the tokens
- minor changes to token caching and string flattening

Encoder: Error if token content is neither String nor Symbol.

HTML encoder:
- more warnings for unclosed tokens
- output now UTF-8

Ruby Scanner:
- bug: symbols before => now do not include =; {:foo=>bar} is valid Ruby code
- try to close all open tokens
- constants now all with specific namespace (for speed, I hope)

Styles: new :entity/en class.

Test suite now gives hinted HTML output.

1 files changed, 87 insertions, 81 deletions

diff --git a/demo/demo_css.out b/demo/demo_css.out
index 1429a8f..d1c6259 100644
--- a/demo/demo_css.out
+++ b/demo/demo_css.out
@@ -4,95 +4,101 @@
 	font-family: 'Courier New', 'Terminal', monospace;

 	color: #100;

 }

-.CodeRay pre { margin: 0px; }

+.CodeRay pre { margin: 0px }

 

 div.CodeRay { }

 

-span.CodeRay { white-space: pre; border: 0px; padding: 2px; }

+span.CodeRay { white-space: pre; border: 0px; padding: 2px }

 

-table.CodeRay { border-collapse: collapse; width: 100%; padding: 2px; }

-table.CodeRay td { padding: 2px 4px; vertical-align: top; }

+table.CodeRay { border-collapse: collapse; width: 100%; padding: 2px }

+table.CodeRay td { padding: 2px 4px; vertical-align: top }

 

 .CodeRay .line_numbers, .CodeRay .no {

 	background-color: #def;

 	color: gray;

 	text-align: right;

 }

-.CodeRay .line_numbers tt { font-weight: bold; }

-.CodeRay .no { padding: 0px 4px; }

-.CodeRay .code { width: 100%; }

-

-ol.CodeRay { font-size: 10pt; }

-ol.CodeRay li { white-space: pre; }

-

-.CodeRay .code pre { overflow: auto; }

-

-.CodeRay .af { color:#00C; }

-.CodeRay .an { color:#007; }

-.CodeRay .av { color:#700; }

-.CodeRay .aw { color:#C00; }

-.CodeRay .bi { color:#509; font-weight:bold; }

-.CodeRay .c  { color:#888; }

-

-.CodeRay .ch { color:#04D; }

-.CodeRay .ch .k { color:#04D; }

-.CodeRay .ch .dl { color:#039; }

-

-.CodeRay .cl { color:#B06; font-weight:bold; }

-.CodeRay .co { color:#036; font-weight:bold; }

-.CodeRay .cr { color:#0A0; }

-.CodeRay .cv { color:#369; }

-.CodeRay .df { color:#099; font-weight:bold; }

-.CodeRay .di { color:#088; font-weight:bold; }

-.CodeRay .dl { color:black; }

-.CodeRay .do { color:#970; }

-.CodeRay .ds { color:#D42; font-weight:bold; }

-.CodeRay .e  { color:#666; font-weight:bold; }

-.CodeRay .er { color:#F00; background-color:#FAA; }

-.CodeRay .ex { color:#F00; font-weight:bold; }

-.CodeRay .fl { color:#60E; font-weight:bold; }

-.CodeRay .fu { color:#06B; font-weight:bold; }

-.CodeRay .gv { color:#d70; font-weight:bold; }

-.CodeRay .hx { color:#058; font-weight:bold; }

-.CodeRay .i  { color:#00D; font-weight:bold; }

-.CodeRay .ic { color:#B44; font-weight:bold; }

-.CodeRay .il { }

-.CodeRay .in { color:#B2B; font-weight:bold; }

-.CodeRay .iv { color:#33B; }

-.CodeRay .la { color:#970; font-weight:bold; }

-.CodeRay .lv { color:#963; }

-.CodeRay .oc { color:#40E; font-weight:bold; }

-.CodeRay .on { color:#000; font-weight:bold; }

+.CodeRay .line_numbers tt { font-weight: bold }

+.CodeRay .no { padding: 0px 4px }

+.CodeRay .code { width: 100% }

+

+ol.CodeRay { font-size: 10pt }

+ol.CodeRay li { white-space: pre }

+

+.CodeRay .code pre { overflow: auto }

+

+.CodeRay .af { color:#00C }

+.CodeRay .an { color:#007 }

+.CodeRay .av { color:#700 }

+.CodeRay .aw { color:#C00 }

+.CodeRay .bi { color:#509; font-weight:bold }

+.CodeRay .c  { color:#888 }

+

+.CodeRay .ch { color:#04D }

+.CodeRay .ch .k { color:#04D }

+.CodeRay .ch .dl { color:#039 }

+

+.CodeRay .cl { color:#B06; font-weight:bold }

+.CodeRay .co { color:#036; font-weight:bold }

+.CodeRay .cr { color:#0A0 }

+.CodeRay .cv { color:#369 }

+.CodeRay .df { color:#099; font-weight:bold }

+.CodeRay .di { color:#088; font-weight:bold }

+.CodeRay .dl { color:black }

+.CodeRay .do { color:#970 }

+.CodeRay .ds { color:#D42; font-weight:bold }

+.CodeRay .e  { color:#666; font-weight:bold }

+.CodeRay .en { color:#800; font-weight:bold }

+.CodeRay .er { color:#F00; background-color:#FAA }

+.CodeRay .ex { color:#F00; font-weight:bold }

+.CodeRay .fl { color:#60E; font-weight:bold }

+.CodeRay .fu { color:#06B; font-weight:bold }

+.CodeRay .gv { color:#d70; font-weight:bold }

+.CodeRay .hx { color:#058; font-weight:bold }

+.CodeRay .i  { color:#00D; font-weight:bold }

+.CodeRay .ic { color:#B44; font-weight:bold }

+

+.CodeRay .il { background: #eee }

+.CodeRay .il .il { background: #ddd }

+.CodeRay .il .il .il { background: #ccc }

+.CodeRay .il .dl { font-weight: bold ! important; color: #888 ! important }

+

+.CodeRay .in { color:#B2B; font-weight:bold }

+.CodeRay .iv { color:#33B }

+.CodeRay .la { color:#970; font-weight:bold }

+.CodeRay .lv { color:#963 }

+.CodeRay .oc { color:#40E; font-weight:bold }

+.CodeRay .on { color:#000; font-weight:bold }

 .CodeRay .op { }

-.CodeRay .pc { color:#038; font-weight:bold; }

-.CodeRay .pd { color:#369; font-weight:bold; }

-.CodeRay .pp { color:#579; }

-.CodeRay .pt { color:#339; font-weight:bold; }

-.CodeRay .r  { color:#080; font-weight:bold; }

-

-.CodeRay .rx { background-color:#fff0ff; }

-.CodeRay .rx .k { color:#808; }

-.CodeRay .rx .dl { color:#404; }

-.CodeRay .rx .mod { color:#C2C; }

-.CodeRay .rx .fu  { color:#404; font-weight: bold; }

-

-.CodeRay .s  { background-color:#fff0f0; }

-.CodeRay .s  .s { background-color:#ffe0e0; }

-.CodeRay .s  .s  .s { background-color:#ffd0d0; }

-.CodeRay .s  .k { color:#D20; }

-.CodeRay .s  .dl { color:#710; }

-

-.CodeRay .sh { background-color:#f0fff0; }

-.CodeRay .sh .k { color:#2B2; }

-.CodeRay .sh .dl { color:#161; }

-

-.CodeRay .sy { color:#A60; }

-.CodeRay .sy .k { color:#A60; }

-.CodeRay .sy .dl { color:#630; }

-

-.CodeRay .ta { color:#070; }

-.CodeRay .tf { color:#070; font-weight:bold; }

-.CodeRay .ts { color:#D70; font-weight:bold; }

-.CodeRay .ty { color:#339; font-weight:bold; }

-.CodeRay .v  { color:#036; }

-.CodeRay .xt { color:#444; }

+.CodeRay .pc { color:#038; font-weight:bold }

+.CodeRay .pd { color:#369; font-weight:bold }

+.CodeRay .pp { color:#579 }

+.CodeRay .pt { color:#339; font-weight:bold }

+.CodeRay .r  { color:#080; font-weight:bold }

+

+.CodeRay .rx { background-color:#fff0ff }

+.CodeRay .rx .k { color:#808 }

+.CodeRay .rx .dl { color:#404 }

+.CodeRay .rx .mod { color:#C2C }

+.CodeRay .rx .fu  { color:#404; font-weight: bold }

+

+.CodeRay .s  { background-color:#fff0f0 }

+.CodeRay .s  .s { background-color:#ffe0e0 }

+.CodeRay .s  .s  .s { background-color:#ffd0d0 }

+.CodeRay .s  .k { color:#D20 }

+.CodeRay .s  .dl { color:#710 }

+

+.CodeRay .sh { background-color:#f0fff0 }

+.CodeRay .sh .k { color:#2B2 }

+.CodeRay .sh .dl { color:#161 }

+

+.CodeRay .sy { color:#A60 }

+.CodeRay .sy .k { color:#A60 }

+.CodeRay .sy .dl { color:#630 }

+

+.CodeRay .ta { color:#070 }

+.CodeRay .tf { color:#070; font-weight:bold }

+.CodeRay .ts { color:#D70; font-weight:bold }

+.CodeRay .ty { color:#339; font-weight:bold }

+.CodeRay .v  { color:#036 }

+.CodeRay .xt { color:#444 }