summaryrefslogtreecommitdiff
path: root/Changes.textile
blob: f63c8c05e8b4347ef775760215ec93d24588a02c (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
h1=. CodeRay Version History
 
p=. _This files lists all changes in the CodeRay library since the 0.8.4 release._
 
{{toc}}
 
h2. Changes in 1.0
 
h3. Direct Streaming
 
CodeRay 1.0 introduces _Direct Streaming_ as a faster and simpler alternative to Tokens. It means that all Scanners, Encoders and Filters had to be rewritten, and that older scanners using the Tokens API are no longer compatible with this version.
 
The benefit of this change is more speed (benchmarks show 10% to 50% more tokens per second compared to CodeRay 0.9), a simpler API, and less code.
 
Changes related to the new tokens handling include:
* *CHANGED*: The Scanners now call Encoders directly; tokens are not added to a Tokens array, but are send to the Encoder as a method call. The Tokens representation (which can be seen as a cache now) is still present, but as a special case; Tokens just encodes the given tokens into an Array for later use.
* *CHANGED*: The token actions (@text_token@, @begin_group@ etc.) are now public methods of @Encoder@ and @Tokens@.
* *REWRITE* of all Scanners, Encoders, Filters, and Tokens.
* *RENAMED* @:open@ and @:close@ actions to @:begin_group@ and @:end_group@.
* *RENAMED* @open_token@ and @close_token@ methods to @begin_group@ and @end_group@.
* *NEW* method @#tokens@ allows to add several tokens to the stream. @Tokens@ and @Encoders::Encoder@ define this method.
* *CHANGED* The above name changes also affect the JSON, XML, and YAML encoders. CodeRay 1.0 output will be incompatible with earlier versions.
* *REMOVED* @TokenStream@ and the @Streamable@ API and all related features like @NotStreamableError@ are now obsolete and have been removed.

h3. General changes
 
* *IMPROVED* documentation in general; additions, corrections and cleanups.
* *NEW*: Extended support and usage of HTML5 and CSS 3 features.
* *IMPROVED* Ruby 1.9 support (_._ not in @$LOAD_PATH@)
* *IMPROVED* speed of HTML encoding when using CSS classes.
* *IMPROVED*: Faster startup by replacing @require@ with @autoload@. CodeRay
  features will now only be loaded when they're needed.
* *FIXED* some image links in the documentation.
* *ADDED* a lot of tests.

h3. Helpers
 
The helper classes were cleaned up; see below for details.
 
* *CHANGED* @Plugin@ API was simplified and stripped of all unnecessary features.
* *CHANGED* Moved @GZip@ and @FileType@ libraries into @CodeRay@; cleaned them up.

h3. @Tokens@
 
* *REMOVED* methods @#stream?@, @#each_text_token@.
* *REMOVED* @#text@ and @#text_size@ methods. Use the @Text@ encoder instead.
* *REMOVED* special implementation of @#each@ taking a filter parameter. Use @TokenKindFilter@ instead.
* *NEW* methods @encode_with@, @count@, @begin_group@, @end_group@, @begin_line@, and @end_line@.

h3. *RENAMED*: @TokenKinds@
 
Renamed from @Tokens::ClassOfKind@ (was also @Tokens::AbbreviationForKind@ for a while).
The term "token class" is no longer used in CodeRay. Instead, tokens have _kinds_.
See "#122":http://redmine.rubychan.de/issues/122.
 
* *RENAMED* @:pre_constant@ and @:pre_type@ to @:predefined_constant@ and @predefined_type@.
* *REMOVED* token kinds @:attribute_name_fat@, @:attribute_value_fat@, @:operator_fat@,
  @:tag_fat@, @:xml_text@, @:nesting_delimiter@, @:open@, and @:close@.
* *ADDED* token kinds @:filename@, @:namespace@, and @:eyecatcher@.
* *CHANGED*: Don't raise error for unknown token kinds unless in @$CODERAY_DEBUG@ mode.
* *CHANGED* the value for a token kind that is not highlighted from
  @:NO_HIGHLIGHT@ to @false@.

h3. @Duo@
 
* *NEW* method @call@ for allowing code like @CodeRay::Duo[:python => :yaml].(code)@ in Ruby 1.9.

h3. @Encoders::CommentFilter@
 
* *NEW* alias @:remove_comments@

h3. @Encoders::Filter@
 
* *NEW* option @tokens@.
* *CHANGED*: Now it simply delegates to the output.
* *REMOVED* @include_text_token?@ and @include_block_token?@ methods.

h3. @Encoders::HTML@
 
The HTML encoder was cleaned up and simplified.
 
* *CHANGED* the default style to @:alpha@.
* *NEW*: HTML5 and CSS 3 compatible.
  See "#215":http://redmine.rubychan.de/issues/215.
* *ADDED* support for @:line_number_anchors@.
  See "#208":http://redmine.rubychan.de/issues/208.
* *CHANGED*: Use double click to toggle line numbers in table mode (as single 
  click jumps to an anchor.)
* *REMOVED* support for @:line_numbers => :list@.
* *FIXED* splitting of lines for @:line_numbers => :inline@, so that the line
  numbers don't get colored, too.
* *RENAMED* @Output#numerize@ to @#number@, which is an actual English word.

h3. @Encoders::LinesOfCode@
 
* *CHANGED*: @compile@ and @finish@ methods are now protected.

h3. @Encoders::Terminal@
 
* *RENAMED* from @Encoders::Term@, added @:term@ alias.
* *CLEANUP*: Use @#setup@'s @super@, don't use @:procedure@ token class.
* *CHANGED*: @#token@'s second parameter is no longer optional.
* *REMOVED* colors for obsolete token kinds.
* *FIXED* handling of line tokens.

h3. @Encoders::Text@
 
* *FIXED* default behavior of stripping the trailing newline.

h3. *RENAMED*: @Encoders::TokenKindFilter@
 
Renamed from @TokenClassFilter@.
 
* *NEW*: Handles token groups.
  See "#223":http://redmine.rubychan.de/issues/223.
* *RENAMED* @include_block_token?@ to @include_group?@.

h3. @Encoders::Statistic@
 
* *CHANGED*: Tokens actions are counted separately.

h3. @Scanners::Scanner@
 
* *REMOVED* helper method @String#to_unix@.
* *REMOVED* method @#streamable?@.
* *REMOVED* @#marshal_load@ and @#marshal_dump@.
* *RENAMED* class method @normify@ to @normalize@; it also deals with encoding now.
* *NEW* methods @#file_extension@ and @#encoding@.
* *NEW*: The @#tokenize@ method also takes an Array of Strings as source. The
  code is highlighted as one and split into parts of the input lengths
  after that using @Tokens#split_into_parts@.

h3. *NEW*: @Scanners::Clojure@
 
Thanks to Licenser, CodeRay now supports the Clojure language.

h3. @Scanners::CSS@
 
* *NEW*: Rudimentary support for the @attr@, @counter@, and @counters@ functions.
  See "#224":http://redmine.rubychan.de/issues/224.
* *NEW*: Rudimentary support for CSS 3 colors.
* *CHANGED*: Attribute selectors are highlighted as @:attribute_name@ instead of @:string@.
* *CHANGED*: Comments are scanned as one token instead of three.

h3. @Scanners::Debug@
 
* *NEW*: Support for line tokens (@begin_line@ and @end_line@ represented by @[@ and @]@.)
* *FIXED*: Don't send @:error@ and @nil@ tokens for buggy input any more.
* *FIXED*: Closes unclosed tokens at the end of @scan_tokens@.
* *IMPROVED*: Highlight unknown tokens as @:error@.
* *CHANGED*: Raises an error when trying to end an invalid token group.

h3. @Scanners::Delphi@
 
* *FIXED*: Closes open string groups.

h3. @Scanners::Diff@
 
* *NEW*: Highlighting of code based on file names.
  See ticket "#52":http://redmine.rubychan.de/issues/52.
  
  Use the @:highlight_code@ option to turn this feature off. It's enabled
  by default.
  
  This is a very original feature. It enables multi-language highlighting for
  diff files, which is especially helpful for CodeRay development itself. The
  updated version of the scanner test suite generated .debug.diff.html files
  using this.
  
  Note: This is still experimental. Tokens spanning more than one line
  may get highlighted incorrectly. CodeRay tries to keep scanner states
  between the lines and changes, but the quality of the results depend on
  the scanner.
* *NEW*: Inline change highlighting, as suggested by Eric Thomas.
  See ticket "#227":http://redmine.rubychan.de/issues/227 for details.
  
  Use the @:inline_diff@ option to turn this feature off. It's enabled by
  default.
  
  For single-line changes (that is, a single deleted line followed by a single
  inserted line), this feature surrounds the changed parts with an
  @:eyecatcher@ group which appears in a more saturated background color.
  The implementation is quite complex, and highly experimental. The problem
  with multi-layer tokenizing is that the tokens have to be split into parts.
  If the inline change starts, say, in the middle of a string, then additional
  @:end_group@ and @:begin_group@ tokens must be inserted to keep the group
  nesting intact. The extended @Scanner#tokenize@ method and the new
  @Tokens#split_into_parts@ method take care of this.
* *NEW*: Highlight the file name in the change headers as @:filename@.
* *CHANGED*: Highlight unknown lines as @:comment@ instead of @:head@.

h3. @Scanners::HTML@
 
* *FIXED*: Closes open string groups.

h3. @Scanners::JavaScript@
 
* *IMPROVED*: Added @NaN@ and @Infinity@ to list of predefined constants.
* *IMPROVED* recognition of RegExp literals with leading spaces.

h3. @Scanners::Java@
 
* *NEW*: Package names are highlighted as @:namespace@.
  See "#210":http://redmine.rubychan.de/issues/210.

h3. *RENAMED*: @Scanners::Text@ (was @Scanners::Plaintext@)
 
* *IMPROVED*: Just returns the string without scanning (faster).
 
 This is much faster than scanning until @/\z/@ in Ruby 1.8.

h3. @Scanners::Python@
 
* *CHANGED*: Docstrings are highlighted as @:comment@.
  See "#190":http://redmine.rubychan.de/issues/190.

h3. *NEW*: @Scanners::Raydebug@
 
Copied from @Scanners::Debug@, highlights the token dump instead of importing it. It also reacts to the @.raydebug@ file name suffix now.

h3. @Scanners::Ruby@
 
* *ADDED* more predefined keywords (see http://murfy.de/ruby-constants).
* *IMPROVED* support for singleton method definitions.
  See "#147":http://redmine.rubychan.de/issues/147.
* *FIXED*: Don't highlight methods with a capital letter as constants
  (eg. GL.PushMatrix).
* *NEW*: Highlight buggy floats (like .5) as @:error@.
* *CLEANUP* of documentation, names of constants and variables, state handling.
  
  Moved @StringState@ class from @patterns.rb@ into a separate file.
* *NEW*: Complicated rule for recognition of @foo=@ style method names.
* *NEW*: Handles @:keep_state@ option (a bit; experimental).
  
  Actually, Ruby checks if there is @[~>=]@, but not @=>@ following the name.
  
* *REMOVED* @EncodingError@

h3. @Scanners::Scheme@
 
* *CHANGED*: Does use @:operator@ instead of @:operator_fat@ now.

h3. @Scanners::SQL@
 
* *IMPROVED*: Extended list of keywords and functions (thanks to
  Joshua Galvez, Etienne Massip, and others).
 
  See "#221":http://redmine.rubychan.de/issues/221.
* *FIXED*: Closes open string groups.
* *FIXED*: Words after @.@ are always recognized as @:ident@.

h3. @Scanners::YAML@
 
* *FIXED*: Allow spaces before colon in mappings.
 
  See "#231":http://redmine.rubychan.de/issues/231.

h3. *NEW*: @Styles::Alpha@
 
A style that uses transparent HSLA colors as defined in CSS 3.
See "#199":http://redmine.rubychan.de/issues/199.
 
It also uses the CSS 3 property @user-select: none@ to keep the user from selecting the line numbers. This is especially nice for @:inline@ line numbers.
See "#226":http://redmine.rubychan.de/issues/226.

h3. @WordList@
 
Stripped down to 19 LOC.

* *REMOVED* caching option because it creates memory leaks.
* *REMOVED* block option.

h3. @FileType@
 
* *REMOVED* @FileType#shebang@ is a protected method now.
* *NEW*: Recognizes @.gemspec@, @.rjs@, @.rpdf@ extensions, @Gemfile@, and @Capfile@ as Ruby.
  
  Thanks to the authors of the TextMate Ruby bundle!

h3. @Plugin@
 
* *IMPROVED*: @register_for@ sets the @plugin_id@; it can now be a @Symbol@.
* *ADDED* @PluginHost#const_missing@ method: Plugins are loaded automatically.
  Using @Scanners::JavaScript@ in your code loads @scanners/java_script.rb@.
* *ADDED* @#all_plugins@ method to simplify getting 
  information about all available plugins (suggested by bnhymn).

h3. @GZip@
 
* *MOVED* into @CodeRay@ namespace.
* *MOVED* file from @gzip_simple.rb@ to @gzip.rb@.
* *REMOVED* @String@ extensions.

h3. Internal API changes
 
* *FIXED* @Encoders::HTML#token@'s second parameter is no longer optional.
* *CHANGED* @Encoders::HTML::Output@'s API.
* *REMOVED* lots of unused methods.


h2. Changes in 0.9.8 "banister" [2011-05-01]
 
Fixes for JRuby's 1.9 mode and minor issues.
 
h3. Rake tasks
 
* *REMOVED* obsolete @has_rdoc@ gem specification, fixing a warning.

h3. @Scanners::Scanner@
 
* *NEW* method @#scan_rest@ replaces @scan_until(/\z/)@, which is broken in JRuby 1.6 --1.9 mode.
  See "#297":http://redmine.rubychan.de/issues/297.

h3. @Scanners::CSS@
 
* *FIXED* LOC counting (should be 0).
  See "#296":http://redmine.rubychan.de/issues/296.

h3. @Scanners::Ruby@
 
* *FIXED* the @IDENT@ pattern not to use character properties, which are broken in JRuby 1.6 --1.9 mode.
  See "#297":http://redmine.rubychan.de/issues/297, thanks to banister for reporting!

h3. @Scanners::SQL@
 
* *ADDED* more keywords: @between@, @databases@, @distinct@, @fields@, @full@, @having@, @is@, @prompt@, @tables@.
  See "#221":http://redmine.rubychan.de/issues/221, thanks to Etienne Massip again.

h3. @FileType@
 
* *NEW* regonizes ColdFusion file type extensions @.cfm@ and @.cfc@ as XML.
  See "#298":http://redmine.rubychan.de/issues/298, thanks to Emidio Stani.


h2. Changes in 0.9.7 "Etienne" [2011-01-14]
 
Fixes a dangerous JavaScript scanner bug, and a testing problem with Ruby 1.9.1.
 
h3. Tests
 
* *FIXED* The functional tests now load the lib directory (instead of the gem) in Ruby 1.9.1.

h3. @Scanners::JavaScript@
 
* *FIXED* @KEY_CHECK_PATTERN@ regexp
  See "#264":http://redmine.rubychan.de/issues/264, thanks to Etienne Massip!


h2. Changes in 0.9.6 "WoNáDo" [2010-11-25]
 
Minor improvements to the Ruby scanner and a fix for Ruby 1.9.
 
h3. @Scanners::Ruby@
 
* *IMPROVED* handling of new hash syntax (keys are marked as @:key@ now,
  colon is a separate @:operator@ token, all idents can be used as keys)
  See "#257":http://code.licenser.net/issues/257, thanks to WoNáDo!
* *ADDED* @__ENCODING__@ magic constant (Ruby 1.9)
* *FIXED*: Scanner no longer tries to modify the input string on Ruby 1.9.
  See "#260":http://code.licenser.net/issues/260, thanks to Jan Lelis!


h2. Changes in 0.9.5 "Germany.rb" [2010-09-28]
 
Support for Rubinius ("#251":http://redmine.rubychan.de/issues/251), improved mutlibyte handling, Ruby 1.9 syntax, and valid HTML.
 
h3. @Encoders::HTML@
 
* *FIXED*: Line tokens use @span@ with @display: block@ instead of @div@, which was invalid HTML ("#255":http://redmine.rubychan.de/issues/255).

h3. @Scanner::Scanner@
 
* *IMPROVED* handling of encodings in Ruby 1.9: UTF-8 and Windows-1252 are checked.
* *NEW*: Invalid chars will be converted to @?@ in Ruby 1.9.
* *FIXED* @string=@ method for Rubinius. See "issue 481":http://github.com/evanphx/rubinius/issues/481 on their site.

h3. @Scanners::CSS@
 
* *FIXED*: Don't use non-ASCII regexps.

h3. @Scanners::Diff@
 
* *FIXED*: Highlight unexpected lines as @:comment@.

h3. @Scanners::PHP@
 
* *FIXED*: Use @ASCII-8BIT@ encoding for now.

h3. @Scanners::Ruby@
 
* *ADDED* support for some Ruby 1.9 syntax ("#254":http://redmine.rubychan.de/issues/254):
** the @->@ lambda shortcut
** new Hash syntax using colons (@{ a: b }@)
* *FIXED*: Use @UTF-8@ encoding.
* *IMPROVED* unicode support on Ruby 1.8 ("#253":http://redmine.rubychan.de/issues/253).
* *FIXED* recognition of non-ASCII identifiers in Ruby 1.9, JRuby, and Rubinius ("#253":http://redmine.rubychan.de/issues/253).
* *CHANGED* heredoc recognition to ignore delimiters starting with a digit. This is incorrect, but causes less false positives.

h3. @Scanners::SQL@
 
* *FIXED* scanning of comments; nice catch, Rubinius!
  ("#252":http://redmine.rubychan.de/issues/252)


h2. Changes in 0.9.4 "Ramadan" [2010-08-31]
 
Updated command line interface and minor scanner fixes for the Diff, HTML, and RHTML scanners.
 
h3. @coderay@ executable
 
* *FIXED*: Partly rewritten, simplified, fixed.
  ("#244":http://redmine.rubychan.de/issues/244)

h3. @Scanners::Diff@
 
* *FIXED* handling of change headers with code on the same line as the @@ marker.
  ("#247":http://redmine.rubychan.de/issues/242)

h3. @Scanners::HTML@
 
* *FIXED* a missing regexp modifier that slowed down the scanning.
  ("#245":http://redmine.rubychan.de/issues/245)

h3. @Scanners::RHTML@
 
* *FIXED* highlighting of ERB comment blocks.
  ("#246":http://redmine.rubychan.de/issues/246)


h2. Changes in 0.9.3 "Eyjafjallajökull" [2010-04-18]
 
* *FIXED*: Documentation of Tokens.
  ("#218":http://redmine.rubychan.de/issues/218)
 
h3. @coderay@ executable
 
* *NEW*: automatic TTY detection (uses @Term@ encoder)
* *NEW*: optional 3rd parameter for the filename
* *FIXED*: Converted to UNIX format.
* *FIXED*: Warn about generated files.
* *FIXED*: Ensure line break after the output (especially for LoC counter).

h3. @Scanners::JavaScript@
 
* *FIXED*: Don't keep state of XML scanner between calls for E4X literals.

h3. @Scanners::Java@, @Scanners::JSON@
 
* *FIXED*: Close unfinished strings with the correct token kind.


h2. Changes in 0.9.2 "Flameeyes" [2010-03-14]
 
* *NEW* Basic tests and a _Rakefile_ are now included in the Gem. [Flameeyes]
  A @doc@ task is also included.
* *FIXED* Use @$CODERAY_DEBUG@ for debugging instead of @$DEBUG@. [Trans]
  ("#192":http://redmine.rubychan.de/issues/192)
* *REMOVED* @Term::Ansicolor@ was bundled under _lib/_, but not used. [Flameeyes]
  ("#205":http://redmine.rubychan.de/issues/205)
* *WORKAROUND* for Ruby bug 
  "#2745":http://redmine.ruby-lang.org/issues/show/2745
 
h3. @Encoders::Term@
 
* *FIXED* strings are closed correctly
  ("#138":http://redmine.rubychan.de/issues/138)
* *FIXED* several token kinds had no associated color
  ("#139":http://redmine.rubychan.de/issues/139)
* *NEW* alias @terminal@
  
  *NOTE:* This encoder will be renamed to @Encoders::Terminal@ in the next release.

h3. @Scanners::Debug@
 
* *FIXED* Don't close tokens that are not open. Send @:error@ token instead.

h3. @Scanners::Groovy@
 
* *FIXED* token kind of closing brackets is @:operator@ instead of @nil@
  ("#148":http://redmine.rubychan.de/issues/148)

h3. @Scanners::PHP@
 
* *FIXED* allow @\@ operator (namespace separator)
  ("#209":http://redmine.rubychan.de/issues/209)

h3. @Scanners::YAML@
 
* *FIXED* doesn't send debug tokens when @$DEBUG@ is true [Trans]
  ("#149":http://redmine.rubychan.de/issues/149)


h2. Changes in 0.9.1 [2009-12-31]
 
h3. Token classes
 
* *NEW* token classes @:complex@, @:decorator@, @:imaginary@
  (all for Python)
* *REMOVED* token class @:procedure@
  – use @:function@ or @:method@ instead.

h3. @Tokens@
 
* *NEW* method @#scanner@
  
  Stores the scanner.
* *REMOVED* methods @.write_token@, @.read_token@, @.escape@, @.unescape@
  
  They were only used by the @Tokens@ encoder, which was removed also.

h3. @Encoders::Encoder@
 
* *REMOVED* Don't require the _stringio_ library.
* *NEW* public methods @#open_token@, @#close_token@, @#begin_line@, @#end_line@
  These methods are called automatically, like @#text_token@.
* *NEW* proteced method @#append_encoded_token_to_output@

h3. @Encoders::Tokens@
 
* *REMOVED* – use @Tokens#dump@ and @Tokens.load@.

h3. @Encoders::Filter@
 
* *NEW*
  A @Filter@ encoder has another @Tokens@ instance as output.

h3. @Encoders::TokenClassFilter@
 
* *NEW*
  
  It takes 2 options, @:exclude@ and @:include@, that specify which token classes
  to include or exclude for the output. They can be a single token class,
  an @Array@ of classes, or the value @:all@.

h3. @Encoders::CommentFilter@
 
* *NEW*
  
  Removes tokens of the @:comment@ class.

h3. @Encoders::LinesOfCode@
 
* *NEW*
  
  Counts the lines of code according to the @KINDS_NOT_LOC@ token class list
  defined by the scanner. It uses the new @TokenClassFilter@.
  
  Alias: @:loc@, as in @tokens.loc@.

h3. @Encoders::JSON@
 
* *NEW*
  
  Outputs tokens in a simple JSON format.

h3. @Encoders::Term@
 
* *NEW* (beta, by Rob Aldred)
  
  Outputs code highlighted for a color terminal.

h3. @Encoders::HTML@
 
* *NEW* option @:title@ (default value is _CodeRay output_)
  
  Setting this changes the title of the HTML page.
* *NEW* option @:highlight_lines@ (default: @nil@)
  
  Highlights the given set of line numbers.
- *REMOVED* option @:level@
  
  It didn't do anything. CodeRay always outputs XHTML.

h3. @Encoders::Text@
 
* Uses @Encoder@ interface with @super@ and @#text_token@.

h3. @Encoders::XML@
 
* @FIXED@ ("#94":http://redmine.rubychan.de/issues/94)
  
  It didn't work at all.

h3. Scanners
 
* *NEW* Mapped @:h@ to @:c@, @:cplusplus@ and @:'c++'@ to @:cpp@,
  @:ecma@, @:ecmascript@, @:ecma_script@ to @:java_script@,
  @:pascal@ to @:delphi@, and @:plain@ to @:plaintext@.

h3. @Scanners::Scanner@
 
* *NEW* constant @KINDS_NOT_LOC@
  
  A list of all token classes not considered in LOC count.
  Added appropriate values for scanners.
* *NEW* method @#lang@ returns the scanner's lang, which is its @plugin_id@.
* *FIXED* automatic, safe UTF-8 detection _[Ruby 1.9]_
* *FIXED* column takes care of multibyte encodings _[Ruby 1.9]_
* *FIXED* is dumpable (@Tokens@ store their scanner in an @@scanner@ variable)

h3. @Scanners::Cpp@
 
* *NEW* (C++)

h3. @Scanners::Groovy@
 
* *NEW* (beta)

h3. @Scanners::Python@
 
* *NEW*

h3. @Scanners::PHP@
 
* *NEW* (based on Stefan Walk's work)

h3. @Scanners::SQL@
 
* *NEW* (based on code by Josh Goebel)

h3. @Scanners::C@
 
* *IMPROVED* added a list of @:directive@ tokens that were @:reserved@ before
* *IMPROVED* detection of labels
* *IMPROVED* allow @1L@ and @1LL@ style literals

h3. @Scanners::CSS@
 
* *IMPROVED* element selectors are highlighted as @:type@ instead of @:keyword@

h3. @Scanners::Delphi@
 
* *IMPROVED* Don't cache tokens in CaseIgnoringWordList.

h3. @Scanners::Java@
 
* *IMPROVED* @assert@ is highlighted as a @:keyword@ now
* *IMPROVED* @const@ and @goto@ are highlighted as @:reserved@
* *IMPROVED* @false@, @true@, and @null@ are highlighted as @:pre_constant@
* *IMPROVED* @threadsafe@ is no longer a @:directive@
* *IMPROVED* @String@ is highlighted as a @:pre_type@
* *IMPROVED* built-in classes ending with _Error_ or _Exception_ are
  highlighted as a @:exception@ instead of @:pre_type@

h3. @Scanners::JavaScript@
 
* *NEW* a list of @PREDEFINED_CONSTANTS@ to be highlighted as @:pre_constant@
* *NEW* XML literals are recognized and highlighted
* *NEW* function name highlighting
* *IMPROVED* @.1@ is highlighted a number
* *FIXED* strings close with the correct kind when terminated unexpectedly

h3. @Scanners::JSON@
 
* *IMPROVED* constants (@true@, @false@, @nil@) are highlighted as @:value@

h3. @Scanners::Ruby@
 
* *IMPROVED* @Patterns::KEYWORDS_EXPECTING_VALUE@ for more accurate
  @value_expected@ detection
* *IMPROVED* handling of @\@ as a string delimiter
* *IMPROVED* handling of unicode strings; automatic switching to unicode
* *IMPROVED* highlighting of @self.method@ definitions
* *REMOVED* @Patterns::FANCY_START_SAVE@ (obsolete)
* *FIXED* encoding issues _[Ruby 1.9]_
* *FIXED* a problem in early Ruby 1.8.6 patch versions with @Regexp.escape@

h3. @Scanners::YAML@
 
* *IMPROVED* indentation detection

h3. @Styles::Cycnus@
 
* changed a few colors (exceptions, inline strings, predefined types)

h3. @Plugin@
 
* *NEW* method @#title@
  
  Set and get the plugin's title. Titles can be arbitrary strings.
* *NEW* method @#helper@ loads helpers from different plugins
  
  Use this syntax: @helper 'other_plugin/helper_name'@

h3. @FileType@
 
* *NEW* @FileType[]@ takes @Pathname@ instances
* *NEW* regonizes @.cc@, @.cpp@, @.cp@, @.cxx@, @.c++@, @.C@, @.hh@, @.hpp@, @.h++@, @.cu@ extensions (C++)
  
  Thanks to Sander Cox and the TextMate C bundle.
* *NEW* regonizes @.pas@, @.dpr@ extensions (Delphi)
* *NEW* regonizes @.gvy@, @.groovy@ extensions (Groovy)
* *NEW* regonizes @.php@, @.php3@, @.php4@, @.php5@ extensions (PHP)
* *NEW* regonizes @.py@, @.py3@, @.pyw@ extensions (Python)
* *NEW* regonizes @.rxml@ extension (Ruby)
* *NEW* regonizes @.sql@ extension (SQL)
* File types list was sorted alphabetically.

h3. @CaseIgnoringWordList@
 
* *FIXED* ("#97":http://redmine.rubychan.de/issues/97)
  
  The default value is no longer ignored.

h3. @ForRedCloth@
 
* *FIXED* for RedCloth versions 4.2.0+ ("#119":http://redmine.rubychan.de/issues/119)

h3. Cleanups
 
* warnings about character classes _[Ruby 1.9]_
* encoding issues _[Ruby 1.9]_
* documentation, code