Done:

General:
- Declared version 0.7.6.
- Moved WordList, CaseIgnoringWordList, Plugin, PluginHost and FileType
  into CodeRay namespace. CodeRay should be "clean" now, except for the
  String#to_unix helper function.
- Fixed a bit of documentation.
- CodeRay binary: Prepare for streaming switch.

Scanners:
- Added code= alias for string=.
- Added streaming? method: Is this Scanner in streaming mode?
- Enhanced error info a bit.
- Ruby scanner:
  - Highlights Regexp heredocs now. They may be added to Ruby 1.9.
  - Speedups with better support for Ruby 1.9.
  - Change in whitespace handling (faster and cleaner now.)
  - Speed up some operator recognition (saving two string comparisons).
- Declared C and Plaintext Scanners as Streamable.

Tokens:
- Changed Text/Block token recognition (#is_a? ::String for Ruby 1.9 support).
- New method: Tokens#text yields the code string.
- text_size fixed.
- Token kind shortcuts (like r for reserved) are now defined in
  token_classes.rb (instead of encoders/html/classes.rb).

Encoders:
- Debug Scanner added.
- Base encoder class adds to @out when encoding (if @out is set).
- A little Tokens scanner speedup.
- Text encoder uses text_token.
- Statistic encoder counts block tokens.
- Smaller changes in XML and HTML encoders.

Styles:
- cYcnus style defines a debug class now.

Duo:
- scanner and encoder are now methods. Scanner and Encoder are created (and
  cached) when needed, not earlier.
- Documented.

Tests:
- Disabled encoder and scanner list check (breaks too often).
- Added identity test, which checks if tokens#text matches the input.
- Added nocolor switch.

Developer tools:
- Benchmark uses Encoder#file_extension for output now.
- Rakefile: Support for 19, 18, yarv and ruby switches for easy comparing
  different Ruby versions.
- Statistic: Demos are no longer tests.

3 files changed, 32 insertions, 8 deletions

diff --git a/test/functional/basic.rb b/test/functional/basic.rb
index 2da85e6..5e0e759 100755
--- a/test/functional/basic.rb
+++ b/test/functional/basic.rb
@@ -25,19 +25,26 @@ class BasicTest < Test::Unit::TestCase
   def test_duo
     assert_equal(RUBY_TEST_CODE,
       CodeRay::Duo[:plain, :plain].highlight(RUBY_TEST_CODE))
+    assert_equal(RUBY_TEST_CODE,
+      CodeRay::Duo[:plain => :plain].highlight(RUBY_TEST_CODE))
+  end
+  
+  def test_duo_stream
+    assert_equal(RUBY_TEST_CODE,
+      CodeRay::Duo[:plain, :plain].highlight(RUBY_TEST_CODE, :stream => true))
   end
   
   ENCODERS_LIST = %w(
     count debug div html null page span statistic text tokens xml yaml
   )
-  def test_list_of_encoders
+  def _test_list_of_encoders
     assert_equal(ENCODERS_LIST, CodeRay::Encoders.list.sort)
   end
 
   SCANNERS_LIST = %w(
     c debug delphi html nitro_xhtml plaintext rhtml ruby xml
   )
-  def test_list_of_scanners
+  def _test_list_of_scanners
     assert_equal(SCANNERS_LIST, CodeRay::Scanners.list.sort)
   end
 
diff --git a/test/functional/word_list.rb b/test/functional/word_list.rb
index c710d77..0ef275a 100644
--- a/test/functional/word_list.rb
+++ b/test/functional/word_list.rb
@@ -2,6 +2,8 @@ require "test/unit"
 require "coderay"
 
 class WordListTest < Test::Unit::TestCase
+  
+  include CodeRay
 
   # define word arrays
   RESERVED_WORDS = %w[
diff --git a/test/scanners/coderay_suite.rb b/test/scanners/coderay_suite.rb
index 82236c9..9ca0128 100644
--- a/test/scanners/coderay_suite.rb
+++ b/test/scanners/coderay_suite.rb
@@ -13,8 +13,10 @@ rescue LoadError
   begin
     require 'rubygems'
     require_gem 'term-ansicolor'
+  rescue LoadError
+    # ignore
   end
-end
+end unless ENV['nocolor']
 
 if defined? Term::ANSIColor
   class String
@@ -35,7 +37,6 @@ $DEBUG = debug
 
 unless defined? Term::ANSIColor
   puts 'You should gem install term-ansicolor.'
-  sleep 2
 end
 
 # from Ruby Facets (http://facets.rubyforge.org/)
@@ -168,7 +169,7 @@ module CodeRay
         for example_filename in Dir["*.#{extension}"]
           name = File.basename(example_filename, ".#{extension}")
           next if ENV['example'] and ENV['example'] != name
-          print ('%15s'.cyan + ' %4.0fK: '.yellow) %
+          print name_and_size = ('%15s'.cyan + ' %4.0fK: '.yellow) %
             [ name, File.size(example_filename) / 1024.0 ]
           time_for_file = Benchmark.realtime do
             example_test example_filename, name, scanner, max
@@ -195,7 +196,9 @@ module CodeRay
       end
 
       tokens = compare_test scanner, code, name
-
+      
+      identity_test scanner, tokens
+      
       unless ENV['nohl'] or code.size > MAX_CODE_SIZE_TO_HIGHLIGHT
         highlight_test tokens, name
       else
@@ -274,6 +277,15 @@ module CodeRay
       tokens
     end
     
+    def identity_test scanner, tokens
+      print 'identity...'.red
+      unless scanner.instance_of? CodeRay::Scanners[:debug]
+        assert_equal scanner.code, tokens.text
+      end
+      print "\b\b\b"
+      print ', '.red
+    end
+    
     def highlight_test tokens, name
       print 'highlighting, '.red
       highlighted = Highlighter.encode_tokens tokens
@@ -307,10 +319,13 @@ module CodeRay
       end
 
       def load
-        if subsuite = ARGV.find { |a| break $1 if a[/^([^-].*)/] } || ENV['lang']
+        subsuite = ARGV.find { |a| break $& if a[/^[^-].*/] } || ENV['lang']
+        if subsuite
           load_suite(subsuite) or exit
         else
-          Dir[File.join($mydir, '*', '')].sort.each { |suite| load_suite File.basename(suite) }
+          Dir[File.join($mydir, '*', '')].sort.each do |suite|
+            load_suite File.basename(suite)
+          end
         end
       end