summaryrefslogtreecommitdiff
path: root/lib/coderay/scanners
Commit message (Collapse)AuthorAgeFilesLines
...
* Added more file extensions to Scannersmurphy2008-09-166-2/+6
| | | | | | | * Added file_extensions to Delphi, XML, Debug, Scheme, and NitroXHTML scanners. * Removed obsolete extension declarations from Scanner test suites. * Minor cleanups.
* [KK] Just a little code cleanup and new task 187.murphy2008-07-291-1/+1
|
* Improved Ruby scanner to use +/- signs only when appropriate (tests adjusted.)murphy2008-01-212-13/+17
| | | | | Ignore test/scanners/*/*.expected.html
* Lib:murphy2008-01-072-24/+29
| | | | | | | | | | | | | | | | | | | | | | | | | | | - Encoder: removed a warning - Encoders::HTML: don't shadow outer variable - Plugin: move require_plugin into class namespace - Ruby Scanner: - "alias" keyword recognition - better regexp/division distinction - recognize ~, !, !=, and !~ as method names (partly Ruby 1.9 only) - reordered states for speed Tests: - updated coderay-suite to use gem instead of require_gem - general improvements (more colors!, new parameter: new, new syntax lang.test for only and new) - fixed ruby suite - adjusted a lot of Ruby tests (alias uses methods now) - new tests: ruby/operators, ruby/regexp Samples: - fixed/updated ('bout time) Rake tasks: - updated to use new rubygems API
* Benchmark, comparison: replaced SilverCity with pygmentsmurphy2007-04-241-1/+1
| | | | | | | A bit of code and documentation cleanup HTML encoder: fixed css class output for NO_HIGHLIGHT tokens when :info is set. Fixed Text encoder.
* New Scanner: Scheme (thanks closure!)murphy2007-01-012-0/+143
| | | | | | | | | | | | | | | Test and example added. Token changed: operator_fat instead of operator_name (for use with LISP-like parentheses). Added file_extension for Scanners. Improved CodeRay::Suite: - uses scanners file extension now - example parameter is now named "only" - only param overwrite MAX_CODE_SIZE_TO_HIGHLIGHT
* Done:murphy2007-01-014-32/+36
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | General: - Declared version 0.7.6. - Moved WordList, CaseIgnoringWordList, Plugin, PluginHost and FileType into CodeRay namespace. CodeRay should be "clean" now, except for the String#to_unix helper function. - Fixed a bit of documentation. - CodeRay binary: Prepare for streaming switch. Scanners: - Added code= alias for string=. - Added streaming? method: Is this Scanner in streaming mode? - Enhanced error info a bit. - Ruby scanner: - Highlights Regexp heredocs now. They may be added to Ruby 1.9. - Speedups with better support for Ruby 1.9. - Change in whitespace handling (faster and cleaner now.) - Speed up some operator recognition (saving two string comparisons). - Declared C and Plaintext Scanners as Streamable. Tokens: - Changed Text/Block token recognition (#is_a? ::String for Ruby 1.9 support). - New method: Tokens#text yields the code string. - text_size fixed. - Token kind shortcuts (like r for reserved) are now defined in token_classes.rb (instead of encoders/html/classes.rb). Encoders: - Debug Scanner added. - Base encoder class adds to @out when encoding (if @out is set). - A little Tokens scanner speedup. - Text encoder uses text_token. - Statistic encoder counts block tokens. - Smaller changes in XML and HTML encoders. Styles: - cYcnus style defines a debug class now. Duo: - scanner and encoder are now methods. Scanner and Encoder are created (and cached) when needed, not earlier. - Documented. Tests: - Disabled encoder and scanner list check (breaks too often). - Added identity test, which checks if tokens#text matches the input. - Added nocolor switch. Developer tools: - Benchmark uses Encoder#file_extension for output now. - Rakefile: Support for 19, 18, yarv and ruby switches for easy comparing different Ruby versions. - Statistic: Demos are no longer tests.
* Fixed RHTML and NitroXHTML scanners to use new :inline_delimiter tokens.murphy2006-10-202-6/+6
|
* Added Debug scanner (for .raydebug files).murphy2006-10-201-0/+60
|
* Improved Delphi scanner.murphy2006-10-201-6/+23
|
* Ruby scanner: #{...} delimiters are now :inline_delimiter to solve CSS problem.murphy2006-10-181-2/+2
| | | | | | New token: :inline_delimiter Scanner tests suite revamped: colorization, split up into methods.
* Fixed warning in bench.rb.murphy2006-10-182-19/+35
| | | | | | | | | | Ruby Scanner: improved handling of ambiguos values. - operator methods are now :ident instead of :operator - fancy_allowed and regexp_allowed are now value_expected - value_expected works in operator method calls - tests enhanced and adjusted Scanner tests: new diffed option, diff doesn't imply noassert anymore
* Fixed functional test.murphy2006-10-182-2/+3
| | | | | | | Fixed Ruby scanner (/= was scanned as regexp!) Made bench.rb run from any folder. Made scanner test diff function work via EDITOR instead of gvimdiff.
* Tests:murphy2006-10-153-1/+8
| | | | | | | | | | | | | | | | - improved coderay_suite.rb (random and shuffled tests, max parameter, scannerlang->lang, sorted test cases...) - changed html output extension to .actual.html to svn:ignore them - fixed some tests (deleted $Id$ etc.) - made XHTML testcase work Scanners: - fixed HTML, Delphi and Nitro scanners thanks to new tests Engine: - Tokens#fix and #fix! added (yet to be tested) - improved Scanner#raise_inspect a bit Converted more files to UNIX format (go away, stinkin' \r!)
* Ruby scanner: deleted regexp highlighting (didn't work anyway)murphy2006-09-281-40/+0
| | | | | HTML highlighter: unknown token classes now yield a CSS class with that name instead of :error A new Ruby test
* Fixed Plaintext scanner (crashed on empty text - ouch!!)murphy2006-07-111-1/+2
| | | | Added coderay_stylesheet executable (should be "coderay stylesheet css" or just "coderay sty" some day.)
* rake test now runs in debug mode.murphy2006-07-113-249/+249
| | | | All .rb files converted to UNIX format (where did the \r come from?)
* Changed error handling of all scanners: :error tokens are OK now, even in ↵murphy2006-07-117-1055/+1090
| | | | | | | | | | | | | debug mode, but token kind is nil unless assigned. Small fixes for C and Ruby scanners. Renamed local variable type to kind in Ruby scanner. Improved RHTML scanner to recognize -%> as delimiter. HTML encoder: improved handling of malformed token strings. Fixed PluginHost#inspect including docu. Scanner#raise_inspect also shows state if given.
* Big re-indenting - no more tabs!murphy2006-07-1010-651/+651
|
* Fixed another bug in the Ruby scanner, this time it was unfinished heredocs ↵murphy2006-07-092-574/+575
| | | | | | with empty delimiter. Fixed documentation uploading.
* [BUG] Fixed bug in Ruby-Scanner (input ends with unfinished escape in ↵murphy2006-07-091-0/+2
| | | | | | string) (thanks to bovi again.) Test case added.
* Fix:murphy2006-06-281-1/+1
| | | | | | | | | | | | | | HTML Scanner: Fixed a nasty bug with invalid entities. Thanks to Daniel Bovensiepen. Scanner: added Scanner.normify. Plugin: small documentation fix. Code statistic: added lib root folder rake_helpers/code_statistics.rb: added ability to include non-Ruby code in the statistics used it to count the mountain of test data There is some mixed space/tab indentation now. Complete 2-space-re-indentation will follow.
* New Version: 0.7.0!murphy2006-04-192-2/+20
| | | | | | | | | | | Adjusted gem.rake. HTML scanner fixed. Enhanced filetype.rb: .rake files, xml, yaml (preparing for YAML scanner.) Enhanced test/ruby/1.in.rb. Added XML scanner with example. plugin.rb: made all_plugin_names public.
* Re-indented everything. Sorry to break the blame chain.murphy2006-04-169-32/+45
|
* All scanners revisited not to produce errors unless in DEBUG mode. [thx@Daniel]murphy2006-04-164-5/+15
| | | | | | | Fixed numerization for input not ending with \n. Added test cases in C scanner tests. Added WoNáDo as contributer for last Ruby scanner fix. Updated TODO.
* styles/_map.rb added. :cycnus is now default style. html/css.rb adjusted.murphy2006-04-152-36/+40
| | | | | | | | | | ROADMAP and TODO updated. Ruby scanner: Fixed a bug: Fancy strings don't allow \w delimiters. They _do_ also allow whitespace delimiters, but adding this would cause many scan errors. So I leave this bug unfixed.
* Large update: Scanners for HTML, RHTML and Nitro-XHTML added.murphy2006-04-106-36/+248
| | | | | | | | | | | | | | | | | | | | | | | | | | | | CSS style changes/enhancments (mainly the new background color for inline code, affects all Ruby code.) Demos and tests adjusted. Plugin: new PluginHost::default method. Scanner: - New setup method - ability to re-use a scanner - ability to keep the tokens - minor changes to token caching and string flattening Encoder: Error if token content is neither String nor Symbol. HTML encoder: - more warnings for unclosed tokens - output now UTF-8 Ruby Scanner: - bug: symbols before => now do not include =; {:foo=>bar} is valid Ruby code - try to close all open tokens - constants now all with specific namespace (for speed, I hope) Styles: new :entity/en class. Test suite now gives hinted HTML output.
* Added HTML scanner!murphy2006-04-045-11/+164
| | | | | | | Added test/html/suite.rb and tolkien.in.html test. Benchmark produces inline line numbers now. Minor changes to Ruby and C scanners. Rakefile: unit tests now in -d mode.
* Ruby Scanner: Capital methods (Kernel#Array etc.) are now highlighted as idents.murphy2006-03-241-2/+2
| | | | | Tests adjusted. highlight.rb updated.
* HTML Encoder: inline numerization corrected.murphy2006-03-231-3/+3
| | | | | | | | New demo: demo/demo_html_list.rb. Some demos adjusted. C Scanner: error messages fixed. Rakefile: test_demos now in debug mode. test/suite.rb is using reusable Scanners now.
* Deleted silly debug print.murphy2006-03-151-1/+1
|
* Ruby scanner fixed - nasty bug in ruby/patterns.rb.murphy2006-03-151-7/+7
| | | | | besetzung.in.rb test added. highlight.rb updated.
* Ruby scanner errors fixed.murphy2006-03-151-4/+4
| | | | | demo_dump and demo_encoder updated for new YAML version. Fixes in output.rb and numerization.rb
* scanners/ruby/patterns.rb:murphy2005-12-111-1/+1
| | | | | | | | Changed recognition of numerics + and - prefixes now belong to the number. tests and demos adjusted. helpers/plugin.rb: Typo in doc.
* Plugin System extended:murphy2005-12-102-1/+1
| | | | | | | | | | - helper method - path names are expanded encoders/html.rb, div.rb, span.rb, ruby.rb: - using new methods (fixes problems with rdoc and test tasks): renamed scanners/ruby/helper.rb to patterns.rb encoders/statistic.rb: sorting made deterministic (for unit testing)
* helpers/plugin.rb: New plugin mapping systemmurphy2005-11-061-0/+10
| | | | | | _maps added; demos adjusted TODO updated
* helpers/word_list.rb:murphy2005-11-052-2/+2
| | | | | | | | new CaseIgnoringWordList class simplified implementation Performance gain should be small. Adjusted scanner.rb and scanners/delphi.rb
* New helper module scheme, step 3murphy2005-11-052-1/+1
|
* New helper module scheme, step 2murphy2005-11-051-213/+0
|
* New helper module scheme, step 1murphy2005-11-051-0/+213
|
* scanners/ruby.rb: changed operator handling = fix and speedup.murphy2005-10-291-19/+23
| | | | Test fixtures adjusted.
* scanners/ruby.rb, ruby_helper.rb: Fixes, changes.murphy2005-10-292-19/+61
|
* demo/demo_server.rb: Debug mode.murphy2005-10-221-1/+2
| | | | | tokens.rb: orig_each alias deleted; using super now. scanners/ruby.rb: experimental Regexp/Fancy string recognition for "method /.../" cases.
* scanners/ruby.rb: fixed string inline codemurphy2005-10-161-4/+3
| | | | | encoders/helpers/html_css.rb and html_helper.rb: added :inline token encoders/helpers/html_output.rb: made wrapped_in an attr_accessor; fixed option handling in numerize!
* Two new encoders: debug and xml.murphy2005-10-042-23/+26
| | | | | | | | | | | | | | | | encoder.rb: new token handling encoders/statistic.rb: using new handling ruby_helper.rb: small improvements ruby.rb: - escapes in subtoken - Float detection changed - some multi-char operators are now scanned as one token - def and module definition handling changed bin/coderay: improved, new interface (still in progress) plugin.rb: more expressive load error message
* Demos updated, rewritten, enhanced, tested.murphy2005-10-011-9/+13
| | | | | | | | | | | | | | | | | Some code cleanups. Bugs fixed, scanner and encoder improved: count.rb: marked Streamable html_css.rb: style for inline numbers html.rb: changed options; :line_numbers_offset is now :line_number_start html_output.rb: offset for inline numbers fixed html.rb: token text no longer changed by gsub! while highlighting (this is even faster!) text.rb, plugin.rb: reindented ruby.rb: eleminated multiple assignments for speed tokens.rb: reindented, Tokens#to_s added, #<< returns self Plugin system: bugs fixed, error messages improved.
* Scanners in alpha state moved to hidden.no author2005-09-263-491/+0
|
* New Repository, initial importno author2005-09-268-0/+1319
View Lorry Controller Status