summaryrefslogtreecommitdiff
path: root/src/cmd/compile/internal/syntax
Commit message (Collapse)AuthorAgeFilesLines
* [dev.inline] cmd/internal/src: replace src.Pos with syntax.PosRobert Griesemer2017-01-098-340/+38
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This replaces the src.Pos LineHist-based position tracking with the syntax.Pos implementation and updates all uses. The LineHist table is not used anymore - the respective code is still there but should be removed eventually. CL forthcoming. Passes toolstash -cmp when comparing to the master repo (with the exception of a couple of swapped assembly instructions, likely due to different instruction scheduling because the line-based sorting has changed; though this is won't affect correctness). The sizes of various important compiler data structures have increased significantly (see the various sizes_test.go files); this is probably the reason for an increase of compilation times (to be addressed). Here are the results of compilebench -count 5, run on a "quiet" machine (no apps running besides a terminal): name old time/op new time/op delta Template 256ms ± 1% 280ms ±15% +9.54% (p=0.008 n=5+5) Unicode 132ms ± 1% 132ms ± 1% ~ (p=0.690 n=5+5) GoTypes 891ms ± 1% 917ms ± 2% +2.88% (p=0.008 n=5+5) Compiler 3.84s ± 2% 3.99s ± 2% +3.95% (p=0.016 n=5+5) MakeBash 47.1s ± 1% 47.2s ± 2% ~ (p=0.841 n=5+5) name old user-ns/op new user-ns/op delta Template 309M ± 1% 326M ± 2% +5.18% (p=0.008 n=5+5) Unicode 165M ± 1% 168M ± 4% ~ (p=0.421 n=5+5) GoTypes 1.14G ± 2% 1.18G ± 1% +3.47% (p=0.008 n=5+5) Compiler 5.00G ± 1% 5.16G ± 1% +3.12% (p=0.008 n=5+5) Change-Id: I241c4246cdff627d7ecb95cac23060b38f9775ec Reviewed-on: https://go-review.googlesource.com/34273 Run-TryBot: Robert Griesemer <gri@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Matthew Dempsky <mdempsky@google.com>
* [dev.inline] cmd/compile/internal/syntax: add predicates to syntax.PosRobert Griesemer2016-12-102-4/+68
| | | | | | | | | | | This moves syntax.Pos closer to cmd/internal/src.Pos so that we can more easily replace src.Pos with syntax.Pos going forward. Change-Id: I9f93a65fecb4c22591edca4b9d6cda39cf0e872e Reviewed-on: https://go-review.googlesource.com/34270 Run-TryBot: Robert Griesemer <gri@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Matthew Dempsky <mdempsky@google.com>
* [dev.inline] cmd/compile/internal/syntax: add tests for //line directivesRobert Griesemer2016-12-093-11/+46
| | | | | | | | Change-Id: I77dc73bfe79e43bbadf85d7eb3c5f8990ec72023 Reviewed-on: https://go-review.googlesource.com/34248 Run-TryBot: Robert Griesemer <gri@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Matthew Dempsky <mdempsky@google.com>
* [dev.inline] cmd/compile/internal/syntax: report byte offset rather then ↵Robert Griesemer2016-12-094-72/+81
| | | | | | | | | | | | | | | rune count for column value This will only become user-visible if error messages show column information. Per the discussion in #10324. For #10324. Change-Id: I5959c1655aba74bb1a22fdc261cd728ffcfa6912 Reviewed-on: https://go-review.googlesource.com/34244 Run-TryBot: Robert Griesemer <gri@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Matthew Dempsky <mdempsky@google.com>
* [dev.inline] cmd/compile/internal/syntax: remove gcCompat uses in scannerRobert Griesemer2016-12-093-32/+25
| | | | | | | | | | | | | - make the scanner unconditionally gc compatible - consistently use "invalid" instead "illegal" in errors Reviewed in and cherry-picked from https://go-review.googlesource.com/#/c/33896/. Change-Id: I4c4253e7392f3311b0d838bbe503576c9469b203 Reviewed-on: https://go-review.googlesource.com/34237 Run-TryBot: Robert Griesemer <gri@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Matthew Dempsky <mdempsky@google.com>
* [dev.inline] cmd/compile/internal/syntax: use syntax.Pos for all external ↵Robert Griesemer2016-12-096-55/+83
| | | | | | | | | | | | | | | | | | | positions - use syntax.Pos in syntax.Error (rather than line, col) - use syntax.Pos in syntax.PragmaHandler (rather than just line) - update uses - better documentation in various places Also: - make Pos methods use Pos receiver (rather than *Pos) Reviewed in and cherry-picked from https://go-review.googlesource.com/#/c/33891/. With minor adjustments to noder.go to make merge compile. Change-Id: I5507cea6c2be46a7677087c1aeb69382d31033eb Reviewed-on: https://go-review.googlesource.com/34236 Reviewed-by: Matthew Dempsky <mdempsky@google.com>
* [dev.inline] cmd/compile/internal/syntax: clean up error and pragma handlingRobert Griesemer2016-12-094-102/+128
| | | | | | | | | | | | | | | | | | | | | | | | | Reviewed in and cherry-picked from https://go-review.googlesource.com/#/c/33873/. - simplify error handling in source.go (move handling of first error into parser, where it belongs) - clean up error handling in scanner.go - move pragma and position base handling from scanner to parser where it belongs - have separate error methods in parser to avoid confusion with handlers from scanner.go and source.go - (source.go) and (scanner.go, source.go, tokens.go) may be stand-alone packages if so desired, which means these files are now less entangled and easier to maintain Change-Id: I81510fc7ef943b78eaa49092c0eab2075a05878c Reviewed-on: https://go-review.googlesource.com/34235 Reviewed-by: Matthew Dempsky <mdempsky@google.com> Run-TryBot: Robert Griesemer <gri@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
* [dev.inline] cmd/compile/internal/syntax: simplified position codeRobert Griesemer2016-12-094-66/+53
| | | | | | | | | | Reviewed in and cherry-picked from https://go-review.googlesource.com/#/c/33805/. Change-Id: I859d9bd5f2256ca78f7b24b330290f7ae600854d Reviewed-on: https://go-review.googlesource.com/34234 Reviewed-by: Matthew Dempsky <mdempsky@google.com> Run-TryBot: Robert Griesemer <gri@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
* [dev.inline] cmd/compile/internal/syntax: process //line pragmas in scannerRobert Griesemer2016-12-097-42/+77
| | | | | | | | | | | | Reviewed in and cherry-picked from https://go-review.googlesource.com/#/c/33764/. Minor adjustment in noder.go to make merge compile again. Change-Id: Ib5029b52b59944f207b0f2438c8a5aa576eb25b8 Reviewed-on: https://go-review.googlesource.com/34233 Run-TryBot: Robert Griesemer <gri@golang.org> Reviewed-by: Matthew Dempsky <mdempsky@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org>
* [dev.inline] cmd/compile/internal/syntax: introduce general position info ↵Robert Griesemer2016-12-099-107/+376
| | | | | | | | | | | for nodes Reviewed in and cherry-picked from https://go-review.googlesource.com/#/c/33758/. Minor adjustments in noder.go to fix merge. Change-Id: Ibe429e327c7f8554f8ac205c61ce3738013aed98 Reviewed-on: https://go-review.googlesource.com/34231 Reviewed-by: Matthew Dempsky <mdempsky@google.com>
* cmd/compile/internal/syntax: remove dead type declarationRobert Griesemer2016-12-021-2/+0
| | | | | | | | Change-Id: I8a3ce0fa513ff943009c5669531132cd23ecf155 Reviewed-on: https://go-review.googlesource.com/33870 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
* cmd/compile/internal/syntax: remove unused node fieldRobert Griesemer2016-11-291-1/+2
| | | | | | | | | | | | The doc field is not yet used - remove it for now (we may end up with a different solution for 1.9). This reduces memory consumption for parsing all of std lib by about 40MB and makes parsing slightly faster. Change-Id: Iafb00b9c7f1be9c66fdfb29096d3da5049b2fcf5 Reviewed-on: https://go-review.googlesource.com/33661 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> Reviewed-by: Matthew Dempsky <mdempsky@google.com>
* cmd/compile: don't panic on syntax error in select statementRobert Griesemer2016-11-291-17/+12
| | | | | | | | | | Fixes #18092. Change-Id: I54e2da2e0f168c068f5e4a1b22ba508d78259168 Reviewed-on: https://go-review.googlesource.com/33658 TryBot-Result: Gobot Gobot <gobot@golang.org> Run-TryBot: Robert Griesemer <gri@golang.org> Reviewed-by: Matthew Dempsky <mdempsky@google.com>
* all: fix vet nitsJosh Bleecher Snyder2016-11-111-1/+1
| | | | | | | | | | | | | | | | Fixes these complaints from vet: cmd/compile/internal/gc/noder.go:32: cmd/compile/internal/syntax.Error composite literal uses unkeyed fields cmd/compile/internal/gc/noder.go:1035: cmd/compile/internal/syntax.Error composite literal uses unkeyed fields cmd/compile/internal/gc/noder.go:1051: cmd/compile/internal/syntax.Error composite literal uses unkeyed fields cmd/compile/internal/syntax/parser_test.go:182: possible formatting directive in Error call net/http/client_test.go:1334: possible formatting directive in Fatal call Change-Id: I5f90ec30f3c106c7e66c92e2b6f8d3b4874fec66 Reviewed-on: https://go-review.googlesource.com/33133 Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
* cmd/compile/internal/syntax: fix error handling for Read/Parse callsRobert Griesemer2016-11-097-77/+106
| | | | | | | | | | | | | | | - define syntax.Error for cleaner error reporting - abort parsing after first error if no error handler is installed - make sure to always report the first error, if any - document behavior of API calls - while at it: rename ReadXXX -> ParseXXX (clearer) - adjust cmd/compile noder.go accordingly Fixes #17774. Change-Id: I7893eedea454a64acd753e32f7a8bf811ddbb03c Reviewed-on: https://go-review.googlesource.com/32950 Reviewed-by: Matthew Dempsky <mdempsky@google.com>
* cmd/go/internal/syntax: reintroduce reverted commentsRobert Griesemer2016-11-051-0/+16
| | | | | | | | | | | | These comments were originally introduced together with the changes for alias declarations, and then reverted when we backed out alias support. Reintroduce them. Change-Id: I3ef2c4f4672d6af8a900f5d73df273edf28d1a14 Reviewed-on: https://go-review.googlesource.com/32826 Reviewed-by: Matthew Dempsky <mdempsky@google.com>
* Revert "cmd/compile/internal/syntax: support for alias declarations"Robert Griesemer2016-11-056-129/+38
| | | | | | | | | | | | This reverts commit 32db3f2756324616b7c856ac9501deccc2491239. Reason: Decision to back out current alias implementation. For #16339. Change-Id: Ib05e3d96041d8347e49cae292f66bec791a1fdc8 Reviewed-on: https://go-review.googlesource.com/32825 Reviewed-by: Matthew Dempsky <mdempsky@google.com>
* cmd/compile/internal/syntax: don't depend on hardwired $GOROOT nameRobert Griesemer2016-11-011-2/+1
| | | | | | | | | | Fixes #17697. Change-Id: I3c47e139b09bde81566e29a1ac0ec8c58d55a34a Reviewed-on: https://go-review.googlesource.com/32539 Run-TryBot: Robert Griesemer <gri@golang.org> Reviewed-by: Matthew Dempsky <mdempsky@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org>
* cmd/compile/internal/syntax: avoid deadlockJosh Bleecher Snyder2016-11-011-2/+3
| | | | | | | | | | | | | | When the err from ReadFile is non-nil, we call t.Fatal(err). Switch t.Fatal to t.Error + return. ensure that close(results) happens on that code path as well. Updates #17697. Change-Id: Ifaacf27a76c175446d642086ff32f4386428080d Reviewed-on: https://go-review.googlesource.com/32486 Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com> Reviewed-by: Robert Griesemer <gri@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
* cmd/compile/internal/syntax: don't panic if no error handler is providedRobert Griesemer2016-11-013-6/+29
| | | | | | | | | | | | If no error handler is provided, terminate parsing with first error and report that error. Fixes #17697. Change-Id: I9070faf7239bd53725de141507912b92ded3474b Reviewed-on: https://go-review.googlesource.com/32456 Run-TryBot: Robert Griesemer <gri@golang.org> Reviewed-by: Matthew Dempsky <mdempsky@google.com>
* cmd/compile, go/parser: disallow "type T = p.T" - must use "=>"Robert Griesemer2016-10-271-2/+1
| | | | | | | | | | | | | I had added this originally so we can play with different notations but it doesn't make sense to keep it around since gofmt will convert a type alias declaration using "=" into one using "=>" anyhow. More importantly, the spec doesn't permit it. Change-Id: Icb010b5a9976aebf877e48b3ce9d7245559ca494 Reviewed-on: https://go-review.googlesource.com/32105 Run-TryBot: Robert Griesemer <gri@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Matthew Dempsky <mdempsky@google.com>
* cmd/compile: simplify parsing of type aliasesRobert Griesemer2016-10-253-14/+8
| | | | | | Change-Id: Ia86841cf84bc17ff6ecc6e5ac4cec86384a0da00 Reviewed-on: https://go-review.googlesource.com/31719 Reviewed-by: Matthew Dempsky <mdempsky@google.com>
* cmd/compile: add go:notinheap type pragmaAustin Clements2016-10-153-6/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This adds a //go:notinheap pragma for declarations of types that must not be heap allocated. We ensure these rules by disallowing new(T), make([]T), append([]T), or implicit allocation of T, by disallowing conversions to notinheap types, and by propagating notinheap to any struct or array that contains notinheap elements. The utility of this pragma is that we can eliminate write barriers for writes to pointers to go:notinheap types, since the write barrier is guaranteed to be a no-op. This will let us mark several scheduler and memory allocator structures as go:notinheap, which will let us disallow write barriers in the scheduler and memory allocator much more thoroughly and also eliminate some problematic hybrid write barriers. This also makes go:nowritebarrierrec and go:yeswritebarrierrec much more powerful. Currently we use go:nowritebarrier all over the place, but it's almost never what you actually want: when write barriers are illegal, they're typically illegal for a whole dynamic scope. Partly this is because go:nowritebarrier has been around longer, but it's also because go:nowritebarrierrec couldn't be used in situations that had no-op write barriers or where some nested scope did allow write barriers. go:notinheap eliminates many no-op write barriers and go:yeswritebarrierrec makes it possible to opt back in to write barriers, so these two changes will let us use go:nowritebarrierrec far more liberally. This updates #13386, which is about controlling pointers from non-GC'd memory to GC'd memory. That would require some additional pragma (or pragmas), but could build on this pragma. Change-Id: I6314f8f4181535dd166887c9ec239977b54940bd Reviewed-on: https://go-review.googlesource.com/30939 Reviewed-by: Keith Randall <khr@golang.org> Reviewed-by: Matthew Dempsky <mdempsky@google.com>
* cmd/compile/internal/syntax: support for alias declarationsRobert Griesemer2016-09-166-39/+137
| | | | | | | | | | | | | | | | | | | | | | | | | Permits parsing of alias declarations with -newparser const/type/var/func T => p.T but the compiler will reject it with an error. For now this also accepts type T = p.T so we can experiment with a type-alias only scenario. - renamed _Arrow token to _Larrow (<-) - introduced _Rarrow token (=>) - introduced AliasDecl node - extended scanner to accept _Rarrow - extended parser and printer to handle alias declarations Change-Id: I0170d10a87df8255db9186d466b6fd405228c38e Reviewed-on: https://go-review.googlesource.com/29355 Run-TryBot: Robert Griesemer <gri@golang.org> Reviewed-by: Matthew Dempsky <mdempsky@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org>
* cmd/compile/internal/gc: make Nod privateDave Cheney2016-09-161-3/+3
| | | | | | | | | Follow up to CL 29134. Generated with gofmt -r 'Nod -> nod', plus three manual adjustments to the comments in syntax/parser.go Change-Id: I02920f7ab10c70b6e850457b42d5fe35f1f3821a Reviewed-on: https://go-review.googlesource.com/29136 Reviewed-by: Matthew Dempsky <mdempsky@google.com>
* cmd/compile/internal/syntax: remove strbyteseqlMatthew Dempsky2016-09-091-13/+1
| | | | | | | | | | | cmd/compile already optimizes "string(b) == s" to avoid allocating a temporary string. Change-Id: I4244fbeae8d350261494135c357f9a6e2ab7ace3 Reviewed-on: https://go-review.googlesource.com/28931 Run-TryBot: Matthew Dempsky <mdempsky@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
* cmd/compile: handle pragmas immediately with -newparser=1Matthew Dempsky2016-08-318-30/+86
| | | | | | | | | | | | | | | | | | | | | Instead of saving all pragmas and processing them after parsing is finished, process them immediately during scanning like the current lexer does. This is a bit unfortunate because it means we can't use syntax.ParseFile to concurrently parse files yet, but it fixes how we report syntax errors in the presence of //line pragmas. While here, add a bunch more gcCompat entries to syntax/parser.go to get "go build -toolexec='toolstash -cmp' std cmd" passing. There are still a few remaining cases only triggered building unit tests, but this seems like a nice checkpoint. Change-Id: Iaf3bbcf2849857a460496f31eea228e0c585ce13 Reviewed-on: https://go-review.googlesource.com/28226 Run-TryBot: Matthew Dempsky <mdempsky@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Robert Griesemer <gri@golang.org>
* net/http, cmd/compile: minor vet fixesJosh Bleecher Snyder2016-08-251-1/+1
| | | | | | | | | | Updates #11041 Change-Id: Ia0151723e3bc0d163cc687a02bfc5e0285d95ffa Reviewed-on: https://go-review.googlesource.com/27810 Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
* cmd/compile/internal/syntax: match old parser errors and line numbersMatthew Dempsky2016-08-193-80/+146
| | | | | | | | | | | | | | | | | | | | | | | | This makes a bunch of changes to package syntax to tweak line numbers for AST nodes. For example, short variable declaration statements are now associated with the location of the ":=" token, and function calls are associated with the location of the final ")" token. These help satisfy many unit tests that assume the old parser's behavior. Because many of these changes are questionable, they're guarded behind a new "gcCompat" const to make them easy to identify and revisit in the future. A handful of remaining tests are too difficult to make behave identically. These have been updated to execute with -newparser=0 and comments explaining why they need to be fixed. all.bash now passes with both the old and new parsers. Change-Id: Iab834b71ca8698d39269f261eb5c92a0d55a3bf4 Reviewed-on: https://go-review.googlesource.com/27199 Run-TryBot: Matthew Dempsky <mdempsky@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Robert Griesemer <gri@golang.org>
* cmd/compile/internal/syntax: expose additional information for gcMatthew Dempsky2016-08-192-7/+23
| | | | | | | | | | | | | | | | gc needs access to line offsets for Nodes. It also needs access to the end line offset for function bodies so it knows what line number to use for things like implicit returns and defer executions. Lastly, include an extra bool to distinguish between simple and full slice expressions. This is redundant in valid parse trees, but needed by gc for producing complete warnings in invalid inputs. Change-Id: I64baf334a35c72336d26fa6755c67eb9d6f4e93c Reviewed-on: https://go-review.googlesource.com/27196 Reviewed-by: Robert Griesemer <gri@golang.org> Run-TryBot: Matthew Dempsky <mdempsky@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org>
* cmd/compile/internal/syntax: fast Go syntax trees, initial commit.Robert Griesemer2016-08-1812-0/+5350
Syntax tree nodes, scanner, parser, basic printers. Builds syntax trees for entire Go std lib at a rate of ~1.8M lines/s in warmed up state (MacMini, 2.3 GHz Intel Core i7, 8GB RAM): $ go test -run StdLib -fast parsed 1074617 lines (2832 files) in 579.66364ms (1853863 lines/s) allocated 282.212Mb (486.854Mb/s) PASS Change-Id: Ie26d9a7bf4e5ff07457aedfcc9b89f0eba72ae3f Reviewed-on: https://go-review.googlesource.com/27195 Run-TryBot: Matthew Dempsky <mdempsky@google.com> Reviewed-by: Robert Griesemer <gri@golang.org>
View Lorry Controller Status