Merge branch 'bug/259' into 'master'

Handle SyntaxErrors in a slightly smarter way

SyntaxErrors are strange and mystical beasts. On top of the problems we
encountered previously in GitLab#237, it's now apparent that
SyntaxErrors can also occur across multiple lines (in fact, across the
rest of a file). In the event of a "multi-line" SyntaxError, we need to
determine what row to report and what the column number is.

For now, we're going to use the row number of the first line and limit
the column number to be less than the end of the line. It may not be
perfect, but it is slightly better.

Related-to #237  
Closes #259 

cc @arcanemagus

See merge request !151

1 files changed, 15 insertions, 16 deletions

diff --git a/src/flake8/checker.py b/src/flake8/checker.py
index cf25117..7f148f4 100644
--- a/src/flake8/checker.py
+++ b/src/flake8/checker.py
@@ -477,23 +477,22 @@ class FileChecker(object):
             # NOTE(sigmavirus24): SyntaxErrors report 1-indexed column
             # numbers. We need to decrement the column number by 1 at
             # least.
-            offset = 1
+            column_offset = 1
+            # See also: https://gitlab.com/pycqa/flake8/issues/237
             physical_line = token[-1]
-            if len(physical_line) == column and physical_line[-1] == '\n':
-                # NOTE(sigmavirus24): By default, we increment the column
-                # value so that it's always 1-indexed. The SyntaxError that
-                # we are trying to handle here will end up being 2 past
-                # the end of the line. This happens because the
-                # SyntaxError is technically the character after the
-                # new-line. For example, if the code is ``foo(\n`` then
-                # ``\n`` will be 4, the empty string will be 5 but most
-                # tools want to report the at column 4, i.e., the opening
-                # parenthesis. Semantically, having a column number of 6 is
-                # correct but not useful for tooling (e.g., editors that
-                # constantly run Flake8 for users).
-                # See also: https://gitlab.com/pycqa/flake8/issues/237
-                offset += 1
-            column -= offset
+
+            # NOTE(sigmavirus24): SyntaxErrors also don't exactly have a
+            # "physical" line so much as what was accumulated by the point
+            # tokenizing failed.
+            # See also: https://gitlab.com/pycqa/flake8/issues/237
+            lines = physical_line.rstrip('\n').split('\n')
+            row_offset = len(lines) - 1
+            logical_line = lines[0]
+            logical_line_length = len(logical_line)
+            if column > logical_line_length:
+                column = logical_line_length
+            row -= row_offset
+            column -= column_offset
         return row, column
 
     def run_ast_checks(self):