Tim Peters again:

The new version (attached) is fast enough all the time in every real module
I have <whew!>.  You can make it slow by, e.g., creating an open list with
5,000 90-character identifiers (+ trailing comma) each on its own line, then
adding an item to the end -- but that still consumes less than a second on
my P5-166.  Response time in real code appears instantaneous.

Fixed some bugs.

New feature:  when hitting ENTER and the cursor is beyond the line's leading
indentation, whitespace is removed on both sides of the cursor; before
whitespace was removed only on the left; e.g., assuming the cursor is
between the comma and the space:

def something(arg1, arg2):
                   ^ cursor to the left of here, and hit ENTER
               arg2):   # new line used to end up here
              arg2):    # but now lines up the way you expect

New hack:  AutoIndent has grown a context_use_ps1 Boolean config option,
defaulting to 0 (false) and set to 1 (only) by PyShell.  Reason:  handling
the fancy stuff requires looking backward for a parsing synch point; ps1
lines are the only sensible thing to look for in a shell window, but are a
bad thing to look for in a file window (ps1 lines show up in my module
docstrings often).  PythonWin's shell should set this true too.

Persistent problem:  strings containing def/class can still screw things up
completely.  No improvement.  Simplest workaround is on the user's head, and
consists of inserting e.g.

def _(): pass

(or any other def/class) after the end of the multiline string that's
screwing them up.  This is especially irksome because IDLE's syntax coloring
is *not* confused, so when this happens the colors don't match the
indentation behavior they see.

Tools/idle/AutoIndent.py

@@ -101,6 +101,10 @@ class AutoIndent:
     indentwidth = 4
     tabwidth = TK_TABWIDTH_DEFAULT
 
+    # If context_use_ps1 is true, parsing searches back for a ps1 line;
+    # else searches back for closest preceding def or class.
+    context_use_ps1 = 0
+
     # When searching backwards for the closest preceding def or class,
     # first start num_context_lines[0] lines back, then
     # num_context_lines[1] lines back if that didn't work, and so on.
@@ -108,11 +112,10 @@ class AutoIndent:
     # conceivable file).
     # Making the initial values larger slows things down more often.
     # OTOH, if you happen to find a line that looks like a def or class
-    # in a multiline string, and the start of the string isn't in the
-    # chunk, the parsing is utterly hosed.  Can't think of a way to
-    # stop that without always reparsing from the start of the file.
-    # doctest.py is a killer example of this (IDLE is useless for
-    # editing that!).
+    # in a multiline string, the parsing is utterly hosed.  Can't think
+    # of a way to stop that without always reparsing from the start
+    # of the file.  doctest.py is a killer example of this (IDLE is
+    # useless for editing that!).
     num_context_lines = 50, 500, 5000000
 
     def __init__(self, editwin):
@@ -126,6 +129,8 @@ class AutoIndent:
                 self.indentwidth = value
             elif key == 'tabwidth':
                 self.tabwidth = value
+            elif key == 'context_use_ps1':
+                self.context_use_ps1 = value
             else:
                 raise KeyError, "bad option name: %s" % `key`
 
@@ -240,13 +245,17 @@ class AutoIndent:
                 text.insert("insert linestart", '\n')
                 return "break"
             indent = line[:i]
-            # strip trailing whitespace
+            # strip whitespace before insert point
             i = 0
             while line and line[-1] in " \t":
                 line = line[:-1]
                 i = i+1
             if i:
                 text.delete("insert - %d chars" % i, "insert")
+            # strip whitespace after insert point
+            while text.get("insert") in " \t":
+                text.delete("insert")
+            # start new line
             text.insert("insert", '\n')
             # adjust indentation for continuations and block open/close
             lno = index2line(text.index('insert'))
@@ -255,7 +264,7 @@ class AutoIndent:
                 startat = max(lno - context, 1)
                 rawtext = text.get(`startat` + ".0", "insert")
                 y.set_str(rawtext)
-                bod = y.find_last_def_or_class()
+                bod = y.find_last_def_or_class(self.context_use_ps1)
                 if bod is not None or startat == 1:
                     break
             y.set_lo(bod or 0)

Tools/idle/PyShell.py

@@ -291,7 +291,7 @@ class PyShell(OutputWindow):
         __builtin__.quit = __builtin__.exit = "To exit, type Ctrl-D."
 
         self.auto = self.extensions["AutoIndent"] # Required extension
-        self.auto.config(usetabs=1, indentwidth=8)
+        self.auto.config(usetabs=1, indentwidth=8, context_use_ps1=1)
 
         text = self.text
         text.configure(wrap="char")