Commit c6a07689 authored by Jelle Zijlstra's avatar Jelle Zijlstra

fstrings: remove addressed TODOs

Docstrings don't support f-strings in cpython, so they don't need to here either. The scanner
seems to be working.

Current status:
- All but three of the tests from CPython's test_fstring pass. Two of the remaining ones rely
  on exec and the third relies on a runtime NameError that occurs at compile time in Cython. These
  tests still need to be added to the Cython test suite.
- I'm not overly convinced that the refcounting is correct in the C code that's currently
  emitted.
- There's some room for optimization, such as removing empty strings from JoinedStrNode,
  concatenating adjacent raw strings at compile time, and using _PyUnicodeWriter.
- Currently the feature is available in both Python 2 and Python 3 mode. It mostly works in
  Python 2, except for the !a conversion char.
parent 6524acb7
......@@ -979,7 +979,7 @@ def p_string_literal(s, kind_override=None):
bytes_value, unicode_value = chars.getstrings()
if is_python3_source and has_non_ascii_literal_characters:
# Python 3 forbids literal non-ASCII characters in byte strings
if kind not in 'uf':
if kind not in ('u', 'f'):
s.error("bytes can only contain ASCII literal characters.",
pos=pos, fatal=False)
bytes_value = None
......@@ -1111,9 +1111,7 @@ def p_f_string_expr(s, unicode_value, pos, starting_index):
name = 'format string expression'
code_source = StringSourceDescriptor(name, expr_str)
buf = StringIO(expr_str)
# scanner = PyrexScanner(buf, code_source, source_encoding = encoding,
# scope = scope, context = context, initial_pos = initial_pos)
scanner = PyrexScanner(buf, code_source, parent_scanner=s, source_encoding=s.source_encoding) # TODO other params
scanner = PyrexScanner(buf, code_source, parent_scanner=s, source_encoding=s.source_encoding)
expr = p_testlist(scanner) # TODO is testlist right here?
# validate the conversion char
......@@ -3451,7 +3449,6 @@ def p_ignorable_statement(s):
def p_doc_string(s):
if s.sy == 'BEGIN_STRING':
pos = s.position()
# TODO: should this support f-strings?
kind, bytes_result, unicode_result = p_cat_string_literal(s)
s.expect_newline("Syntax error in doc string", ignore_semicolon=True)
if kind in ('u', ''):
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment