Skip a UTF-8 byte order mark and end a line at a lone CR
Why
Some Windows editors write a byte order mark, and bsc and bsc -format rejected it byte by byte. A CR without an LF ended a // comment but did not start a new line, so the next statement counted as on the same line. The scanner now starts after a byte order mark and ends a line at a CR not followed by LF, as editors and JavaScript do; code frames show lines the same way, and the formatter keeps the byte order mark.
removedDiagnostic byte order mark
\xEF\xBB\xBFlet x = 1diagnostic
Before
After
Syntax error!A.res:1:1 Not sure what to do with this character: "\239".… the same for "\187" and "\191"none
removedDiagnostic lone CR
let x = 1 // c\rlet y = 2diagnostic
Before
After
Syntax error!B.res:1:10-18 Consecutive statements on a line must be separated by ';' or a newlinenone
modifiedFormat lone CR
let x = 1 // c\rlet y = 2text
Syntax errorrescript
let x = 1 // clet y = 2
Fixes #9108
Review
- Why convert a CR-only file to LF when formatting? The formatter writes CRLF when the first line break is CRLF and LF otherwise (#9116); CR-only files are rare enough not to get their own newline.