[wp-trac] [WordPress Trac] #35022: WP allows Unicode 0x00a0 spaces in editor but shortcode parser can't handle them

[WordPress Trac]

#35022: WP allows Unicode 0x00a0 spaces in editor but shortcode parser can't handle
them
--------------------------+--------------------
 Reporter:  steevithak    |       Owner:
     Type:  defect (bug)  |      Status:  new
 Priority:  normal        |   Milestone:  4.4.3
Component:  Shortcodes    |     Version:  4.4
 Severity:  normal        |  Resolution:
 Keywords:                |     Focuses:
--------------------------+--------------------
Changes (by boonebgorges):

 * keywords:  needs-unit-tests =>
 * milestone:  Awaiting Review => 4.4.3


Comment:

 I bisected the behavior back to [34747], as demonstrated by
 [attachment:35022-test.diff]. By adding `\xA0` to the `do_shortcode()`
 regex pattern as suggested by @steevithak, I was able to get the first
 assertion in my new test to pass.

 Interestingly, it appears that attributes following an A0 non-breaking
 space have never been parsed correctly. So the second assertion in
 [attachment:35022-test.diff] fails. A bit of debugging shows that `$text =
 preg_replace("/[\x{00a0}\x{200b}]+/u", " ", $text);` in
 `shortcode_parse_args()` is resulting in a `PREG_BAD_UTF8_ERROR`. It could
 be that this is a result of my local config?

--
Ticket URL: <https://core.trac.wordpress.org/ticket/35022#comment:3>
WordPress Trac <https://core.trac.wordpress.org/>
WordPress publishing platform