<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN"
"http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head><meta http-equiv="content-type" content="text/html; charset=utf-8" />
<title>[58281] trunk: HTML API: Add custom text decoder.</title>
</head>
<body>
<style type="text/css"><!--
#msg dl.meta { border: 1px #006 solid; background: #369; padding: 6px; color: #fff; }
#msg dl.meta dt { float: left; width: 6em; font-weight: bold; }
#msg dt:after { content:':';}
#msg dl, #msg dt, #msg ul, #msg li, #header, #footer, #logmsg { font-family: verdana,arial,helvetica,sans-serif; font-size: 10pt; }
#msg dl a { font-weight: bold}
#msg dl a:link { color:#fc3; }
#msg dl a:active { color:#ff0; }
#msg dl a:visited { color:#cc6; }
h3 { font-family: verdana,arial,helvetica,sans-serif; font-size: 10pt; font-weight: bold; }
#msg pre { white-space: pre-line; overflow: auto; background: #ffc; border: 1px #fa0 solid; padding: 6px; }
#logmsg { background: #ffc; border: 1px #fa0 solid; padding: 1em 1em 0 1em; }
#logmsg p, #logmsg pre, #logmsg blockquote { margin: 0 0 1em 0; }
#logmsg p, #logmsg li, #logmsg dt, #logmsg dd { line-height: 14pt; }
#logmsg h1, #logmsg h2, #logmsg h3, #logmsg h4, #logmsg h5, #logmsg h6 { margin: .5em 0; }
#logmsg h1:first-child, #logmsg h2:first-child, #logmsg h3:first-child, #logmsg h4:first-child, #logmsg h5:first-child, #logmsg h6:first-child { margin-top: 0; }
#logmsg ul, #logmsg ol { padding: 0; list-style-position: inside; margin: 0 0 0 1em; }
#logmsg ul { text-indent: -1em; padding-left: 1em; }#logmsg ol { text-indent: -1.5em; padding-left: 1.5em; }
#logmsg > ul, #logmsg > ol { margin: 0 0 1em 0; }
#logmsg pre { background: #eee; padding: 1em; }
#logmsg blockquote { border: 1px solid #fa0; border-left-width: 10px; padding: 1em 1em 0 1em; background: white;}
#logmsg dl { margin: 0; }
#logmsg dt { font-weight: bold; }
#logmsg dd { margin: 0; padding: 0 0 0.5em 0; }
#logmsg dd:before { content:'\00bb';}
#logmsg table { border-spacing: 0px; border-collapse: collapse; border-top: 4px solid #fa0; border-bottom: 1px solid #fa0; background: #fff; }
#logmsg table th { text-align: left; font-weight: normal; padding: 0.2em 0.5em; border-top: 1px dotted #fa0; }
#logmsg table td { text-align: right; border-top: 1px dotted #fa0; padding: 0.2em 0.5em; }
#logmsg table thead th { text-align: center; border-bottom: 1px solid #fa0; }
#logmsg table th.Corner { text-align: left; }
#logmsg hr { border: none 0; border-top: 2px dashed #fa0; height: 1px; }
#header, #footer { color: #fff; background: #636; border: 1px #300 solid; padding: 6px; }
#patch { width: 100%; }
#patch h4 {font-family: verdana,arial,helvetica,sans-serif;font-size:10pt;padding:8px;background:#369;color:#fff;margin:0;}
#patch .propset h4, #patch .binary h4 {margin:0;}
#patch pre {padding:0;line-height:1.2em;margin:0;}
#patch .diff {width:100%;background:#eee;padding: 0 0 10px 0;overflow:auto;}
#patch .propset .diff, #patch .binary .diff {padding:10px 0;}
#patch span {display:block;padding:0 10px;}
#patch .modfile, #patch .addfile, #patch .delfile, #patch .propset, #patch .binary, #patch .copfile {border:1px solid #ccc;margin:10px 0;}
#patch ins {background:#dfd;text-decoration:none;display:block;padding:0 10px;}
#patch del {background:#fdd;text-decoration:none;display:block;padding:0 10px;}
#patch .lines, .info {color:#888;background:#fff;}
--></style>
<div id="msg">
<dl class="meta" style="font-size: 105%">
<dt style="float: left; width: 6em; font-weight: bold">Revision</dt> <dd><a style="font-weight: bold" href="https://core.trac.wordpress.org/changeset/58281">58281</a><script type="application/ld+json">{"@context":"http://schema.org","@type":"EmailMessage","description":"Review this Commit","action":{"@type":"ViewAction","url":"https://core.trac.wordpress.org/changeset/58281","name":"Review Commit"}}</script></dd>
<dt style="float: left; width: 6em; font-weight: bold">Author</dt> <dd>dmsnell</dd>
<dt style="float: left; width: 6em; font-weight: bold">Date</dt> <dd>2024-06-02 15:14:35 +0000 (Sun, 02 Jun 2024)</dd>
</dl>
<pre style='padding-left: 1em; margin: 2em 0; border-left: 2px solid #ccc; line-height: 1.25; font-size: 105%; font-family: sans-serif'>HTML API: Add custom text decoder.
Provides a custom decoder for strings coming from HTML attributes and
markup. This custom decoder is necessary because of deficiencies in
PHP's `html_entity_decode()` function:
- It isn't aware of 720 of the possible named character references in
HTML, leaving many out that should be translated.
- It isn't aware of the ambiguous ampersand rule, which allows
conversion of character references in certain contexts when they
are missing their closing `;`.
- It doesn't draw a distinction for the ambiguous ampersand rule
when decoding attribute values instead of markup values.
- Use of `html_entity_decode()` requires manually passing non-default
paramter values to ensure it decodes properly.
This decoder also provides some conveniences, such as making a
single-pass and interruptable decode operation possible. This will
provide a number of opportunities to optimize detection and decoding
of things like value prefixes, and whether a value contains a given
substring.
Developed in https://github.com/WordPress/wordpress-develop/pull/6387
Discussed in https://core.trac.wordpress.org/ticket/61072
Props dmsnell, gziolo, jonsurrell, jorbin, westonruter, zieladam.
Fixes <a href="https://core.trac.wordpress.org/ticket/61072">#61072</a>.</pre>
<h3>Modified Paths</h3>
<ul>
<li><a href="#trunksrcwpincludesclasswptokenmapphp">trunk/src/wp-includes/class-wp-token-map.php</a></li>
<li><a href="#trunksrcwpincludeshtmlapiclasswphtmltagprocessorphp">trunk/src/wp-includes/html-api/class-wp-html-tag-processor.php</a></li>
<li><a href="#trunksrcwpsettingsphp">trunk/src/wp-settings.php</a></li>
<li><a href="#trunktestsphpunittestshtmlapiwpHtmlProcessorHtml5libphp">trunk/tests/phpunit/tests/html-api/wpHtmlProcessorHtml5lib.php</a></li>
<li><a href="#trunktestsphpunittestswptokenmapwpTokenMapphp">trunk/tests/phpunit/tests/wp-token-map/wpTokenMap.php</a></li>
</ul>
<h3>Added Paths</h3>
<ul>
<li><a href="#trunksrcwpincludeshtmlapiclasswphtmldecoderphp">trunk/src/wp-includes/html-api/class-wp-html-decoder.php</a></li>
<li><a href="#trunktestsphpunittestshtmlapiwpHtmlDecoderphp">trunk/tests/phpunit/tests/html-api/wpHtmlDecoder.php</a></li>
</ul>
</div>
<div id="patch">
<h3>Diff</h3>
<a id="trunksrcwpincludesclasswptokenmapphp"></a>
<div class="modfile"><h4 style="background-color: #eee; color: inherit; margin: 1em 0; padding: 1.3em; font-size: 115%">Modified: trunk/src/wp-includes/class-wp-token-map.php</h4>
<pre class="diff"><span>
<span class="info" style="display: block; padding: 0 10px; color: #888">--- trunk/src/wp-includes/class-wp-token-map.php 2024-06-02 09:53:47 UTC (rev 58280)
+++ trunk/src/wp-includes/class-wp-token-map.php 2024-06-02 15:14:35 UTC (rev 58281)
</span><span class="lines" style="display: block; padding: 0 10px; color: #888">@@ -435,8 +435,8 @@
</span><span class="cx" style="display: block; padding: 0 10px"> *
</span><span class="cx" style="display: block; padding: 0 10px"> * @since 6.6.0
</span><span class="cx" style="display: block; padding: 0 10px"> *
</span><del style="background-color: #fdd; text-decoration:none; display:block; padding: 0 10px">- * @param string $word Determine if this word is a lookup key in the map.
- * @param ?string $case_sensitivity 'ascii-case-insensitive' to ignore ASCII case or default of 'case-sensitive'.
</del><ins style="background-color: #dfd; text-decoration:none; display:block; padding: 0 10px">+ * @param string $word Determine if this word is a lookup key in the map.
+ * @param string $case_sensitivity Optional. Pass 'ascii-case-insensitive' to ignore ASCII case when matching. Default 'case-sensitive'.
</ins><span class="cx" style="display: block; padding: 0 10px"> * @return bool Whether there's an entry for the given word in the map.
</span><span class="cx" style="display: block; padding: 0 10px"> */
</span><span class="cx" style="display: block; padding: 0 10px"> public function contains( $word, $case_sensitivity = 'case-sensitive' ) {
</span><span class="lines" style="display: block; padding: 0 10px; color: #888">@@ -521,10 +521,10 @@
</span><span class="cx" style="display: block; padding: 0 10px"> * @since 6.6.0
</span><span class="cx" style="display: block; padding: 0 10px"> *
</span><span class="cx" style="display: block; padding: 0 10px"> * @param string $text String in which to search for a lookup key.
</span><del style="background-color: #fdd; text-decoration:none; display:block; padding: 0 10px">- * @param ?int $offset How many bytes into the string where the lookup key ought to start.
- * @param ?int &$matched_token_byte_length Holds byte-length of found token matched, otherwise not set.
- * @param ?string $case_sensitivity 'ascii-case-insensitive' to ignore ASCII case or default of 'case-sensitive'.
- * @return string|false Mapped value of lookup key if found, otherwise `false`.
</del><ins style="background-color: #dfd; text-decoration:none; display:block; padding: 0 10px">+ * @param int $offset Optional. How many bytes into the string where the lookup key ought to start. Default 0.
+ * @param ?int &$matched_token_byte_length Optional. Holds byte-length of found token matched, otherwise not set. Default null.
+ * @param string $case_sensitivity Optional. Pass 'ascii-case-insensitive' to ignore ASCII case when matching. Default 'case-sensitive'.
+ * @return string|null Mapped value of lookup key if found, otherwise `null`.
</ins><span class="cx" style="display: block; padding: 0 10px"> */
</span><span class="cx" style="display: block; padding: 0 10px"> public function read_token( $text, $offset = 0, &$matched_token_byte_length = null, $case_sensitivity = 'case-sensitive' ) {
</span><span class="cx" style="display: block; padding: 0 10px"> $ignore_case = 'ascii-case-insensitive' === $case_sensitivity;
</span><span class="lines" style="display: block; padding: 0 10px; color: #888">@@ -539,7 +539,7 @@
</span><span class="cx" style="display: block; padding: 0 10px"> // Perhaps a short word then.
</span><span class="cx" style="display: block; padding: 0 10px"> return strlen( $this->small_words ) > 0
</span><span class="cx" style="display: block; padding: 0 10px"> ? $this->read_small_token( $text, $offset, $matched_token_byte_length, $case_sensitivity )
</span><del style="background-color: #fdd; text-decoration:none; display:block; padding: 0 10px">- : false;
</del><ins style="background-color: #dfd; text-decoration:none; display:block; padding: 0 10px">+ : null;
</ins><span class="cx" style="display: block; padding: 0 10px"> }
</span><span class="cx" style="display: block; padding: 0 10px">
</span><span class="cx" style="display: block; padding: 0 10px"> $group = $this->large_words[ $group_at / ( $this->key_length + 1 ) ];
</span><span class="lines" style="display: block; padding: 0 10px; color: #888">@@ -564,7 +564,7 @@
</span><span class="cx" style="display: block; padding: 0 10px"> // Perhaps a short word then.
</span><span class="cx" style="display: block; padding: 0 10px"> return strlen( $this->small_words ) > 0
</span><span class="cx" style="display: block; padding: 0 10px"> ? $this->read_small_token( $text, $offset, $matched_token_byte_length, $case_sensitivity )
</span><del style="background-color: #fdd; text-decoration:none; display:block; padding: 0 10px">- : false;
</del><ins style="background-color: #dfd; text-decoration:none; display:block; padding: 0 10px">+ : null;
</ins><span class="cx" style="display: block; padding: 0 10px"> }
</span><span class="cx" style="display: block; padding: 0 10px">
</span><span class="cx" style="display: block; padding: 0 10px"> /**
</span><span class="lines" style="display: block; padding: 0 10px; color: #888">@@ -572,11 +572,11 @@
</span><span class="cx" style="display: block; padding: 0 10px"> *
</span><span class="cx" style="display: block; padding: 0 10px"> * @since 6.6.0.
</span><span class="cx" style="display: block; padding: 0 10px"> *
</span><del style="background-color: #fdd; text-decoration:none; display:block; padding: 0 10px">- * @param string $text String in which to search for a lookup key.
- * @param ?int $offset How many bytes into the string where the lookup key ought to start.
- * @param ?int &$matched_token_byte_length Holds byte-length of found lookup key if matched, otherwise not set.
- * @param ?string $case_sensitivity 'ascii-case-insensitive' to ignore ASCII case or default of 'case-sensitive'.
- * @return string|false Mapped value of lookup key if found, otherwise `false`.
</del><ins style="background-color: #dfd; text-decoration:none; display:block; padding: 0 10px">+ * @param string $text String in which to search for a lookup key.
+ * @param int $offset Optional. How many bytes into the string where the lookup key ought to start. Default 0.
+ * @param ?int &$matched_token_byte_length Optional. Holds byte-length of found lookup key if matched, otherwise not set. Default null.
+ * @param string $case_sensitivity Optional. Pass 'ascii-case-insensitive' to ignore ASCII case when matching. Default 'case-sensitive'.
+ * @return string|null Mapped value of lookup key if found, otherwise `null`.
</ins><span class="cx" style="display: block; padding: 0 10px"> */
</span><span class="cx" style="display: block; padding: 0 10px"> private function read_small_token( $text, $offset, &$matched_token_byte_length, $case_sensitivity = 'case-sensitive' ) {
</span><span class="cx" style="display: block; padding: 0 10px"> $ignore_case = 'ascii-case-insensitive' === $case_sensitivity;
</span><span class="lines" style="display: block; padding: 0 10px; color: #888">@@ -616,7 +616,7 @@
</span><span class="cx" style="display: block; padding: 0 10px"> return $this->small_mappings[ $at / ( $this->key_length + 1 ) ];
</span><span class="cx" style="display: block; padding: 0 10px"> }
</span><span class="cx" style="display: block; padding: 0 10px">
</span><del style="background-color: #fdd; text-decoration:none; display:block; padding: 0 10px">- return false;
</del><ins style="background-color: #dfd; text-decoration:none; display:block; padding: 0 10px">+ return null;
</ins><span class="cx" style="display: block; padding: 0 10px"> }
</span><span class="cx" style="display: block; padding: 0 10px">
</span><span class="cx" style="display: block; padding: 0 10px"> /**
</span><span class="lines" style="display: block; padding: 0 10px; color: #888">@@ -692,7 +692,7 @@
</span><span class="cx" style="display: block; padding: 0 10px"> *
</span><span class="cx" style="display: block; padding: 0 10px"> * @since 6.6.0
</span><span class="cx" style="display: block; padding: 0 10px"> *
</span><del style="background-color: #fdd; text-decoration:none; display:block; padding: 0 10px">- * @param ?string $indent Use this string for indentation, or rely on the default horizontal tab character.
</del><ins style="background-color: #dfd; text-decoration:none; display:block; padding: 0 10px">+ * @param string $indent Optional. Use this string for indentation, or rely on the default horizontal tab character. Default "\t".
</ins><span class="cx" style="display: block; padding: 0 10px"> * @return string Value which can be pasted into a PHP source file for quick loading of table.
</span><span class="cx" style="display: block; padding: 0 10px"> */
</span><span class="cx" style="display: block; padding: 0 10px"> public function precomputed_php_source_table( $indent = "\t" ) {
</span></span></pre></div>
<a id="trunksrcwpincludeshtmlapiclasswphtmldecoderphp"></a>
<div class="addfile"><h4 style="background-color: #eee; color: inherit; margin: 1em 0; padding: 1.3em; font-size: 115%">Added: trunk/src/wp-includes/html-api/class-wp-html-decoder.php</h4>
<pre class="diff"><span>
<span class="info" style="display: block; padding: 0 10px; color: #888">--- trunk/src/wp-includes/html-api/class-wp-html-decoder.php (rev 0)
+++ trunk/src/wp-includes/html-api/class-wp-html-decoder.php 2024-06-02 15:14:35 UTC (rev 58281)
</span><span class="lines" style="display: block; padding: 0 10px; color: #888">@@ -0,0 +1,461 @@
</span><ins style="background-color: #dfd; text-decoration:none; display:block; padding: 0 10px">+<?php
+
+/**
+ * HTML API: WP_HTML_Decoder class
+ *
+ * Decodes spans of raw text found inside HTML content.
+ *
+ * @package WordPress
+ * @subpackage HTML-API
+ * @since 6.6.0
+ */
+class WP_HTML_Decoder {
+ /**
+ * Indicates if an attribute value starts with a given raw string value.
+ *
+ * Use this method to determine if an attribute value starts with a given string, regardless
+ * of how it might be encoded in HTML. For instance, `http:` could be represented as `http:`
+ * or as `http:` or as `http:` or as `http:`, or in many other ways.
+ *
+ * Example:
+ *
+ * $value = 'http://wordpress.org/';
+ * true === WP_HTML_Decoder::attribute_starts_with( $value, 'http:', 'ascii-case-insensitive' );
+ * false === WP_HTML_Decoder::attribute_starts_with( $value, 'https:', 'ascii-case-insensitive' );
+ *
+ * @since 6.6.0
+ *
+ * @param string $haystack String containing the raw non-decoded attribute value.
+ * @param string $search_text Does the attribute value start with this plain string.
+ * @param string $case_sensitivity Optional. Pass 'ascii-case-insensitive' to ignore ASCII case when matching.
+ * Default 'case-sensitive'.
+ * @return bool Whether the attribute value starts with the given string.
+ */
+ public static function attribute_starts_with( $haystack, $search_text, $case_sensitivity = 'case-sensitive' ) {
+ $search_length = strlen( $search_text );
+ $loose_case = 'ascii-case-insensitive' === $case_sensitivity;
+ $haystack_end = strlen( $haystack );
+ $search_at = 0;
+ $haystack_at = 0;
+
+ while ( $search_at < $search_length && $haystack_at < $haystack_end ) {
+ $chars_match = $loose_case
+ ? strtolower( $haystack[ $haystack_at ] ) === strtolower( $search_text[ $search_at ] )
+ : $haystack[ $haystack_at ] === $search_text[ $search_at ];
+
+ $is_introducer = '&' === $haystack[ $haystack_at ];
+ $next_chunk = $is_introducer
+ ? self::read_character_reference( 'attribute', $haystack, $haystack_at, $token_length )
+ : null;
+
+ // If there's no character reference and the characters don't match, the match fails.
+ if ( null === $next_chunk && ! $chars_match ) {
+ return false;
+ }
+
+ // If there's no character reference but the character do match, then it could still match.
+ if ( null === $next_chunk && $chars_match ) {
+ ++$haystack_at;
+ ++$search_at;
+ continue;
+ }
+
+ // If there is a character reference, then the decoded value must exactly match what follows in the search string.
+ if ( 0 !== substr_compare( $search_text, $next_chunk, $search_at, strlen( $next_chunk ), $loose_case ) ) {
+ return false;
+ }
+
+ // The character reference matched, so continue checking.
+ $haystack_at += $token_length;
+ $search_at += strlen( $next_chunk );
+ }
+
+ return true;
+ }
+
+ /**
+ * Returns a string containing the decoded value of a given HTML text node.
+ *
+ * Text nodes appear in HTML DATA sections, which are the text segments inside
+ * and around tags, excepting SCRIPT and STYLE elements (and some others),
+ * whose inner text is not decoded. Use this function to read the decoded
+ * value of such a text span in an HTML document.
+ *
+ * Example:
+ *
+ * '“😄”' === WP_HTML_Decode::decode_text_node( '“😄”' );
+ *
+ * @since 6.6.0
+ *
+ * @param string $text Text containing raw and non-decoded text node to decode.
+ * @return string Decoded UTF-8 value of given text node.
+ */
+ public static function decode_text_node( $text ) {
+ return static::decode( 'data', $text );
+ }
+
+ /**
+ * Returns a string containing the decoded value of a given HTML attribute.
+ *
+ * Text found inside an HTML attribute has different parsing rules than for
+ * text found inside other markup, or DATA segments. Use this function to
+ * read the decoded value of an HTML string inside a quoted attribute.
+ *
+ * Example:
+ *
+ * '“😄”' === WP_HTML_Decode::decode_attribute( '“😄”' );
+ *
+ * @since 6.6.0
+ *
+ * @param string $text Text containing raw and non-decoded attribute value to decode.
+ * @return string Decoded UTF-8 value of given attribute value.
+ */
+ public static function decode_attribute( $text ) {
+ return static::decode( 'attribute', $text );
+ }
+
+ /**
+ * Decodes a span of HTML text, depending on the context in which it's found.
+ *
+ * This is a low-level method; prefer calling WP_HTML_Decoder::decode_attribute() or
+ * WP_HTML_Decoder::decode_text_node() instead. It's provided for cases where this
+ * may be difficult to do from calling code.
+ *
+ * Example:
+ *
+ * '©' = WP_HTML_Decoder::decode( 'data', '©' );
+ *
+ * @since 6.6.0
+ *
+ * @access private
+ *
+ * @param string $context `attribute` for decoding attribute values, `data` otherwise.
+ * @param string $text Text document containing span of text to decode.
+ * @return string Decoded UTF-8 string.
+ */
+ public static function decode( $context, $text ) {
+ $decoded = '';
+ $end = strlen( $text );
+ $at = 0;
+ $was_at = 0;
+
+ while ( $at < $end ) {
+ $next_character_reference_at = strpos( $text, '&', $at );
+ if ( false === $next_character_reference_at || $next_character_reference_at >= $end ) {
+ break;
+ }
+
+ $character_reference = self::read_character_reference( $context, $text, $next_character_reference_at, $token_length );
+ if ( isset( $character_reference ) ) {
+ $at = $next_character_reference_at;
+ $decoded .= substr( $text, $was_at, $at - $was_at );
+ $decoded .= $character_reference;
+ $at += $token_length;
+ $was_at = $at;
+ continue;
+ }
+
+ ++$at;
+ }
+
+ if ( 0 === $was_at ) {
+ return $text;
+ }
+
+ if ( $was_at < $end ) {
+ $decoded .= substr( $text, $was_at, $end - $was_at );
+ }
+
+ return $decoded;
+ }
+
+ /**
+ * Attempt to read a character reference at the given location in a given string,
+ * depending on the context in which it's found.
+ *
+ * If a character reference is found, this function will return the translated value
+ * that the reference maps to. It will then set `$match_byte_length` the
+ * number of bytes of input it read while consuming the character reference. This
+ * gives calling code the opportunity to advance its cursor when traversing a string
+ * and decoding.
+ *
+ * Example:
+ *
+ * null === WP_HTML_Decoder::read_character_reference( 'attribute', 'Ships…', 0 );
+ * '…' === WP_HTML_Decoder::read_character_reference( 'attribute', 'Ships…', 5, $token_length );
+ * 8 === $token_length; // `…`
+ *
+ * null === WP_HTML_Decoder::read_character_reference( 'attribute', '¬in', 0 );
+ * '∉' === WP_HTML_Decoder::read_character_reference( 'attribute', '∉', 0, $token_length );
+ * 7 === $token_length; // `∉`
+ *
+ * '¬' === WP_HTML_Decoder::read_character_reference( 'data', '¬in', 0, $token_length );
+ * 4 === $token_length; // `¬`
+ * '∉' === WP_HTML_Decoder::read_character_reference( 'data', '∉', 0, $token_length );
+ * 7 === $token_length; // `∉`
+ *
+ * @since 6.6.0
+ *
+ * @param string $context `attribute` for decoding attribute values, `data` otherwise.
+ * @param string $text Text document containing span of text to decode.
+ * @param int $at Optional. Byte offset into text where span begins, defaults to the beginning (0).
+ * @param int &$match_byte_length Optional. Set to byte-length of character reference if provided and if a match
+ * is found, otherwise not set. Default null.
+ * @return string|false Decoded character reference in UTF-8 if found, otherwise `false`.
+ */
+ public static function read_character_reference( $context, $text, $at = 0, &$match_byte_length = null ) {
+ /**
+ * Mappings for HTML5 named character references.
+ *
+ * @var WP_Token_Map $html5_named_character_references
+ */
+ global $html5_named_character_references;
+
+ $length = strlen( $text );
+ if ( $at + 1 >= $length ) {
+ return null;
+ }
+
+ if ( '&' !== $text[ $at ] ) {
+ return null;
+ }
+
+ /*
+ * Numeric character references.
+ *
+ * When truncated, these will encode the code point found by parsing the
+ * digits that are available. For example, when `🅰` is truncated
+ * to `DZ` it will encode `DZ`. It does not:
+ * - know how to parse the original `🅰`.
+ * - fail to parse and return plaintext `DZ`.
+ * - fail to parse and return the replacement character `�`
+ */
+ if ( '#' === $text[ $at + 1 ] ) {
+ if ( $at + 2 >= $length ) {
+ return null;
+ }
+
+ /** Tracks inner parsing within the numeric character reference. */
+ $digits_at = $at + 2;
+
+ if ( 'x' === $text[ $digits_at ] || 'X' === $text[ $digits_at ] ) {
+ $numeric_base = 16;
+ $numeric_digits = '0123456789abcdefABCDEF';
+ $max_digits = 6; // 
+ ++$digits_at;
+ } else {
+ $numeric_base = 10;
+ $numeric_digits = '0123456789';
+ $max_digits = 7; // 
+ }
+
+ // Cannot encode invalid Unicode code points. Max is to U+10FFFF.
+ $zero_count = strspn( $text, '0', $digits_at );
+ $digit_count = strspn( $text, $numeric_digits, $digits_at + $zero_count );
+ $after_digits = $digits_at + $zero_count + $digit_count;
+ $has_semicolon = $after_digits < $length && ';' === $text[ $after_digits ];
+ $end_of_span = $has_semicolon ? $after_digits + 1 : $after_digits;
+
+ // `&#` or `&#x` without digits returns into plaintext.
+ if ( 0 === $digit_count && 0 === $zero_count ) {
+ return null;
+ }
+
+ // Whereas `&#` and only zeros is invalid.
+ if ( 0 === $digit_count ) {
+ $match_byte_length = $end_of_span - $at;
+ return '�';
+ }
+
+ // If there are too many digits then it's not worth parsing. It's invalid.
+ if ( $digit_count > $max_digits ) {
+ $match_byte_length = $end_of_span - $at;
+ return '�';
+ }
+
+ $digits = substr( $text, $digits_at + $zero_count, $digit_count );
+ $code_point = intval( $digits, $numeric_base );
+
+ /*
+ * Noncharacters, 0x0D, and non-ASCII-whitespace control characters.
+ *
+ * > A noncharacter is a code point that is in the range U+FDD0 to U+FDEF,
+ * > inclusive, or U+FFFE, U+FFFF, U+1FFFE, U+1FFFF, U+2FFFE, U+2FFFF,
+ * > U+3FFFE, U+3FFFF, U+4FFFE, U+4FFFF, U+5FFFE, U+5FFFF, U+6FFFE,
+ * > U+6FFFF, U+7FFFE, U+7FFFF, U+8FFFE, U+8FFFF, U+9FFFE, U+9FFFF,
+ * > U+AFFFE, U+AFFFF, U+BFFFE, U+BFFFF, U+CFFFE, U+CFFFF, U+DFFFE,
+ * > U+DFFFF, U+EFFFE, U+EFFFF, U+FFFFE, U+FFFFF, U+10FFFE, or U+10FFFF.
+ *
+ * A C0 control is a code point that is in the range of U+00 to U+1F,
+ * but ASCII whitespace includes U+09, U+0A, U+0C, and U+0D.
+ *
+ * These characters are invalid but still decode as any valid character.
+ * This comment is here to note and explain why there's no check to
+ * remove these characters or replace them.
+ *
+ * @see https://infra.spec.whatwg.org/#noncharacter
+ */
+
+ /*
+ * Code points in the C1 controls area need to be remapped as if they
+ * were stored in Windows-1252. Note! This transformation only happens
+ * for numeric character references. The raw code points in the byte
+ * stream are not translated.
+ *
+ * > If the number is one of the numbers in the first column of
+ * > the following table, then find the row with that number in
+ * > the first column, and set the character reference code to
+ * > the number in the second column of that row.
+ */
+ if ( $code_point >= 0x80 && $code_point <= 0x9F ) {
+ $windows_1252_mapping = array(
+ 0x20AC, // 0x80 -> EURO SIGN (€).
+ 0x81, // 0x81 -> (no change).
+ 0x201A, // 0x82 -> SINGLE LOW-9 QUOTATION MARK (‚).
+ 0x0192, // 0x83 -> LATIN SMALL LETTER F WITH HOOK (ƒ).
+ 0x201E, // 0x84 -> DOUBLE LOW-9 QUOTATION MARK („).
+ 0x2026, // 0x85 -> HORIZONTAL ELLIPSIS (…).
+ 0x2020, // 0x86 -> DAGGER (†).
+ 0x2021, // 0x87 -> DOUBLE DAGGER (‡).
+ 0x02C6, // 0x88 -> MODIFIER LETTER CIRCUMFLEX ACCENT (ˆ).
+ 0x2030, // 0x89 -> PER MILLE SIGN (‰).
+ 0x0160, // 0x8A -> LATIN CAPITAL LETTER S WITH CARON (Š).
+ 0x2039, // 0x8B -> SINGLE LEFT-POINTING ANGLE QUOTATION MARK (‹).
+ 0x0152, // 0x8C -> LATIN CAPITAL LIGATURE OE (Œ).
+ 0x8D, // 0x8D -> (no change).
+ 0x017D, // 0x8E -> LATIN CAPITAL LETTER Z WITH CARON (Ž).
+ 0x8F, // 0x8F -> (no change).
+ 0x90, // 0x90 -> (no change).
+ 0x2018, // 0x91 -> LEFT SINGLE QUOTATION MARK (‘).
+ 0x2019, // 0x92 -> RIGHT SINGLE QUOTATION MARK (’).
+ 0x201C, // 0x93 -> LEFT DOUBLE QUOTATION MARK (“).
+ 0x201D, // 0x94 -> RIGHT DOUBLE QUOTATION MARK (”).
+ 0x2022, // 0x95 -> BULLET (•).
+ 0x2013, // 0x96 -> EN DASH (–).
+ 0x2014, // 0x97 -> EM DASH (—).
+ 0x02DC, // 0x98 -> SMALL TILDE (˜).
+ 0x2122, // 0x99 -> TRADE MARK SIGN (™).
+ 0x0161, // 0x9A -> LATIN SMALL LETTER S WITH CARON (š).
+ 0x203A, // 0x9B -> SINGLE RIGHT-POINTING ANGLE QUOTATION MARK (›).
+ 0x0153, // 0x9C -> LATIN SMALL LIGATURE OE (œ).
+ 0x9D, // 0x9D -> (no change).
+ 0x017E, // 0x9E -> LATIN SMALL LETTER Z WITH CARON (ž).
+ 0x0178, // 0x9F -> LATIN CAPITAL LETTER Y WITH DIAERESIS (Ÿ).
+ );
+
+ $code_point = $windows_1252_mapping[ $code_point - 0x80 ];
+ }
+
+ $match_byte_length = $end_of_span - $at;
+ return self::code_point_to_utf8_bytes( $code_point );
+ }
+
+ /** Tracks inner parsing within the named character reference. */
+ $name_at = $at + 1;
+ // Minimum named character reference is two characters. E.g. `GT`.
+ if ( $name_at + 2 > $length ) {
+ return null;
+ }
+
+ $name_length = 0;
+ $replacement = $html5_named_character_references->read_token( $text, $name_at, $name_length );
+ if ( false === $replacement ) {
+ return null;
+ }
+
+ $after_name = $name_at + $name_length;
+
+ // If the match ended with a semicolon then it should always be decoded.
+ if ( ';' === $text[ $name_at + $name_length - 1 ] ) {
+ $match_byte_length = $after_name - $at;
+ return $replacement;
+ }
+
+ /*
+ * At this point though there's a match for an entry in the named
+ * character reference table but the match doesn't end in `;`.
+ * It may be allowed if it's followed by something unambiguous.
+ */
+ $ambiguous_follower = (
+ $after_name < $length &&
+ $name_at < $length &&
+ (
+ ctype_alnum( $text[ $after_name ] ) ||
+ '=' === $text[ $after_name ]
+ )
+ );
+
+ // It's non-ambiguous, safe to leave it in.
+ if ( ! $ambiguous_follower ) {
+ $match_byte_length = $after_name - $at;
+ return $replacement;
+ }
+
+ // It's ambiguous, which isn't allowed inside attributes.
+ if ( 'attribute' === $context ) {
+ return null;
+ }
+
+ $match_byte_length = $after_name - $at;
+ return $replacement;
+ }
+
+ /**
+ * Encode a code point number into the UTF-8 encoding.
+ *
+ * This encoder implements the UTF-8 encoding algorithm for converting
+ * a code point into a byte sequence. If it receives an invalid code
+ * point it will return the Unicode Replacement Character U+FFFD `�`.
+ *
+ * Example:
+ *
+ * '🅰' === WP_HTML_Decoder::code_point_to_utf8_bytes( 0x1f170 );
+ *
+ * // Half of a surrogate pair is an invalid code point.
+ * '�' === WP_HTML_Decoder::code_point_to_utf8_bytes( 0xd83c );
+ *
+ * @since 6.6.0
+ *
+ * @see https://www.rfc-editor.org/rfc/rfc3629 For the UTF-8 standard.
+ *
+ * @param int $code_point Which code point to convert.
+ * @return string Converted code point, or `�` if invalid.
+ */
+ public static function code_point_to_utf8_bytes( $code_point ) {
+ // Pre-check to ensure a valid code point.
+ if (
+ $code_point <= 0 ||
+ ( $code_point >= 0xD800 && $code_point <= 0xDFFF ) ||
+ $code_point > 0x10FFFF
+ ) {
+ return '�';
+ }
+
+ if ( $code_point <= 0x7F ) {
+ return chr( $code_point );
+ }
+
+ if ( $code_point <= 0x7FF ) {
+ $byte1 = ( $code_point >> 6 ) | 0xC0;
+ $byte2 = $code_point & 0x3F | 0x80;
+
+ return pack( 'CC', $byte1, $byte2 );
+ }
+
+ if ( $code_point <= 0xFFFF ) {
+ $byte1 = ( $code_point >> 12 ) | 0xE0;
+ $byte2 = ( $code_point >> 6 ) & 0x3F | 0x80;
+ $byte3 = $code_point & 0x3F | 0x80;
+
+ return pack( 'CCC', $byte1, $byte2, $byte3 );
+ }
+
+ // Any values above U+10FFFF are eliminated above in the pre-check.
+ $byte1 = ( $code_point >> 18 ) | 0xF0;
+ $byte2 = ( $code_point >> 12 ) & 0x3F | 0x80;
+ $byte3 = ( $code_point >> 6 ) & 0x3F | 0x80;
+ $byte4 = $code_point & 0x3F | 0x80;
+
+ return pack( 'CCCC', $byte1, $byte2, $byte3, $byte4 );
+ }
+}
</ins><span class="cx" style="display: block; padding: 0 10px">Property changes on: trunk/src/wp-includes/html-api/class-wp-html-decoder.php
</span><span class="cx" style="display: block; padding: 0 10px">___________________________________________________________________
</span></span></pre></div>
<a id="svneolstyle"></a>
<div class="addfile"><h4 style="background-color: #eee; color: inherit; margin: 1em 0; padding: 1.3em; font-size: 115%">Added: svn:eol-style</h4></div>
<ins style="background-color: #dfd; text-decoration:none; display:block; padding: 0 10px">+native
</ins><span class="cx" style="display: block; padding: 0 10px">\ No newline at end of property
</span><a id="trunksrcwpincludeshtmlapiclasswphtmltagprocessorphp"></a>
<div class="modfile"><h4 style="background-color: #eee; color: inherit; margin: 1em 0; padding: 1.3em; font-size: 115%">Modified: trunk/src/wp-includes/html-api/class-wp-html-tag-processor.php</h4>
<pre class="diff"><span>
<span class="info" style="display: block; padding: 0 10px; color: #888">--- trunk/src/wp-includes/html-api/class-wp-html-tag-processor.php 2024-06-02 09:53:47 UTC (rev 58280)
+++ trunk/src/wp-includes/html-api/class-wp-html-tag-processor.php 2024-06-02 15:14:35 UTC (rev 58281)
</span><span class="lines" style="display: block; padding: 0 10px; color: #888">@@ -15,10 +15,6 @@
</span><span class="cx" style="display: block; padding: 0 10px"> * - Prune the whitespace when removing classes/attributes: e.g. "a b c" -> "c" not " c".
</span><span class="cx" style="display: block; padding: 0 10px"> * This would increase the size of the changes for some operations but leave more
</span><span class="cx" style="display: block; padding: 0 10px"> * natural-looking output HTML.
</span><del style="background-color: #fdd; text-decoration:none; display:block; padding: 0 10px">- * - Properly decode HTML character references in `get_attribute()`. PHP's
- * `html_entity_decode()` is wrong in a couple ways: it doesn't account for the
- * no-ambiguous-ampersand rule, and it improperly handles the way semicolons may
- * or may not terminate a character reference.
</del><span class="cx" style="display: block; padding: 0 10px"> *
</span><span class="cx" style="display: block; padding: 0 10px"> * @package WordPress
</span><span class="cx" style="display: block; padding: 0 10px"> * @subpackage HTML-API
</span><span class="lines" style="display: block; padding: 0 10px; color: #888">@@ -2499,7 +2495,7 @@
</span><span class="cx" style="display: block; padding: 0 10px"> * 3. Double-quoting ends at the last character in the update.
</span><span class="cx" style="display: block; padding: 0 10px"> */
</span><span class="cx" style="display: block; padding: 0 10px"> $enqueued_value = substr( $enqueued_text, $equals_at + 2, -1 );
</span><del style="background-color: #fdd; text-decoration:none; display:block; padding: 0 10px">- return html_entity_decode( $enqueued_value );
</del><ins style="background-color: #dfd; text-decoration:none; display:block; padding: 0 10px">+ return WP_HTML_Decoder::decode_attribute( $enqueued_value );
</ins><span class="cx" style="display: block; padding: 0 10px"> }
</span><span class="cx" style="display: block; padding: 0 10px">
</span><span class="cx" style="display: block; padding: 0 10px"> /**
</span><span class="lines" style="display: block; padding: 0 10px; color: #888">@@ -2572,7 +2568,7 @@
</span><span class="cx" style="display: block; padding: 0 10px">
</span><span class="cx" style="display: block; padding: 0 10px"> $raw_value = substr( $this->html, $attribute->value_starts_at, $attribute->value_length );
</span><span class="cx" style="display: block; padding: 0 10px">
</span><del style="background-color: #fdd; text-decoration:none; display:block; padding: 0 10px">- return html_entity_decode( $raw_value );
</del><ins style="background-color: #dfd; text-decoration:none; display:block; padding: 0 10px">+ return WP_HTML_Decoder::decode_attribute( $raw_value );
</ins><span class="cx" style="display: block; padding: 0 10px"> }
</span><span class="cx" style="display: block; padding: 0 10px">
</span><span class="cx" style="display: block; padding: 0 10px"> /**
</span><span class="lines" style="display: block; padding: 0 10px; color: #888">@@ -2872,7 +2868,7 @@
</span><span class="cx" style="display: block; padding: 0 10px"> return $text;
</span><span class="cx" style="display: block; padding: 0 10px"> }
</span><span class="cx" style="display: block; padding: 0 10px">
</span><del style="background-color: #fdd; text-decoration:none; display:block; padding: 0 10px">- $decoded = html_entity_decode( $text, ENT_QUOTES | ENT_HTML5 | ENT_SUBSTITUTE );
</del><ins style="background-color: #dfd; text-decoration:none; display:block; padding: 0 10px">+ $decoded = WP_HTML_Decoder::decode_text_node( $text );
</ins><span class="cx" style="display: block; padding: 0 10px">
</span><span class="cx" style="display: block; padding: 0 10px"> /*
</span><span class="cx" style="display: block; padding: 0 10px"> * TEXTAREA skips a leading newline, but this newline may appear not only as the
</span></span></pre></div>
<a id="trunksrcwpsettingsphp"></a>
<div class="modfile"><h4 style="background-color: #eee; color: inherit; margin: 1em 0; padding: 1.3em; font-size: 115%">Modified: trunk/src/wp-settings.php</h4>
<pre class="diff"><span>
<span class="info" style="display: block; padding: 0 10px; color: #888">--- trunk/src/wp-settings.php 2024-06-02 09:53:47 UTC (rev 58280)
+++ trunk/src/wp-settings.php 2024-06-02 15:14:35 UTC (rev 58281)
</span><span class="lines" style="display: block; padding: 0 10px; color: #888">@@ -253,6 +253,7 @@
</span><span class="cx" style="display: block; padding: 0 10px"> require ABSPATH . WPINC . '/html-api/class-wp-html-attribute-token.php';
</span><span class="cx" style="display: block; padding: 0 10px"> require ABSPATH . WPINC . '/html-api/class-wp-html-span.php';
</span><span class="cx" style="display: block; padding: 0 10px"> require ABSPATH . WPINC . '/html-api/class-wp-html-text-replacement.php';
</span><ins style="background-color: #dfd; text-decoration:none; display:block; padding: 0 10px">+require ABSPATH . WPINC . '/html-api/class-wp-html-decoder.php';
</ins><span class="cx" style="display: block; padding: 0 10px"> require ABSPATH . WPINC . '/html-api/class-wp-html-tag-processor.php';
</span><span class="cx" style="display: block; padding: 0 10px"> require ABSPATH . WPINC . '/html-api/class-wp-html-unsupported-exception.php';
</span><span class="cx" style="display: block; padding: 0 10px"> require ABSPATH . WPINC . '/html-api/class-wp-html-active-formatting-elements.php';
</span></span></pre></div>
<a id="trunktestsphpunittestshtmlapiwpHtmlDecoderphp"></a>
<div class="addfile"><h4 style="background-color: #eee; color: inherit; margin: 1em 0; padding: 1.3em; font-size: 115%">Added: trunk/tests/phpunit/tests/html-api/wpHtmlDecoder.php</h4>
<pre class="diff"><span>
<span class="info" style="display: block; padding: 0 10px; color: #888">--- trunk/tests/phpunit/tests/html-api/wpHtmlDecoder.php (rev 0)
+++ trunk/tests/phpunit/tests/html-api/wpHtmlDecoder.php 2024-06-02 15:14:35 UTC (rev 58281)
</span><span class="lines" style="display: block; padding: 0 10px; color: #888">@@ -0,0 +1,141 @@
</span><ins style="background-color: #dfd; text-decoration:none; display:block; padding: 0 10px">+<?php
+/**
+ * Unit tests covering WP_HTML_Decoder functionality.
+ *
+ * @package WordPress
+ * @subpackage HTML-API
+ */
+
+/**
+ * @group html-api
+ *
+ * @coversDefaultClass WP_HTML_Decoder
+ */
+class Tests_HtmlApi_WpHtmlDecoder extends WP_UnitTestCase {
+ /**
+ * Ensures proper decoding of edge cases.
+ *
+ * @ticket 61072
+ *
+ * @dataProvider data_edge_cases
+ *
+ * @param $raw_text_node Raw input text.
+ * @param $decoded_value The expected decoded text result.
+ */
+ public function test_edge_cases( $raw_text_node, $decoded_value ) {
+ $this->assertSame(
+ $decoded_value,
+ WP_HTML_Decoder::decode_text_node( $raw_text_node ),
+ 'Improperly decoded raw text node.'
+ );
+ }
+
+ public static function data_edge_cases() {
+ return array(
+ 'Single ampersand' => array( '&', '&' ),
+ );
+ }
+
+ /**
+ * Ensures proper detection of attribute prefixes ignoring ASCII case.
+ *
+ * @ticket 61072
+ *
+ * @dataProvider data_case_variants_of_attribute_prefixes
+ *
+ * @param string $attribute_value Raw attribute value from HTML string.
+ * @param string $search_string Prefix contained in encoded attribute value.
+ */
+ public function test_detects_ascii_case_insensitive_attribute_prefixes( $attribute_value, $search_string ) {
+ $this->assertTrue(
+ WP_HTML_Decoder::attribute_starts_with( $attribute_value, $search_string, 'ascii-case-insensitive' ),
+ "Should have found that '{$attribute_value}' starts with '{$search_string}'"
+ );
+ }
+
+ /**
+ * Data provider.
+ *
+ * @return Generator.
+ */
+ public static function data_case_variants_of_attribute_prefixes() {
+ $with_javascript_prefix = array(
+ 'javascript:',
+ 'JAVASCRIPT:',
+ 'javascript:',
+ 'javascript:',
+ 'javascript:',
+ 'javascript:',
+ 'javascript:alert(1)',
+ 'JaVaScRiPt:alert(1)',
+ 'javascript:alert(1);',
+ 'javascript:alert(1);',
+ 'javascript:alert(1);',
+ 'javascript:alert(1);',
+ 'javascript:alert(1);',
+ 'javascript:alert(1);',
+ 'javascript:alert(1);',
+ 'javascript:alert(1);',
+ 'javascript:alert(1);',
+ 'javascript:alert('XSS')',
+ 'javascript:javascript:alert(1);',
+ 'javascript:javascript:alert(1);',
+ 'javascript:javascript:alert(1);',
+ 'javascript:javascript:alert(1);',
+ 'javascript:javascript:alert(1);',
+ 'javascript:alert(1)//?:',
+ 'javascript:alert(1)',
+ 'javascript:x=1;alert(1)',
+ );
+
+ foreach ( $with_javascript_prefix as $attribute_value ) {
+ yield $attribute_value => array( $attribute_value, 'javascript:' );
+ }
+ }
+
+ /**
+ * Ensures that `attribute_starts_with` respects the case sensitivity argument.
+ *
+ * @ticket 61072
+ *
+ * @dataProvider data_attributes_with_prefix_and_case_sensitive_match
+ *
+ * @param string $attribute_value Raw attribute value from HTML string.
+ * @param string $search_string Prefix contained or not contained in encoded attribute value.
+ * @param string $case_sensitivity Whether to search with ASCII case sensitivity;
+ * 'ascii-case-insensitive' or 'case-sensitive'.
+ * @param bool $is_match Whether the search string is a prefix for the attribute value,
+ * given the case sensitivity setting.
+ */
+ public function test_attribute_starts_with_heeds_case_sensitivity( $attribute_value, $search_string, $case_sensitivity, $is_match ) {
+ if ( $is_match ) {
+ $this->assertTrue(
+ WP_HTML_Decoder::attribute_starts_with( $attribute_value, $search_string, $case_sensitivity ),
+ 'Should have found attribute prefix with case-sensitive search.'
+ );
+ } else {
+ $this->assertFalse(
+ WP_HTML_Decoder::attribute_starts_with( $attribute_value, $search_string, $case_sensitivity ),
+ 'Should not have matched attribute with prefix with ASCII-case-insensitive search.'
+ );
+ }
+ }
+
+ /**
+ * Data provider.
+ *
+ * @return array[].
+ */
+ public static function data_attributes_with_prefix_and_case_sensitive_match() {
+ return array(
+ array( 'http://wordpress.org', 'http', 'case-sensitive', true ),
+ array( 'http://wordpress.org', 'http', 'ascii-case-insensitive', true ),
+ array( 'http://wordpress.org', 'HTTP', 'case-sensitive', false ),
+ array( 'http://wordpress.org', 'HTTP', 'ascii-case-insensitive', true ),
+ array( 'http://wordpress.org', 'Http', 'case-sensitive', false ),
+ array( 'http://wordpress.org', 'Http', 'ascii-case-insensitive', true ),
+ array( 'http://wordpress.org', 'https', 'case-sensitive', false ),
+ array( 'http://wordpress.org', 'https', 'ascii-case-insensitive', false ),
+ );
+ }
+}
</ins><span class="cx" style="display: block; padding: 0 10px">Property changes on: trunk/tests/phpunit/tests/html-api/wpHtmlDecoder.php
</span><span class="cx" style="display: block; padding: 0 10px">___________________________________________________________________
</span></span></pre></div>
<a id="svneolstyle"></a>
<div class="addfile"><h4 style="background-color: #eee; color: inherit; margin: 1em 0; padding: 1.3em; font-size: 115%">Added: svn:eol-style</h4></div>
<ins style="background-color: #dfd; text-decoration:none; display:block; padding: 0 10px">+native
</ins><span class="cx" style="display: block; padding: 0 10px">\ No newline at end of property
</span><a id="trunktestsphpunittestshtmlapiwpHtmlProcessorHtml5libphp"></a>
<div class="modfile"><h4 style="background-color: #eee; color: inherit; margin: 1em 0; padding: 1.3em; font-size: 115%">Modified: trunk/tests/phpunit/tests/html-api/wpHtmlProcessorHtml5lib.php</h4>
<pre class="diff"><span>
<span class="info" style="display: block; padding: 0 10px; color: #888">--- trunk/tests/phpunit/tests/html-api/wpHtmlProcessorHtml5lib.php 2024-06-02 09:53:47 UTC (rev 58280)
+++ trunk/tests/phpunit/tests/html-api/wpHtmlProcessorHtml5lib.php 2024-06-02 15:14:35 UTC (rev 58281)
</span><span class="lines" style="display: block; padding: 0 10px; color: #888">@@ -31,41 +31,32 @@
</span><span class="cx" style="display: block; padding: 0 10px"> * Skip specific tests that may not be supported or have known issues.
</span><span class="cx" style="display: block; padding: 0 10px"> */
</span><span class="cx" style="display: block; padding: 0 10px"> const SKIP_TESTS = array(
</span><del style="background-color: #fdd; text-decoration:none; display:block; padding: 0 10px">- 'adoption01/line0046' => 'Unimplemented: Reconstruction of active formatting elements.',
- 'adoption01/line0159' => 'Unimplemented: Reconstruction of active formatting elements.',
- 'adoption01/line0318' => 'Unimplemented: Reconstruction of active formatting elements.',
- 'entities02/line0100' => 'Encoded characters without semicolon termination in attribute values are not handled properly',
- 'entities02/line0114' => 'Encoded characters without semicolon termination in attribute values are not handled properly',
- 'entities02/line0128' => 'Encoded characters without semicolon termination in attribute values are not handled properly',
- 'entities02/line0142' => 'Encoded characters without semicolon termination in attribute values are not handled properly',
- 'entities02/line0156' => 'Encoded characters without semicolon termination in attribute values are not handled properly',
- 'inbody01/line0001' => 'Bug.',
- 'inbody01/line0014' => 'Bug.',
- 'inbody01/line0029' => 'Bug.',
- 'menuitem-element/line0012' => 'Bug.',
- 'plain-text-unsafe/line0001' => 'HTML entities may be mishandled.',
- 'plain-text-unsafe/line0105' => 'Binary.',
- 'tests1/line0342' => "Closing P tag implicitly creates opener, which we don't visit.",
- 'tests1/line0720' => 'Unimplemented: Reconstruction of active formatting elements.',
- 'tests1/line0833' => 'Bug.',
- 'tests15/line0001' => 'Unimplemented: Reconstruction of active formatting elements.',
- 'tests15/line0022' => 'Unimplemented: Reconstruction of active formatting elements.',
- 'tests2/line0317' => 'HTML entities may be mishandled.',
- 'tests2/line0408' => 'HTML entities may be mishandled.',
- 'tests2/line0650' => 'Whitespace only test never enters "in body" parsing mode.',
- 'tests20/line0497' => "Closing P tag implicitly creates opener, which we don't visit.",
- 'tests23/line0001' => 'Unimplemented: Reconstruction of active formatting elements.',
- 'tests23/line0041' => 'Unimplemented: Reconstruction of active formatting elements.',
- 'tests23/line0069' => 'Unimplemented: Reconstruction of active formatting elements.',
- 'tests23/line0101' => 'Unimplemented: Reconstruction of active formatting elements.',
- 'tests25/line0169' => 'Bug.',
- 'tests26/line0263' => 'Bug: An active formatting element should be created for a trailing text node.',
- 'tests7/line0354' => 'Bug.',
- 'tests8/line0001' => 'Bug.',
- 'tests8/line0020' => 'Bug.',
- 'tests8/line0037' => 'Bug.',
- 'tests8/line0052' => 'Bug.',
- 'webkit01/line0174' => 'Bug.',
</del><ins style="background-color: #dfd; text-decoration:none; display:block; padding: 0 10px">+ 'adoption01/line0046' => 'Unimplemented: Reconstruction of active formatting elements.',
+ 'adoption01/line0159' => 'Unimplemented: Reconstruction of active formatting elements.',
+ 'adoption01/line0318' => 'Unimplemented: Reconstruction of active formatting elements.',
+ 'inbody01/line0001' => 'Bug.',
+ 'inbody01/line0014' => 'Bug.',
+ 'inbody01/line0029' => 'Bug.',
+ 'menuitem-element/line0012' => 'Bug.',
+ 'tests1/line0342' => "Closing P tag implicitly creates opener, which we don't visit.",
+ 'tests1/line0720' => 'Unimplemented: Reconstruction of active formatting elements.',
+ 'tests1/line0833' => 'Bug.',
+ 'tests15/line0001' => 'Unimplemented: Reconstruction of active formatting elements.',
+ 'tests15/line0022' => 'Unimplemented: Reconstruction of active formatting elements.',
+ 'tests2/line0650' => 'Whitespace only test never enters "in body" parsing mode.',
+ 'tests20/line0497' => "Closing P tag implicitly creates opener, which we don't visit.",
+ 'tests23/line0001' => 'Unimplemented: Reconstruction of active formatting elements.',
+ 'tests23/line0041' => 'Unimplemented: Reconstruction of active formatting elements.',
+ 'tests23/line0069' => 'Unimplemented: Reconstruction of active formatting elements.',
+ 'tests23/line0101' => 'Unimplemented: Reconstruction of active formatting elements.',
+ 'tests25/line0169' => 'Bug.',
+ 'tests26/line0263' => 'Bug: An active formatting element should be created for a trailing text node.',
+ 'tests7/line0354' => 'Bug.',
+ 'tests8/line0001' => 'Bug.',
+ 'tests8/line0020' => 'Bug.',
+ 'tests8/line0037' => 'Bug.',
+ 'tests8/line0052' => 'Bug.',
+ 'webkit01/line0174' => 'Bug.',
</ins><span class="cx" style="display: block; padding: 0 10px"> );
</span><span class="cx" style="display: block; padding: 0 10px">
</span><span class="cx" style="display: block; padding: 0 10px">
</span><span class="lines" style="display: block; padding: 0 10px; color: #888">@@ -107,10 +98,6 @@
</span><span class="cx" style="display: block; padding: 0 10px"> continue;
</span><span class="cx" style="display: block; padding: 0 10px"> }
</span><span class="cx" style="display: block; padding: 0 10px">
</span><del style="background-color: #fdd; text-decoration:none; display:block; padding: 0 10px">- if ( 'entities01.dat' === $entry || 'entities02.dat' === $entry ) {
- continue;
- }
-
</del><span class="cx" style="display: block; padding: 0 10px"> foreach ( self::parse_html5_dat_testfile( $test_dir . $entry ) as $k => $test ) {
</span><span class="cx" style="display: block; padding: 0 10px"> // strip .dat extension from filename
</span><span class="cx" style="display: block; padding: 0 10px"> $test_suite = substr( $entry, 0, -4 );
</span></span></pre></div>
<a id="trunktestsphpunittestswptokenmapwpTokenMapphp"></a>
<div class="modfile"><h4 style="background-color: #eee; color: inherit; margin: 1em 0; padding: 1.3em; font-size: 115%">Modified: trunk/tests/phpunit/tests/wp-token-map/wpTokenMap.php</h4>
<pre class="diff"><span>
<span class="info" style="display: block; padding: 0 10px; color: #888">--- trunk/tests/phpunit/tests/wp-token-map/wpTokenMap.php 2024-06-02 09:53:47 UTC (rev 58280)
+++ trunk/tests/phpunit/tests/wp-token-map/wpTokenMap.php 2024-06-02 15:14:35 UTC (rev 58281)
</span><span class="lines" style="display: block; padding: 0 10px; color: #888">@@ -317,7 +317,7 @@
</span><span class="cx" style="display: block; padding: 0 10px"> $map = self::get_html5_token_map();
</span><span class="cx" style="display: block; padding: 0 10px">
</span><span class="cx" style="display: block; padding: 0 10px"> $skip_bytes = 0;
</span><del style="background-color: #fdd; text-decoration:none; display:block; padding: 0 10px">- $this->assertFalse(
</del><ins style="background-color: #dfd; text-decoration:none; display:block; padding: 0 10px">+ $this->assertNull(
</ins><span class="cx" style="display: block; padding: 0 10px"> $map->read_token( $document, 0, $skip_bytes ),
</span><span class="cx" style="display: block; padding: 0 10px"> "Shouldn't have found token at start of document."
</span><span class="cx" style="display: block; padding: 0 10px"> );
</span></span></pre>
</div>
</div>
</body>
</html>