<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN"
"http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head><meta http-equiv="content-type" content="text/html; charset=utf-8" />
<title>[47648] branches/4.9: Customize: Add additional filters to Customizer to prevent JSON corruption.</title>
</head>
<body>

<style type="text/css"><!--
#msg dl.meta { border: 1px #006 solid; background: #369; padding: 6px; color: #fff; }
#msg dl.meta dt { float: left; width: 6em; font-weight: bold; }
#msg dt:after { content:':';}
#msg dl, #msg dt, #msg ul, #msg li, #header, #footer, #logmsg { font-family: verdana,arial,helvetica,sans-serif; font-size: 10pt;  }
#msg dl a { font-weight: bold}
#msg dl a:link    { color:#fc3; }
#msg dl a:active  { color:#ff0; }
#msg dl a:visited { color:#cc6; }
h3 { font-family: verdana,arial,helvetica,sans-serif; font-size: 10pt; font-weight: bold; }
#msg pre { white-space: pre-line; overflow: auto; background: #ffc; border: 1px #fa0 solid; padding: 6px; }
#logmsg { background: #ffc; border: 1px #fa0 solid; padding: 1em 1em 0 1em; }
#logmsg p, #logmsg pre, #logmsg blockquote { margin: 0 0 1em 0; }
#logmsg p, #logmsg li, #logmsg dt, #logmsg dd { line-height: 14pt; }
#logmsg h1, #logmsg h2, #logmsg h3, #logmsg h4, #logmsg h5, #logmsg h6 { margin: .5em 0; }
#logmsg h1:first-child, #logmsg h2:first-child, #logmsg h3:first-child, #logmsg h4:first-child, #logmsg h5:first-child, #logmsg h6:first-child { margin-top: 0; }
#logmsg ul, #logmsg ol { padding: 0; list-style-position: inside; margin: 0 0 0 1em; }
#logmsg ul { text-indent: -1em; padding-left: 1em; }#logmsg ol { text-indent: -1.5em; padding-left: 1.5em; }
#logmsg > ul, #logmsg > ol { margin: 0 0 1em 0; }
#logmsg pre { background: #eee; padding: 1em; }
#logmsg blockquote { border: 1px solid #fa0; border-left-width: 10px; padding: 1em 1em 0 1em; background: white;}
#logmsg dl { margin: 0; }
#logmsg dt { font-weight: bold; }
#logmsg dd { margin: 0; padding: 0 0 0.5em 0; }
#logmsg dd:before { content:'\00bb';}
#logmsg table { border-spacing: 0px; border-collapse: collapse; border-top: 4px solid #fa0; border-bottom: 1px solid #fa0; background: #fff; }
#logmsg table th { text-align: left; font-weight: normal; padding: 0.2em 0.5em; border-top: 1px dotted #fa0; }
#logmsg table td { text-align: right; border-top: 1px dotted #fa0; padding: 0.2em 0.5em; }
#logmsg table thead th { text-align: center; border-bottom: 1px solid #fa0; }
#logmsg table th.Corner { text-align: left; }
#logmsg hr { border: none 0; border-top: 2px dashed #fa0; height: 1px; }
#header, #footer { color: #fff; background: #636; border: 1px #300 solid; padding: 6px; }
#patch { width: 100%; }
#patch h4 {font-family: verdana,arial,helvetica,sans-serif;font-size:10pt;padding:8px;background:#369;color:#fff;margin:0;}
#patch .propset h4, #patch .binary h4 {margin:0;}
#patch pre {padding:0;line-height:1.2em;margin:0;}
#patch .diff {width:100%;background:#eee;padding: 0 0 10px 0;overflow:auto;}
#patch .propset .diff, #patch .binary .diff  {padding:10px 0;}
#patch span {display:block;padding:0 10px;}
#patch .modfile, #patch .addfile, #patch .delfile, #patch .propset, #patch .binary, #patch .copfile {border:1px solid #ccc;margin:10px 0;}
#patch ins {background:#dfd;text-decoration:none;display:block;padding:0 10px;}
#patch del {background:#fdd;text-decoration:none;display:block;padding:0 10px;}
#patch .lines, .info {color:#888;background:#fff;}
--></style>
<div id="msg">
<dl class="meta" style="font-size: 105%">
<dt style="float: left; width: 6em; font-weight: bold">Revision</dt> <dd><a style="font-weight: bold" href="https://core.trac.wordpress.org/changeset/47648">47648</a><script type="application/ld+json">{"@context":"http://schema.org","@type":"EmailMessage","description":"Review this Commit","action":{"@type":"ViewAction","url":"https://core.trac.wordpress.org/changeset/47648","name":"Review Commit"}}</script></dd>
<dt style="float: left; width: 6em; font-weight: bold">Author</dt> <dd>whyisjake</dd>
<dt style="float: left; width: 6em; font-weight: bold">Date</dt> <dd>2020-04-29 16:15:39 +0000 (Wed, 29 Apr 2020)</dd>
</dl>

<pre style='padding-left: 1em; margin: 2em 0; border-left: 2px solid #ccc; line-height: 1.25; font-size: 105%; font-family: sans-serif'>Customize: Add additional filters to Customizer to prevent JSON corruption.
User: Invalidate `user_activation_key` on password update.
Query: Ensure that only a single post can be returned on date/time based queries.
Cache API: Ensure proper escaping around the stats method in the cache API.
Formatting: Expand `sanitize_file_name` to have better support for utf8 characters.

Brings the changes in <a href="https://core.trac.wordpress.org/changeset/47633">[47633]</a>, <a href="https://core.trac.wordpress.org/changeset/47634">[47634]</a>, <a href="https://core.trac.wordpress.org/changeset/47635">[47635]</a>, <a href="https://core.trac.wordpress.org/changeset/47637">[47637]</a>, and <a href="https://core.trac.wordpress.org/changeset/47638">[47638]</a> to the 4.9 branch.

Props: batmoo, ehti, nickdaugherty, peterwilsoncc, sergeybiryukov, sstoqnov, westi, westonruter, whyisjake, whyisjake, xknown.</pre>

<h3>Modified Paths</h3>
<ul>
<li><a href="#branches49srcwpincludescachephp">branches/4.9/src/wp-includes/cache.php</a></li>
<li><a href="#branches49srcwpincludesclasswpcustomizemanagerphp">branches/4.9/src/wp-includes/class-wp-customize-manager.php</a></li>
<li><a href="#branches49srcwpincludesclasswpqueryphp">branches/4.9/src/wp-includes/class-wp-query.php</a></li>
<li><a href="#branches49srcwpincludesformattingphp">branches/4.9/src/wp-includes/formatting.php</a></li>
<li><a href="#branches49srcwpincludespostphp">branches/4.9/src/wp-includes/post.php</a></li>
<li><a href="#branches49srcwpincludesuserphp">branches/4.9/src/wp-includes/user.php</a></li>
<li><a href="#branches49testsphpunittestscustomizemanagerphp">branches/4.9/tests/phpunit/tests/customize/manager.php</a></li>
<li><a href="#branches49testsphpunittestsformattingSanitizeFileNamephp">branches/4.9/tests/phpunit/tests/formatting/SanitizeFileName.php</a></li>
<li><a href="#branches49testsphpunittestsuserphp">branches/4.9/tests/phpunit/tests/user.php</a></li>
</ul>

<h3>Property Changed</h3>
<ul>
<li><a href="#branches49">branches/4.9/</a></li>
</ul>

</div>
<div id="patch">
<h3>Diff</h3>
<span class="cx" style="display: block; padding: 0 10px">Index: branches/4.9
</span><span class="cx" style="display: block; padding: 0 10px">===================================================================
</span><del style="background-color: #fdd; text-decoration:none; display:block; padding: 0 10px">--- branches/4.9 2020-04-29 16:13:02 UTC (rev 47647)
</del><ins style="background-color: #dfd; text-decoration:none; display:block; padding: 0 10px">+++ branches/4.9  2020-04-29 16:15:39 UTC (rev 47648)
</ins><a id="branches49"></a>
<div class="propset"><h4 style="background-color: #eee; color: inherit; margin: 1em 0; padding: 1.3em; font-size: 115%">Property changes: branches/4.9</h4>
<pre class="diff"><span>
</span></pre></div>
<a id="svnmergeinfo"></a>
<div class="modfile"><h4 style="background-color: #eee; color: inherit; margin: 1em 0; padding: 1.3em; font-size: 115%">Modified: svn:mergeinfo</h4></div>
<span class="cx" style="display: block; padding: 0 10px"> /branches/3.4:21757
</span><span class="cx" style="display: block; padding: 0 10px"> /branches/4.8:42204
</span><span class="cx" style="display: block; padding: 0 10px"> /branches/5.0:43807,43988,43994,44014,44017,44047,45005
</span><del style="background-color: #fdd; text-decoration:none; display:block; padding: 0 10px">-/trunk:18512,42132,42134,42136,42138,42140,42144,42146,42148,42150,42152-42153,42155,42157,42159,42161,42163,42169,42171,42173,42175,42177,42181,42183,42185,42187,42189,42191,42193,42199,42203,42210,42214,42220,42222,42226,42242,42244,42247,42251,42256-42261,42350,42358,42362,42364-42368,42374,42388,42390,42401,42417,42421,42423-42425,42430,42432-42433,42437,42441,42443,42446-42447,42449,42451,42453,42457,42459,42462,42491,42521,42529,42531,42533,42536,42538,42541,42543,42545,42549,42566,42568,42570,42572,42574,42576,42579,42581-42582,42584-42585,42587-42588,42590,42592,42594-42595,42598-42599,42602,42604,42606,42611-42613,42615,42617,42624-42625,42639,42648,42652-42653,42665,42676,42687,42695,42697,42702,42711,42713,42719,42722,42726,42728,42739-42740,42744,42758,42772,42791-42792,42801,42814-42815,42817-42818,42823,42830,42837,42839,42841,42844,42851-42852,42860,42864,42881,42888-42889
 ,42892-42894,42930,42964,42967,42971-42972,42977-42978,42980-42982,42985-42987,42989,42992,42994-42995,42998-42999,43001-43004,43007-43008,43011-43012,43014-43015,43025,43027,43030,43032,43034,43036,43039,43042,43044-43049,43051-43063,43065,43081,43085,43087-43089,43091,43104,43116,43118,43120-43121,43123,43125-43126,43131-43132,43135,43137,43139,43145-43148,43150,43154-43155,43158,43160,43162,43166,43168,43170,43172,43175,43180-43181,43183-43185,43189,43191,43193,43195,43197,43199,43201,43203,43206,43208,43210-43212,43216,43218,43220,43222-43223,43226,43228,43230,43232,43234,43236,43238,43242-43243,43245-43246,43248-43251,43256,43259-43260,43263,43265,43267,43269,43274-43275,43278-43279,43282,43284,43286,43290-43293,43299,43303-43304,43313,43315,43317,43331,43337,43343,43350,43353,43356,43361-43363,43365,43367,43370-43371,43373-43376,43379,43388,43390,43435,43437,43439-43440,43446-43447,43451,43454,43457,43460,43462-43467,43469,43471,43475,43477-43478,43480,43486,43491,43493,43495,
 43499,43504,43506,43508,43511-43513,43518,43525,43527,43529,43531,43541,43550,43559,43567,43580,43593,43597,43599,43609,43636,43638,43653,44021,44048,44833,44842,44993,45936,45971,45990,45997,46893,46895,47341
</del><span class="cx" style="display: block; padding: 0 10px">\ No newline at end of property
</span><ins style="background-color: #dfd; text-decoration:none; display:block; padding: 0 10px">+/trunk:18512,42132,42134,42136,42138,42140,42144,42146,42148,42150,42152-42153,42155,42157,42159,42161,42163,42169,42171,42173,42175,42177,42181,42183,42185,42187,42189,42191,42193,42199,42203,42210,42214,42220,42222,42226,42242,42244,42247,42251,42256-42261,42350,42358,42362,42364-42368,42374,42388,42390,42401,42417,42421,42423-42425,42430,42432-42433,42437,42441,42443,42446-42447,42449,42451,42453,42457,42459,42462,42491,42521,42529,42531,42533,42536,42538,42541,42543,42545,42549,42566,42568,42570,42572,42574,42576,42579,42581-42582,42584-42585,42587-42588,42590,42592,42594-42595,42598-42599,42602,42604,42606,42611-42613,42615,42617,42624-42625,42639,42648,42652-42653,42665,42676,42687,42695,42697,42702,42711,42713,42719,42722,42726,42728,42739-42740,42744,42758,42772,42791-42792,42801,42814-42815,42817-42818,42823,42830,42837,42839,42841,42844,42851-42852,42860,42864,42881,42888-42889
 ,42892-42894,42930,42964,42967,42971-42972,42977-42978,42980-42982,42985-42987,42989,42992,42994-42995,42998-42999,43001-43004,43007-43008,43011-43012,43014-43015,43025,43027,43030,43032,43034,43036,43039,43042,43044-43049,43051-43063,43065,43081,43085,43087-43089,43091,43104,43116,43118,43120-43121,43123,43125-43126,43131-43132,43135,43137,43139,43145-43148,43150,43154-43155,43158,43160,43162,43166,43168,43170,43172,43175,43180-43181,43183-43185,43189,43191,43193,43195,43197,43199,43201,43203,43206,43208,43210-43212,43216,43218,43220,43222-43223,43226,43228,43230,43232,43234,43236,43238,43242-43243,43245-43246,43248-43251,43256,43259-43260,43263,43265,43267,43269,43274-43275,43278-43279,43282,43284,43286,43290-43293,43299,43303-43304,43313,43315,43317,43331,43337,43343,43350,43353,43356,43361-43363,43365,43367,43370-43371,43373-43376,43379,43388,43390,43435,43437,43439-43440,43446-43447,43451,43454,43457,43460,43462-43467,43469,43471,43475,43477-43478,43480,43486,43491,43493,43495,
 43499,43504,43506,43508,43511-43513,43518,43525,43527,43529,43531,43541,43550,43559,43567,43580,43593,43597,43599,43609,43636,43638,43653,44021,44048,44833,44842,44993,45936,45971,45990,45997,46893,46895,47341,47633-47635,47637-47638
</ins><span class="cx" style="display: block; padding: 0 10px">\ No newline at end of property
</span><a id="branches49srcwpincludescachephp"></a>
<div class="modfile"><h4 style="background-color: #eee; color: inherit; margin: 1em 0; padding: 1.3em; font-size: 115%">Modified: branches/4.9/src/wp-includes/cache.php</h4>
<pre class="diff"><span>
<span class="info" style="display: block; padding: 0 10px; color: #888">--- branches/4.9/src/wp-includes/cache.php    2020-04-29 16:13:02 UTC (rev 47647)
+++ branches/4.9/src/wp-includes/cache.php      2020-04-29 16:15:39 UTC (rev 47648)
</span><span class="lines" style="display: block; padding: 0 10px; color: #888">@@ -668,7 +668,7 @@
</span><span class="cx" style="display: block; padding: 0 10px">                echo "</p>";
</span><span class="cx" style="display: block; padding: 0 10px">                echo '<ul>';
</span><span class="cx" style="display: block; padding: 0 10px">                foreach ($this->cache as $group => $cache) {
</span><del style="background-color: #fdd; text-decoration:none; display:block; padding: 0 10px">-                        echo "<li><strong>Group:</strong> $group - ( " . number_format( strlen( serialize( $cache ) ) / KB_IN_BYTES, 2 ) . 'k )</li>';
</del><ins style="background-color: #dfd; text-decoration:none; display:block; padding: 0 10px">+                 echo '<li><strong>Group:</strong> ' . esc_html( $group ) . ' - ( ' . number_format( strlen( serialize( $cache ) ) / KB_IN_BYTES, 2 ) . 'k )</li>';
</ins><span class="cx" style="display: block; padding: 0 10px">                 }
</span><span class="cx" style="display: block; padding: 0 10px">                echo '</ul>';
</span><span class="cx" style="display: block; padding: 0 10px">        }
</span></span></pre></div>
<a id="branches49srcwpincludesclasswpcustomizemanagerphp"></a>
<div class="modfile"><h4 style="background-color: #eee; color: inherit; margin: 1em 0; padding: 1.3em; font-size: 115%">Modified: branches/4.9/src/wp-includes/class-wp-customize-manager.php</h4>
<pre class="diff"><span>
<span class="info" style="display: block; padding: 0 10px; color: #888">--- branches/4.9/src/wp-includes/class-wp-customize-manager.php       2020-04-29 16:13:02 UTC (rev 47647)
+++ branches/4.9/src/wp-includes/class-wp-customize-manager.php 2020-04-29 16:15:39 UTC (rev 47648)
</span><span class="lines" style="display: block; padding: 0 10px; color: #888">@@ -2859,13 +2859,12 @@
</span><span class="cx" style="display: block; padding: 0 10px">                $this->store_changeset_revision = $allow_revision;
</span><span class="cx" style="display: block; padding: 0 10px">                add_filter( 'wp_save_post_revision_post_has_changed', array( $this, '_filter_revision_post_has_changed' ), 5, 3 );
</span><span class="cx" style="display: block; padding: 0 10px"> 
</span><del style="background-color: #fdd; text-decoration:none; display:block; padding: 0 10px">-                // Update the changeset post. The publish_customize_changeset action will cause the settings in the changeset to be saved via WP_Customize_Setting::save().
-               $has_kses = ( false !== has_filter( 'content_save_pre', 'wp_filter_post_kses' ) );
-               if ( $has_kses ) {
-                       kses_remove_filters(); // Prevent KSES from corrupting JSON in post_content.
-               }
-
-               // Note that updating a post with publish status will trigger WP_Customize_Manager::publish_changeset_values().
</del><ins style="background-color: #dfd; text-decoration:none; display:block; padding: 0 10px">+         /*
+                * Update the changeset post. The publish_customize_changeset action will cause the settings in the
+                * changeset to be saved via WP_Customize_Setting::save(). Updating a post with publish status will
+                * trigger WP_Customize_Manager::publish_changeset_values().
+                */
+               add_filter( 'wp_insert_post_data', array( $this, 'preserve_insert_changeset_post_content' ), 5, 3 );
</ins><span class="cx" style="display: block; padding: 0 10px">                 if ( $changeset_post_id ) {
</span><span class="cx" style="display: block; padding: 0 10px">                        if ( $args['autosave'] && 'auto-draft' !== get_post_status( $changeset_post_id ) ) {
</span><span class="cx" style="display: block; padding: 0 10px">                                // See _wp_translate_postdata() for why this is required as it will use the edit_post meta capability.
</span><span class="lines" style="display: block; padding: 0 10px; color: #888">@@ -2892,9 +2891,9 @@
</span><span class="cx" style="display: block; padding: 0 10px">                                $this->_changeset_post_id = $r; // Update cached post ID for the loaded changeset.
</span><span class="cx" style="display: block; padding: 0 10px">                        }
</span><span class="cx" style="display: block; padding: 0 10px">                }
</span><del style="background-color: #fdd; text-decoration:none; display:block; padding: 0 10px">-                if ( $has_kses ) {
-                       kses_init_filters();
-               }
</del><ins style="background-color: #dfd; text-decoration:none; display:block; padding: 0 10px">+
+               remove_filter( 'wp_insert_post_data', array( $this, 'preserve_insert_changeset_post_content' ), 5 );
+
</ins><span class="cx" style="display: block; padding: 0 10px">                 $this->_changeset_data = null; // Reset so WP_Customize_Manager::changeset_data() will re-populate with updated contents.
</span><span class="cx" style="display: block; padding: 0 10px"> 
</span><span class="cx" style="display: block; padding: 0 10px">                remove_filter( 'wp_save_post_revision_post_has_changed', array( $this, '_filter_revision_post_has_changed' ) );
</span><span class="lines" style="display: block; padding: 0 10px; color: #888">@@ -2912,6 +2911,51 @@
</span><span class="cx" style="display: block; padding: 0 10px">        }
</span><span class="cx" style="display: block; padding: 0 10px"> 
</span><span class="cx" style="display: block; padding: 0 10px">        /**
</span><ins style="background-color: #dfd; text-decoration:none; display:block; padding: 0 10px">+         * Preserve the initial JSON post_content passed to save into the post.
+        *
+        * This is needed to prevent KSES and other {@see 'content_save_pre'} filters
+        * from corrupting JSON data.
+        *
+        * Note that WP_Customize_Manager::validate_setting_values() have already
+        * run on the setting values being serialized as JSON into the post content
+        * so it is pre-sanitized.
+        *
+        * Also, the sanitization logic is re-run through the respective
+        * WP_Customize_Setting::sanitize() method when being read out of the
+        * changeset, via WP_Customize_Manager::post_value(), and this sanitized
+        * value will also be sent into WP_Customize_Setting::update() for
+        * persisting to the DB.
+        *
+        * Multiple users can collaborate on a single changeset, where one user may
+        * have the unfiltered_html capability but another may not. A user with
+        * unfiltered_html may add a script tag to some field which needs to be kept
+        * intact even when another user updates the changeset to modify another field
+        * when they do not have unfiltered_html.
+        *
+        * @since 5.4.1
+        *
+        * @param array $data                An array of slashed and processed post data.
+        * @param array $postarr             An array of sanitized (and slashed) but otherwise unmodified post data.
+        * @param array $unsanitized_postarr An array of slashed yet *unsanitized* and unprocessed post data as originally passed to wp_insert_post().
+        * @return array Filtered post data.
+        */
+       public function preserve_insert_changeset_post_content( $data, $postarr, $unsanitized_postarr ) {
+               if (
+                       isset( $data['post_type'] ) &&
+                       isset( $unsanitized_postarr['post_content'] ) &&
+                       'customize_changeset' === $data['post_type'] ||
+                       (
+                               'revision' === $data['post_type'] &&
+                               ! empty( $data['post_parent'] ) &&
+                               'customize_changeset' === get_post_type( $data['post_parent'] )
+                       )
+               ) {
+                       $data['post_content'] = $unsanitized_postarr['post_content'];
+               }
+               return $data;
+       }
+
+       /**
</ins><span class="cx" style="display: block; padding: 0 10px">          * Trash or delete a changeset post.
</span><span class="cx" style="display: block; padding: 0 10px">         *
</span><span class="cx" style="display: block; padding: 0 10px">         * The following re-formulates the logic from `wp_trash_post()` as done in
</span></span></pre></div>
<a id="branches49srcwpincludesclasswpqueryphp"></a>
<div class="modfile"><h4 style="background-color: #eee; color: inherit; margin: 1em 0; padding: 1.3em; font-size: 115%">Modified: branches/4.9/src/wp-includes/class-wp-query.php</h4>
<pre class="diff"><span>
<span class="info" style="display: block; padding: 0 10px; color: #888">--- branches/4.9/src/wp-includes/class-wp-query.php   2020-04-29 16:13:02 UTC (rev 47647)
+++ branches/4.9/src/wp-includes/class-wp-query.php     2020-04-29 16:15:39 UTC (rev 47648)
</span><span class="lines" style="display: block; padding: 0 10px; color: #888">@@ -759,10 +759,6 @@
</span><span class="cx" style="display: block; padding: 0 10px">                        $this->is_single = true;
</span><span class="cx" style="display: block; padding: 0 10px">                } elseif ( $qv['p'] ) {
</span><span class="cx" style="display: block; padding: 0 10px">                        $this->is_single = true;
</span><del style="background-color: #fdd; text-decoration:none; display:block; padding: 0 10px">-                } elseif ( ('' !== $qv['hour']) && ('' !== $qv['minute']) &&('' !== $qv['second']) && ('' != $qv['year']) && ('' != $qv['monthnum']) && ('' != $qv['day']) ) {
-                       // If year, month, day, hour, minute, and second are set, a single
-                       // post is being queried.
-                       $this->is_single = true;
</del><span class="cx" style="display: block; padding: 0 10px">                 } elseif ( '' != $qv['pagename'] || !empty($qv['page_id']) ) {
</span><span class="cx" style="display: block; padding: 0 10px">                        $this->is_page = true;
</span><span class="cx" style="display: block; padding: 0 10px">                        $this->is_single = false;
</span></span></pre></div>
<a id="branches49srcwpincludesformattingphp"></a>
<div class="modfile"><h4 style="background-color: #eee; color: inherit; margin: 1em 0; padding: 1.3em; font-size: 115%">Modified: branches/4.9/src/wp-includes/formatting.php</h4>
<pre class="diff"><span>
<span class="info" style="display: block; padding: 0 10px; color: #888">--- branches/4.9/src/wp-includes/formatting.php       2020-04-29 16:13:02 UTC (rev 47647)
+++ branches/4.9/src/wp-includes/formatting.php 2020-04-29 16:15:39 UTC (rev 47648)
</span><span class="lines" style="display: block; padding: 0 10px; color: #888">@@ -1776,6 +1776,24 @@
</span><span class="cx" style="display: block; padding: 0 10px"> function sanitize_file_name( $filename ) {
</span><span class="cx" style="display: block; padding: 0 10px">        $filename_raw = $filename;
</span><span class="cx" style="display: block; padding: 0 10px">        $special_chars = array("?", "[", "]", "/", "\\", "=", "<", ">", ":", ";", ",", "'", "\"", "&", "$", "#", "*", "(", ")", "|", "~", "`", "!", "{", "}", "%", "+", chr(0));
</span><ins style="background-color: #dfd; text-decoration:none; display:block; padding: 0 10px">+
+       // Check for support for utf8 in the installed PCRE library once and store the result in a static.
+       static $utf8_pcre = null;
+       if ( ! isset( $utf8_pcre ) ) {
+               // phpcs:ignore WordPress.PHP.NoSilencedErrors.Discouraged
+               $utf8_pcre = @preg_match( '/^./u', 'a' );
+       }
+
+       if ( ! seems_utf8( $filename ) ) {
+               $_ext     = pathinfo( $filename, PATHINFO_EXTENSION );
+               $_name    = pathinfo( $filename, PATHINFO_FILENAME );
+               $filename = sanitize_title_with_dashes( $_name ) . '.' . $_ext;
+       }
+
+       if ( $utf8_pcre ) {
+               $filename = preg_replace( "#\x{00a0}#siu", ' ', $filename );
+       }
+
</ins><span class="cx" style="display: block; padding: 0 10px">         /**
</span><span class="cx" style="display: block; padding: 0 10px">         * Filters the list of characters to remove from a filename.
</span><span class="cx" style="display: block; padding: 0 10px">         *
</span><span class="lines" style="display: block; padding: 0 10px; color: #888">@@ -1785,7 +1803,6 @@
</span><span class="cx" style="display: block; padding: 0 10px">         * @param string $filename_raw  Filename as it was passed into sanitize_file_name().
</span><span class="cx" style="display: block; padding: 0 10px">         */
</span><span class="cx" style="display: block; padding: 0 10px">        $special_chars = apply_filters( 'sanitize_file_name_chars', $special_chars, $filename_raw );
</span><del style="background-color: #fdd; text-decoration:none; display:block; padding: 0 10px">-        $filename = preg_replace( "#\x{00a0}#siu", ' ', $filename );
</del><span class="cx" style="display: block; padding: 0 10px">         $filename = str_replace( $special_chars, '', $filename );
</span><span class="cx" style="display: block; padding: 0 10px">        $filename = str_replace( array( '%20', '+' ), '-', $filename );
</span><span class="cx" style="display: block; padding: 0 10px">        $filename = preg_replace( '/[\r\n\t -]+/', '-', $filename );
</span></span></pre></div>
<a id="branches49srcwpincludespostphp"></a>
<div class="modfile"><h4 style="background-color: #eee; color: inherit; margin: 1em 0; padding: 1.3em; font-size: 115%">Modified: branches/4.9/src/wp-includes/post.php</h4>
<pre class="diff"><span>
<span class="info" style="display: block; padding: 0 10px; color: #888">--- branches/4.9/src/wp-includes/post.php     2020-04-29 16:13:02 UTC (rev 47647)
+++ branches/4.9/src/wp-includes/post.php       2020-04-29 16:15:39 UTC (rev 47648)
</span><span class="lines" style="display: block; padding: 0 10px; color: #888">@@ -3136,6 +3136,9 @@
</span><span class="cx" style="display: block; padding: 0 10px"> function wp_insert_post( $postarr, $wp_error = false ) {
</span><span class="cx" style="display: block; padding: 0 10px">        global $wpdb;
</span><span class="cx" style="display: block; padding: 0 10px"> 
</span><ins style="background-color: #dfd; text-decoration:none; display:block; padding: 0 10px">+        // Capture original pre-sanitized array for passing into filters.
+       $unsanitized_postarr = $postarr;
+
</ins><span class="cx" style="display: block; padding: 0 10px">         $user_id = get_current_user_id();
</span><span class="cx" style="display: block; padding: 0 10px"> 
</span><span class="cx" style="display: block; padding: 0 10px">        $defaults = array(
</span><span class="lines" style="display: block; padding: 0 10px; color: #888">@@ -3432,21 +3435,27 @@
</span><span class="cx" style="display: block; padding: 0 10px">                 * Filters attachment post data before it is updated in or added to the database.
</span><span class="cx" style="display: block; padding: 0 10px">                 *
</span><span class="cx" style="display: block; padding: 0 10px">                 * @since 3.9.0
</span><ins style="background-color: #dfd; text-decoration:none; display:block; padding: 0 10px">+                 * @since 5.4.1 `$unsanitized_postarr` argument added.
</ins><span class="cx" style="display: block; padding: 0 10px">                  *
</span><del style="background-color: #fdd; text-decoration:none; display:block; padding: 0 10px">-                 * @param array $data    An array of sanitized attachment post data.
-                * @param array $postarr An array of unsanitized attachment post data.
</del><ins style="background-color: #dfd; text-decoration:none; display:block; padding: 0 10px">+          * @param array $data                An array of slashed, sanitized, and processed attachment post data.
+                * @param array $postarr             An array of slashed and sanitized attachment post data, but not processed.
+                * @param array $unsanitized_postarr An array of slashed yet *unsanitized* and unprocessed attachment post data
+                *                                   as originally passed to wp_insert_post().
</ins><span class="cx" style="display: block; padding: 0 10px">                  */
</span><del style="background-color: #fdd; text-decoration:none; display:block; padding: 0 10px">-                $data = apply_filters( 'wp_insert_attachment_data', $data, $postarr );
</del><ins style="background-color: #dfd; text-decoration:none; display:block; padding: 0 10px">+         $data = apply_filters( 'wp_insert_attachment_data', $data, $postarr, $unsanitized_postarr );
</ins><span class="cx" style="display: block; padding: 0 10px">         } else {
</span><span class="cx" style="display: block; padding: 0 10px">                /**
</span><span class="cx" style="display: block; padding: 0 10px">                 * Filters slashed post data just before it is inserted into the database.
</span><span class="cx" style="display: block; padding: 0 10px">                 *
</span><span class="cx" style="display: block; padding: 0 10px">                 * @since 2.7.0
</span><ins style="background-color: #dfd; text-decoration:none; display:block; padding: 0 10px">+                 * @since 5.4.1 `$unsanitized_postarr` argument added.
</ins><span class="cx" style="display: block; padding: 0 10px">                  *
</span><del style="background-color: #fdd; text-decoration:none; display:block; padding: 0 10px">-                 * @param array $data    An array of slashed post data.
-                * @param array $postarr An array of sanitized, but otherwise unmodified post data.
</del><ins style="background-color: #dfd; text-decoration:none; display:block; padding: 0 10px">+          * @param array $data                An array of slashed, sanitized, and processed post data.
+                * @param array $postarr             An array of sanitized (and slashed) but otherwise unmodified post data.
+                * @param array $unsanitized_postarr An array of slashed yet *unsanitized* and unprocessed post data as
+                *                                   originally passed to wp_insert_post().
</ins><span class="cx" style="display: block; padding: 0 10px">                  */
</span><del style="background-color: #fdd; text-decoration:none; display:block; padding: 0 10px">-                $data = apply_filters( 'wp_insert_post_data', $data, $postarr );
</del><ins style="background-color: #dfd; text-decoration:none; display:block; padding: 0 10px">+         $data = apply_filters( 'wp_insert_post_data', $data, $postarr, $unsanitized_postarr );
</ins><span class="cx" style="display: block; padding: 0 10px">         }
</span><span class="cx" style="display: block; padding: 0 10px">        $data = wp_unslash( $data );
</span><span class="cx" style="display: block; padding: 0 10px">        $where = array( 'ID' => $post_ID );
</span></span></pre></div>
<a id="branches49srcwpincludesuserphp"></a>
<div class="modfile"><h4 style="background-color: #eee; color: inherit; margin: 1em 0; padding: 1.3em; font-size: 115%">Modified: branches/4.9/src/wp-includes/user.php</h4>
<pre class="diff"><span>
<span class="info" style="display: block; padding: 0 10px; color: #888">--- branches/4.9/src/wp-includes/user.php     2020-04-29 16:13:02 UTC (rev 47647)
+++ branches/4.9/src/wp-includes/user.php       2020-04-29 16:15:39 UTC (rev 47648)
</span><span class="lines" style="display: block; padding: 0 10px; color: #888">@@ -1692,7 +1692,7 @@
</span><span class="cx" style="display: block; padding: 0 10px">        $data = apply_filters( 'wp_pre_insert_user_data', $data, $update, $update ? (int) $ID : null );
</span><span class="cx" style="display: block; padding: 0 10px"> 
</span><span class="cx" style="display: block; padding: 0 10px">        if ( $update ) {
</span><del style="background-color: #fdd; text-decoration:none; display:block; padding: 0 10px">-                if ( $user_email !== $old_user_data->user_email ) {
</del><ins style="background-color: #dfd; text-decoration:none; display:block; padding: 0 10px">+         if ( $user_email !== $old_user_data->user_email || $user_pass !== $old_user_data->user_pass ) {
</ins><span class="cx" style="display: block; padding: 0 10px">                         $data['user_activation_key'] = '';
</span><span class="cx" style="display: block; padding: 0 10px">                }
</span><span class="cx" style="display: block; padding: 0 10px">                $wpdb->update( $wpdb->users, $data, compact( 'ID' ) );
</span></span></pre></div>
<a id="branches49testsphpunittestscustomizemanagerphp"></a>
<div class="modfile"><h4 style="background-color: #eee; color: inherit; margin: 1em 0; padding: 1.3em; font-size: 115%">Modified: branches/4.9/tests/phpunit/tests/customize/manager.php</h4>
<pre class="diff"><span>
<span class="info" style="display: block; padding: 0 10px; color: #888">--- branches/4.9/tests/phpunit/tests/customize/manager.php    2020-04-29 16:13:02 UTC (rev 47647)
+++ branches/4.9/tests/phpunit/tests/customize/manager.php      2020-04-29 16:15:39 UTC (rev 47648)
</span><span class="lines" style="display: block; padding: 0 10px; color: #888">@@ -1090,6 +1090,171 @@
</span><span class="cx" style="display: block; padding: 0 10px">        }
</span><span class="cx" style="display: block; padding: 0 10px"> 
</span><span class="cx" style="display: block; padding: 0 10px">        /**
</span><ins style="background-color: #dfd; text-decoration:none; display:block; padding: 0 10px">+         * Test saving changeset post without Kses or other content_save_pre filters mutating content.
+        *
+        * @covers WP_Customize_Manager::save_changeset_post()
+        */
+       public function test_save_changeset_post_without_kses_corrupting_json() {
+               global $wp_customize;
+               $lesser_admin_user_id = self::factory()->user->create( array( 'role' => 'administrator' ) );
+
+               $uuid         = wp_generate_uuid4();
+               $wp_customize = new WP_Customize_Manager(
+                       array(
+                               'changeset_uuid' => $uuid,
+                       )
+               );
+
+               add_filter( 'map_meta_cap', array( $this, 'filter_map_meta_cap_to_disallow_unfiltered_html' ), 10, 2 );
+               kses_init();
+               add_filter( 'content_save_pre', 'capital_P_dangit' );
+               add_post_type_support( 'customize_changeset', 'revisions' );
+
+               $options = array(
+                       'custom_html_1' => '<script>document.write(" Wordpress 1")</script>',
+                       'custom_html_2' => '<script>document.write(" Wordpress 2")</script>',
+                       'custom_html_3' => '<script>document.write(" Wordpress 3")</script>',
+               );
+
+               // Populate setting as user who can bypass content_save_pre filter.
+               wp_set_current_user( self::$admin_user_id );
+               $wp_customize = $this->get_manager_for_testing_json_corruption_protection( $uuid );
+               $wp_customize->set_post_value( 'custom_html_1', $options['custom_html_1'] );
+               $wp_customize->save_changeset_post(
+                       array(
+                               'status' => 'draft',
+                       )
+               );
+
+               // Populate setting as user who cannot bypass content_save_pre filter.
+               wp_set_current_user( $lesser_admin_user_id );
+               $wp_customize = $this->get_manager_for_testing_json_corruption_protection( $uuid );
+               $wp_customize->set_post_value( 'custom_html_2', $options['custom_html_2'] );
+               $wp_customize->save_changeset_post(
+                       array(
+                               'autosave' => true,
+                       )
+               );
+
+               /*
+                * Ensure that the unsanitized value (the "POST data") is preserved in the autosave revision.
+                * The value is sent through the sanitize function when it is read from the changeset.
+                */
+               $autosave_revision = wp_get_post_autosave( $wp_customize->changeset_post_id(), get_current_user_id() );
+               $saved_data        = json_decode( $autosave_revision->post_content, true );
+               $this->assertEquals( $options['custom_html_1'], $saved_data['custom_html_1']['value'] );
+               $this->assertEquals( $options['custom_html_2'], $saved_data['custom_html_2']['value'] );
+
+               // Update post to discard autosave.
+               $wp_customize->save_changeset_post(
+                       array(
+                               'status' => 'draft',
+                       )
+               );
+
+               /*
+                * Ensure that the unsanitized value (the "POST data") is preserved in the post content.
+                * The value is sent through the sanitize function when it is read from the changeset.
+                */
+               $wp_customize = $this->get_manager_for_testing_json_corruption_protection( $uuid );
+               $saved_data   = json_decode( get_post( $wp_customize->changeset_post_id() )->post_content, true );
+               $this->assertEquals( $options['custom_html_1'], $saved_data['custom_html_1']['value'] );
+               $this->assertEquals( $options['custom_html_2'], $saved_data['custom_html_2']['value'] );
+
+               /*
+                * Ensure that the unsanitized value (the "POST data") is preserved in the revisions' content.
+                * The value is sent through the sanitize function when it is read from the changeset.
+                */
+               $revisions  = wp_get_post_revisions( $wp_customize->changeset_post_id() );
+               $revision   = array_shift( $revisions );
+               $saved_data = json_decode( $revision->post_content, true );
+               $this->assertEquals( $options['custom_html_1'], $saved_data['custom_html_1']['value'] );
+               $this->assertEquals( $options['custom_html_2'], $saved_data['custom_html_2']['value'] );
+
+               /*
+                * Now when publishing the changeset, the unsanitized values will be read from the changeset
+                * and sanitized according to the capabilities of the users who originally updated each
+                * setting in the changeset to begin with.
+                */
+               wp_set_current_user( $lesser_admin_user_id );
+               $wp_customize = $this->get_manager_for_testing_json_corruption_protection( $uuid );
+               $wp_customize->set_post_value( 'custom_html_3', $options['custom_html_3'] );
+               $wp_customize->save_changeset_post(
+                       array(
+                               'status' => 'publish',
+                       )
+               );
+
+               // User saved as one who can bypass content_save_pre filter.
+               $this->assertContains( '<script>', get_option( 'custom_html_1' ) );
+               $this->assertContains( 'Wordpress', get_option( 'custom_html_1' ) ); // phpcs:ignore WordPress.WP.CapitalPDangit.Misspelled
+
+               // User saved as one who cannot bypass content_save_pre filter.
+               $this->assertNotContains( '<script>', get_option( 'custom_html_2' ) );
+               $this->assertContains( 'WordPress', get_option( 'custom_html_2' ) );
+
+               // User saved as one who also cannot bypass content_save_pre filter.
+               $this->assertNotContains( '<script>', get_option( 'custom_html_3' ) );
+               $this->assertContains( 'WordPress', get_option( 'custom_html_3' ) );
+       }
+
+       /**
+        * Get a manager for testing JSON corruption protection.
+        *
+        * @param string $uuid UUID.
+        * @return WP_Customize_Manager Manager.
+        */
+       private function get_manager_for_testing_json_corruption_protection( $uuid ) {
+               global $wp_customize;
+               $wp_customize = new WP_Customize_Manager(
+                       array(
+                               'changeset_uuid' => $uuid,
+                       )
+               );
+               for ( $i = 0; $i < 5; $i++ ) {
+                       $wp_customize->add_setting(
+                               sprintf( 'custom_html_%d', $i ),
+                               array(
+                                       'type'              => 'option',
+                                       'sanitize_callback' => array( $this, 'apply_content_save_pre_filters_if_not_main_admin_user' ),
+                               )
+                       );
+               }
+               return $wp_customize;
+       }
+
+       /**
+        * Sanitize content with Kses if the current user is not the main admin.
+        *
+        * @since 5.4.1
+        *
+        * @param string $content Content to sanitize.
+        * @return string Sanitized content.
+        */
+       public function apply_content_save_pre_filters_if_not_main_admin_user( $content ) {
+               if ( get_current_user_id() !== self::$admin_user_id ) {
+                       $content = apply_filters( 'content_save_pre', $content );
+               }
+               return $content;
+       }
+
+       /**
+        * Filter map_meta_cap to disallow unfiltered_html.
+        *
+        * @since 5.4.1
+        *
+        * @param array  $caps User's capabilities.
+        * @param string $cap  Requested cap.
+        * @return array Caps.
+        */
+       public function filter_map_meta_cap_to_disallow_unfiltered_html( $caps, $cap ) {
+               if ( 'unfiltered_html' === $cap ) {
+                       $caps = array( 'do_not_allow' );
+               }
+               return $caps;
+       }
+
+       /**
</ins><span class="cx" style="display: block; padding: 0 10px">          * Call count for customize_changeset_save_data filter.
</span><span class="cx" style="display: block; padding: 0 10px">         *
</span><span class="cx" style="display: block; padding: 0 10px">         * @var int
</span></span></pre></div>
<a id="branches49testsphpunittestsformattingSanitizeFileNamephp"></a>
<div class="modfile"><h4 style="background-color: #eee; color: inherit; margin: 1em 0; padding: 1.3em; font-size: 115%">Modified: branches/4.9/tests/phpunit/tests/formatting/SanitizeFileName.php</h4>
<pre class="diff"><span>
<span class="info" style="display: block; padding: 0 10px; color: #888">--- branches/4.9/tests/phpunit/tests/formatting/SanitizeFileName.php  2020-04-29 16:13:02 UTC (rev 47647)
+++ branches/4.9/tests/phpunit/tests/formatting/SanitizeFileName.php    2020-04-29 16:15:39 UTC (rev 47648)
</span><span class="lines" style="display: block; padding: 0 10px; color: #888">@@ -67,4 +67,20 @@
</span><span class="cx" style="display: block; padding: 0 10px">                // Test a filenames that becomes extensionless.
</span><span class="cx" style="display: block; padding: 0 10px">                $this->assertEquals( 'no-extension', sanitize_file_name( '_.no-extension' ) );
</span><span class="cx" style="display: block; padding: 0 10px">        }
</span><ins style="background-color: #dfd; text-decoration:none; display:block; padding: 0 10px">+
+       /**
+        * @dataProvider data_wp_filenames
+        */
+       function test_replaces_invalid_utf8_characters( $input, $expected ) {
+               $this->assertEquals( $expected, sanitize_file_name( $input ) );
+       }
+
+       function data_wp_filenames() {
+               return array(
+                       array( urldecode( '%B1myfile.png' ), 'myfile.png' ),
+                       array( urldecode( '%B1myfile' ), 'myfile' ),
+                       array( 'demo bar.png', 'demo-bar.png' ),
+                       array( 'demo' . json_decode( '"\u00a0"' ) . 'bar.png', 'demo-bar.png' ),
+               );
+       }
</ins><span class="cx" style="display: block; padding: 0 10px"> }
</span></span></pre></div>
<a id="branches49testsphpunittestsuserphp"></a>
<div class="modfile"><h4 style="background-color: #eee; color: inherit; margin: 1em 0; padding: 1.3em; font-size: 115%">Modified: branches/4.9/tests/phpunit/tests/user.php</h4>
<pre class="diff"><span>
<span class="info" style="display: block; padding: 0 10px; color: #888">--- branches/4.9/tests/phpunit/tests/user.php 2020-04-29 16:13:02 UTC (rev 47647)
+++ branches/4.9/tests/phpunit/tests/user.php   2020-04-29 16:15:39 UTC (rev 47648)
</span><span class="lines" style="display: block; padding: 0 10px; color: #888">@@ -917,7 +917,7 @@
</span><span class="cx" style="display: block; padding: 0 10px">                $this->assertEquals( $u, wp_cache_get( $updated_user->user_nicename, 'userslugs' ) );
</span><span class="cx" style="display: block; padding: 0 10px">        }
</span><span class="cx" style="display: block; padding: 0 10px"> 
</span><del style="background-color: #fdd; text-decoration:none; display:block; padding: 0 10px">-        function test_changing_email_invalidates_password_reset_key() {
</del><ins style="background-color: #dfd; text-decoration:none; display:block; padding: 0 10px">+ public function test_changing_email_invalidates_password_reset_key() {
</ins><span class="cx" style="display: block; padding: 0 10px">                 global $wpdb;
</span><span class="cx" style="display: block; padding: 0 10px"> 
</span><span class="cx" style="display: block; padding: 0 10px">                $user = $this->author;
</span><span class="lines" style="display: block; padding: 0 10px; color: #888">@@ -949,6 +949,26 @@
</span><span class="cx" style="display: block; padding: 0 10px">                $this->assertEmpty( $user->user_activation_key );
</span><span class="cx" style="display: block; padding: 0 10px">        }
</span><span class="cx" style="display: block; padding: 0 10px"> 
</span><ins style="background-color: #dfd; text-decoration:none; display:block; padding: 0 10px">+        public function test_changing_password_invalidates_password_reset_key() {
+               global $wpdb;
+
+               $user = $this->author;
+               $wpdb->update( $wpdb->users, array( 'user_activation_key' => 'key' ), array( 'ID' => $user->ID ) );
+               clean_user_cache( $user );
+
+               $user = get_userdata( $user->ID );
+               $this->assertEquals( 'key', $user->user_activation_key );
+
+               $userdata = array(
+                       'ID'        => $user->ID,
+                       'user_pass' => 'password',
+               );
+               wp_update_user( $userdata );
+
+               $user = get_userdata( $user->ID );
+               $this->assertEmpty( $user->user_activation_key );
+       }
+
</ins><span class="cx" style="display: block; padding: 0 10px">         public function test_search_users_login() {
</span><span class="cx" style="display: block; padding: 0 10px">                $users = get_users( array( 'search' => 'user1', 'fields' => 'ID' ) );
</span><span class="cx" style="display: block; padding: 0 10px"> 
</span></span></pre>
</div>
</div>

</body>
</html>