[wp-meta] [Making WordPress.org] #2280: Broken encoding in some posts on support forums

[Making WordPress.org]

#2280: Broken encoding in some posts on support forums
----------------------------+-----------------
 Reporter:  SergeyBiryukov  |      Owner:
     Type:  defect          |     Status:  new
 Priority:  high            |  Milestone:
Component:  Support Forums  |   Keywords:
----------------------------+-----------------
 Since yesterday (November 28th, Monday), some posts on support forums are
 occasionally displayed in wrong encoding (Windows-1252 instead of UTF-8),
 which leads to broken Cyrillic or Japanese characters.

 A workaround is to click Edit and re-save the affected post, which makes
 it readable. One user reported that email notification for the post was
 readable as well, only the post itself in the forums thread had wrong
 encoding.

 [https://wordpress.slack.com/archives/meta/p1480297680001613 First
 reported in #meta] by @rmccue, regarding a post on make/core:
 {{{
 rmccue: https://make.wordpress.org/core/2016/11/25/preferred-languages-
 research/ has an encoding bug
 rmccue uploaded an image: Screenshot 2016-11-28 11.47.38.png

 dd32: Calling @pento is ^ related to your HyperDB change?

 rmccue: Also seeing the classic ’ -> ’

 pento: Ugh, possibly. Google’s cached version from yesterday isn’t showing
 it, so I assume it’s related.
 }}}

 Then by @takahashi_fumiki, regarding a thread on support forums:
 {{{
 takahashi_fumiki: Every multibyte characters were broken on Forum.
 https://wordpress.org/support/topic/proxy-
 error%e3%81%8c%e8%a1%a8%e7%a4%ba%e3%81%95%e3%82%8c%e3%82%8b/#post-8486994
 But now it seemed fixed. Ping for just confirmation.

 pento: Thanks @takahashi_fumiki, we were using the wrong character set for
 a little bit, it’s fixed now.
 Please let us know if you see any other problems. :)
 }}}

 At this point, the issue was supposedly fixed, but our moderators still
 encountered it:
 {{{
 yui:
 https://ru.wordpress.org/support/topic/%d0%be%d1%88%d0%b8%d0%b1%d0%ba%d0%b8-%d0%bf%d1%80%d0%b8-%d0%b8%d0%bc%d0%bf%d0%be%d1%80%d1%82%d0%b5-xml-%d0%bd%d0%b5-%d1%83%d0%b4%d0%b0%d0%bb%d0%be%d1%81%d1%8c-%d0%b8%d0%bc%d0%bf%d0%be%d1%80%d1%82/#post-228350
 cyrillic chars are broken here
 It seems to be in CP 1252 instead of UTF8
 > we were using the wrong character set for a little bit
 message in topic is fresh
 }}}

 A bit later, I encountered it myself:
 {{{
 sergey: @pento @dd32 The encoding issue still happens on
 https://ru.wordpress.org/support/,
 Cyrillic chars are randomly broken in some posts. Just happened again with
 this reply (1 min ago):
 https://ru.wordpress.org/support/topic/не-работает-встроенная-
 галлерея/#post-228395
 > Ñ ÐºÐ°ÐºÐ¾Ð¹ верÑии было обновление? в
 админку заходили?
 > Обновление базы данных предлагалоÑÑŒ
 (Сделали?) ?
 Clicking Edit and re-saving appears to have fixed it.
 Could it be that the fix has not propagated to all servers yet? (edited)

 pento: @sergey: I’ve re-deployed everything, just to check.
 I’m off to bed, but if there are still problems, get someone to roll it
 back and I’ll try again some other time.

 sergey: @pento: Thanks, will keep an eye on the forums :)
 }}}

 Still happens 12 hours later, see the latest screenshots.

 I don't see any related commits on https://meta.trac.wordpress.org/log/,
 so apparently it was a dotorg-specific change and it has to do with
 HyperDB (per one of the first messages).

 Could someone track it down or roll it back pending further investigation?

--
Ticket URL: <https://meta.trac.wordpress.org/ticket/2280>
Making WordPress.org <https://meta.trac.wordpress.org/>
Making WordPress.org