<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN"
"http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head><meta http-equiv="content-type" content="text/html; charset=utf-8" />
<title>[34848] trunk/tests/phpunit: HTTP timeouts should cause some tests to be skipped, not failed</title>
</head>
<body>
<style type="text/css"><!--
#msg dl.meta { border: 1px #006 solid; background: #369; padding: 6px; color: #fff; }
#msg dl.meta dt { float: left; width: 6em; font-weight: bold; }
#msg dt:after { content:':';}
#msg dl, #msg dt, #msg ul, #msg li, #header, #footer, #logmsg { font-family: verdana,arial,helvetica,sans-serif; font-size: 10pt; }
#msg dl a { font-weight: bold}
#msg dl a:link { color:#fc3; }
#msg dl a:active { color:#ff0; }
#msg dl a:visited { color:#cc6; }
h3 { font-family: verdana,arial,helvetica,sans-serif; font-size: 10pt; font-weight: bold; }
#msg pre { overflow: auto; background: #ffc; border: 1px #fa0 solid; padding: 6px; }
#logmsg { background: #ffc; border: 1px #fa0 solid; padding: 1em 1em 0 1em; }
#logmsg p, #logmsg pre, #logmsg blockquote { margin: 0 0 1em 0; }
#logmsg p, #logmsg li, #logmsg dt, #logmsg dd { line-height: 14pt; }
#logmsg h1, #logmsg h2, #logmsg h3, #logmsg h4, #logmsg h5, #logmsg h6 { margin: .5em 0; }
#logmsg h1:first-child, #logmsg h2:first-child, #logmsg h3:first-child, #logmsg h4:first-child, #logmsg h5:first-child, #logmsg h6:first-child { margin-top: 0; }
#logmsg ul, #logmsg ol { padding: 0; list-style-position: inside; margin: 0 0 0 1em; }
#logmsg ul { text-indent: -1em; padding-left: 1em; }#logmsg ol { text-indent: -1.5em; padding-left: 1.5em; }
#logmsg > ul, #logmsg > ol { margin: 0 0 1em 0; }
#logmsg pre { background: #eee; padding: 1em; }
#logmsg blockquote { border: 1px solid #fa0; border-left-width: 10px; padding: 1em 1em 0 1em; background: white;}
#logmsg dl { margin: 0; }
#logmsg dt { font-weight: bold; }
#logmsg dd { margin: 0; padding: 0 0 0.5em 0; }
#logmsg dd:before { content:'\00bb';}
#logmsg table { border-spacing: 0px; border-collapse: collapse; border-top: 4px solid #fa0; border-bottom: 1px solid #fa0; background: #fff; }
#logmsg table th { text-align: left; font-weight: normal; padding: 0.2em 0.5em; border-top: 1px dotted #fa0; }
#logmsg table td { text-align: right; border-top: 1px dotted #fa0; padding: 0.2em 0.5em; }
#logmsg table thead th { text-align: center; border-bottom: 1px solid #fa0; }
#logmsg table th.Corner { text-align: left; }
#logmsg hr { border: none 0; border-top: 2px dashed #fa0; height: 1px; }
#header, #footer { color: #fff; background: #636; border: 1px #300 solid; padding: 6px; }
#patch { width: 100%; }
#patch h4 {font-family: verdana,arial,helvetica,sans-serif;font-size:10pt;padding:8px;background:#369;color:#fff;margin:0;}
#patch .propset h4, #patch .binary h4 {margin:0;}
#patch pre {padding:0;line-height:1.2em;margin:0;}
#patch .diff {width:100%;background:#eee;padding: 0 0 10px 0;overflow:auto;}
#patch .propset .diff, #patch .binary .diff {padding:10px 0;}
#patch span {display:block;padding:0 10px;}
#patch .modfile, #patch .addfile, #patch .delfile, #patch .propset, #patch .binary, #patch .copfile {border:1px solid #ccc;margin:10px 0;}
#patch ins {background:#dfd;text-decoration:none;display:block;padding:0 10px;}
#patch del {background:#fdd;text-decoration:none;display:block;padding:0 10px;}
#patch .lines, .info {color:#888;background:#fff;}
--></style>
<div id="msg">
<dl class="meta" style="font-size: 105%">
<dt style="float: left; width: 6em; font-weight: bold">Revision</dt> <dd><a style="font-weight: bold" href="https://core.trac.wordpress.org/changeset/34848">34848</a><script type="application/ld+json">{"@context":"http://schema.org","@type":"EmailMessage","description":"Review this Commit","action":{"@type":"ViewAction","url":"https://core.trac.wordpress.org/changeset/34848","name":"Review Commit"}}</script></dd>
<dt style="float: left; width: 6em; font-weight: bold">Author</dt> <dd>jorbin</dd>
<dt style="float: left; width: 6em; font-weight: bold">Date</dt> <dd>2015-10-06 03:36:18 +0000 (Tue, 06 Oct 2015)</dd>
</dl>
<pre style='padding-left: 1em; margin: 2em 0; border-left: 2px solid #ccc; line-height: 1.25; font-size: 105%; font-family: sans-serif'>HTTP timeouts should cause some tests to be skipped, not failed
A number of the HTTP external tests can inconstantly fail. As the HTTP API is one that doesn't change often, this failure creates noise. With the goal of increasing the signal from the unit tests, these tests are now skipped if they timeout. A notice is added when running the external http tests so that the developer knows what skipped tests may mean here.
See <a href="https://core.trac.wordpress.org/ticket/33968">#33968</a></pre>
<h3>Modified Paths</h3>
<ul>
<li><a href="#trunktestsphpunitincludesbootstrapphp">trunk/tests/phpunit/includes/bootstrap.php</a></li>
<li><a href="#trunktestsphpunittestshttpbasephp">trunk/tests/phpunit/tests/http/base.php</a></li>
</ul>
</div>
<div id="patch">
<h3>Diff</h3>
<a id="trunktestsphpunitincludesbootstrapphp"></a>
<div class="modfile"><h4 style="background-color: #eee; color: inherit; margin: 1em 0; padding: 1.3em; font-size: 115%">Modified: trunk/tests/phpunit/includes/bootstrap.php</h4>
<pre class="diff"><span>
<span class="info" style="display: block; padding: 0 10px; color: #888">--- trunk/tests/phpunit/includes/bootstrap.php 2015-10-06 03:35:14 UTC (rev 34847)
+++ trunk/tests/phpunit/includes/bootstrap.php 2015-10-06 03:36:18 UTC (rev 34848)
</span><span class="lines" style="display: block; padding: 0 10px; color: #888">@@ -164,6 +164,13 @@
</span><span class="cx" style="display: block; padding: 0 10px"> foreach ( $skipped_groups as $group_name => $skipped ) {
</span><span class="cx" style="display: block; padding: 0 10px"> echo sprintf( 'Not running %1$s tests. To execute these, use --group %1$s.', $group_name ) . PHP_EOL;
</span><span class="cx" style="display: block; padding: 0 10px"> }
</span><ins style="background-color: #dfd; text-decoration:none; display:block; padding: 0 10px">+
+ if ( ! isset( $skipped_groups['external-http'] ) ){
+ echo PHP_EOL;
+ echo 'External HTTP skipped tests can be caused by timeouts.' . PHP_EOL;
+ echo 'If this changeset inclues changes to HTTP, make sure there are no timeouts.' . PHP_EOL;
+ echo PHP_EOL;
+ }
</ins><span class="cx" style="display: block; padding: 0 10px"> }
</span><span class="cx" style="display: block; padding: 0 10px"> }
</span><span class="cx" style="display: block; padding: 0 10px"> new WP_PHPUnit_Util_Getopt( $_SERVER['argv'] );
</span></span></pre></div>
<a id="trunktestsphpunittestshttpbasephp"></a>
<div class="modfile"><h4 style="background-color: #eee; color: inherit; margin: 1em 0; padding: 1.3em; font-size: 115%">Modified: trunk/tests/phpunit/tests/http/base.php</h4>
<pre class="diff"><span>
<span class="info" style="display: block; padding: 0 10px; color: #888">--- trunk/tests/phpunit/tests/http/base.php 2015-10-06 03:35:14 UTC (rev 34847)
+++ trunk/tests/phpunit/tests/http/base.php 2015-10-06 03:36:18 UTC (rev 34848)
</span><span class="lines" style="display: block; padding: 0 10px; color: #888">@@ -17,6 +17,27 @@
</span><span class="cx" style="display: block; padding: 0 10px">
</span><span class="cx" style="display: block; padding: 0 10px"> protected $http_request_args;
</span><span class="cx" style="display: block; padding: 0 10px">
</span><ins style="background-color: #dfd; text-decoration:none; display:block; padding: 0 10px">+ /**
+ * Mark test as skipped if the HTTP request times out
+ */
+ function skipTestOnTimeout( $response ) {
+ if( ! is_wp_error( $response ) ){
+ return;
+ }
+ if ( 'connect() timed out!' === $response->get_error_message() ){
+ $this->markTestSkipped( 'HTTP timeout' );
+ }
+
+ if ( 0 === strpos( $response->get_error_message(), 'Operation timed out after' ) ){
+ $this->markTestSkipped( 'HTTP timeout' );
+ }
+
+ if ( 'stream_socket_client(): unable to connect to tcp://s.w.org:80 (Connection timed out)' === $response->get_error_message() ){
+ $this->markTestSkipped( 'HTTP timeout' );
+ }
+
+ }
+
</ins><span class="cx" style="display: block; padding: 0 10px"> function setUp() {
</span><span class="cx" style="display: block; padding: 0 10px">
</span><span class="cx" style="display: block; padding: 0 10px"> if ( is_callable( array('WP_Http', '_getTransport') ) ) {
</span><span class="lines" style="display: block; padding: 0 10px; color: #888">@@ -198,6 +219,8 @@
</span><span class="cx" style="display: block; padding: 0 10px"> unlink( $res['filename'] );
</span><span class="cx" style="display: block; padding: 0 10px"> }
</span><span class="cx" style="display: block; padding: 0 10px">
</span><ins style="background-color: #dfd; text-decoration:none; display:block; padding: 0 10px">+ $this->skipTestOnTimeout ($res );
+
</ins><span class="cx" style="display: block; padding: 0 10px"> $this->assertNotWPError( $res );
</span><span class="cx" style="display: block; padding: 0 10px"> $this->assertEquals( '', $res['body'] ); // The body should be empty.
</span><span class="cx" style="display: block; padding: 0 10px"> $this->assertEquals( $size, $res['headers']['content-length'] ); // Check the headers are returned (and the size is the same..)
</span><span class="lines" style="display: block; padding: 0 10px; color: #888">@@ -219,6 +242,8 @@
</span><span class="cx" style="display: block; padding: 0 10px"> unlink( $res['filename'] );
</span><span class="cx" style="display: block; padding: 0 10px"> }
</span><span class="cx" style="display: block; padding: 0 10px">
</span><ins style="background-color: #dfd; text-decoration:none; display:block; padding: 0 10px">+ $this->skipTestOnTimeout ($res );
+
</ins><span class="cx" style="display: block; padding: 0 10px"> $this->assertNotWPError( $res );
</span><span class="cx" style="display: block; padding: 0 10px"> $this->assertEquals( $size, $filesize ); // Check that the file is written to disk correctly without any extra characters
</span><span class="cx" style="display: block; padding: 0 10px">
</span><span class="lines" style="display: block; padding: 0 10px; color: #888">@@ -235,6 +260,8 @@
</span><span class="cx" style="display: block; padding: 0 10px">
</span><span class="cx" style="display: block; padding: 0 10px"> $res = wp_remote_request( $url, array( 'timeout' => 30, 'limit_response_size' => $size ) );
</span><span class="cx" style="display: block; padding: 0 10px">
</span><ins style="background-color: #dfd; text-decoration:none; display:block; padding: 0 10px">+ $this->skipTestOnTimeout ($res );
+
</ins><span class="cx" style="display: block; padding: 0 10px"> $this->assertNotWPError( $res );
</span><span class="cx" style="display: block; padding: 0 10px"> $this->assertEquals( $size, strlen( $res['body'] ) );
</span><span class="cx" style="display: block; padding: 0 10px"> }
</span></span></pre>
</div>
</div>
</body>
</html>