[wp-trac] [WordPress Trac] #30462: Test multiple MySQL versions on Travis

[WordPress Trac]

#30462: Test multiple MySQL versions on Travis
------------------------------+------------------------------
 Reporter:  pento             |       Owner:
     Type:  enhancement       |      Status:  new
 Priority:  normal            |   Milestone:  Awaiting Review
Component:  Build/Test Tools  |     Version:
 Severity:  normal            |  Resolution:
 Keywords:  has-patch         |     Focuses:
------------------------------+------------------------------

Comment (by netweb):

 Replying to [comment:7 pento]:
 > Going by some recent testing, it looks like we should try and cache the
 MySQL builds, or host them ourselves - sometimes the download is really
 slow. I'm not sure if that's the MySQL CDN or the Travis network, though.
 Did some research on this yesterday, caching is only available for "paid
 accounts" but we can work around this with some custom build scripts and
 hosting these files ourselves.

 ''This is beyond the scope of this ticket though and we should look at
 doing "something" with these databases as each DB is coming in at ~300mb
 per DB version! ''

 > As suggested by @netweb, we should test against MariaDB, as well. For
 reference, here's an approximate version match:
 >
 > MariaDB 5.1 = MySQL 5.1
 > MariaDB 5.2 = MySQL 5.1
 > MariaDB 5.3 = MySQL 5.1 + some of 5.5
 > MariaDB 5.5 = MySQL 5.5
 > MariaDB 10.0 = MySQL 5.5 + some of 5.6
 > MariaDB 10.1 (alpha) = MySQL 5.5 + most of 5.6 + some of 5.7
 >
 > As of 10.0, MariaDB is starting to diverge from MySQL more heavily. It
 still aims to be compatible, but incompatibilities will inevitably be
 introduced.
 >
 > I'm inclined to test against MariaDB 10.0 and 10.1, and also add MySQL
 5.7 for alpha testing. It's tricky to drag MariaDB usage out of our MySQL
 stats, but I'd expect MariaDB 5.x to be lower than 10.0, which is already
 pretty small.

 Agreed, I've got two patches in a state of "work in progress" tweaking and
 adjusting the build matrix for performance!

 What I have included thus far is:

   - DB=MySQL5.0
   - DB=MySQL5.1
   - DB=MySQL5.5
   - DB=MySQL5.6
   - DB=MariaDB5.5
   - DB=MariaDB10.0
   - DB=MySQL5.7 (Allowed to fail)

 With 6 versions of PHP we are currently testing against and the above 7
 databases brings us to 42 tests, if we add MariaDB 5.1, 5.2 and 5.3 to
 this we we end up at 60 jobs / build (Or 80 if we include HHVM & HHVM
 Nightly). MySQL Sandbox does not support, nor work, with MariaDB 10.1alpha
 at this stage ([https://travis-ci.org/ntwb/wordpress/jobs/41939687#L229
 src]).

 A sneak peek at this is here https://travis-
 ci.org/ntwb/wordpress/builds/41943470 (I'm not actually running PHPUnit
 tests here, these should be listed as `failing` and not `passing` BTW ;)

 Will upload both patches later this arvo, though I am leaning towards some
 "selective" combination of PHP/DB testing to keep the number of jobs per
 build much lower than 80, possibly lower than 40 for example only test
 MariaDB 5.3 against a single PHP version and not all versions of PHP.

--
Ticket URL: <https://core.trac.wordpress.org/ticket/30462#comment:8>
WordPress Trac <https://core.trac.wordpress.org/>
WordPress publishing platform