Options:
- # Session Start: Wed Oct 22 00:00:00 2014
- # Session Ident: #testing
- # [00:00] * Joins: bhill2 (~bhill2@public.cloak)
- # [00:00] * Quits: bhill2 (~bhill2@public.cloak) (Client closed connection)
- # [00:00] * Joins: bhill2 (~bhill2@public.cloak)
- # [00:12] * Joins: mdyck (~michael@public.cloak)
- # [00:19] * Parts: mdyck (~michael@public.cloak) (mdyck)
- # [00:20] * Joins: mdyck (~michael@public.cloak)
- # [00:20] * Parts: mdyck (~michael@public.cloak) (mdyck)
- # [00:21] * Joins: mdyck (~michael@public.cloak)
- # [00:22] * Quits: mdyck (~michael@public.cloak) ("Leaving.")
- # [00:22] * Joins: mdyck (~michael@public.cloak)
- # [00:22] * Quits: mdyck (~michael@public.cloak) ("Leaving.")
- # [00:32] * Quits: bhill2 (~bhill2@public.cloak) (Client closed connection)
- # [00:38] * Joins: bhill2 (~bhill2@public.cloak)
- # [00:43] * Joins: ptressel (~chatzilla@public.cloak)
- # [00:56] * Joins: MK (kawamori@public.cloak)
- # [01:09] * Quits: bhill2 (~bhill2@public.cloak) (Client closed connection)
- # [01:09] * Joins: bhill2 (~bhill2@public.cloak)
- # [01:13] * Joins: bhill2_ (~bhill2@public.cloak)
- # [01:17] * Joins: bhill2__ (~bhill2@public.cloak)
- # [01:17] * Quits: bhill2_ (~bhill2@public.cloak) (Client closed connection)
- # [01:19] * Quits: bhill2 (~bhill2@public.cloak) (Ping timeout: 180 seconds)
- # [01:40] * Quits: bhill2__ (~bhill2@public.cloak) (Client closed connection)
- # [01:43] * Joins: bhill2 (~bhill2@public.cloak)
- # [01:50] * Quits: bhill2 (~bhill2@public.cloak) (Client closed connection)
- # [01:55] * Quits: Lachy (~Lachy@public.cloak) ("My MacBook Pro has gone to sleep. ZZZzzz…")
- # [01:59] * Quits: MK (kawamori@public.cloak) (Ping timeout: 180 seconds)
- # [01:59] * Joins: Lachy (~Lachy@public.cloak)
- # [02:03] * Quits: AutomatedTester (~AutomatedTester@public.cloak) (Client closed connection)
- # [02:11] * Joins: MK (kawamori@public.cloak)
- # [02:12] * Quits: ArtB (~abarsto@public.cloak) ("Leaving.")
- # [02:55] * Joins: mdas (~mdas@public.cloak)
- # [03:05] * Quits: MK (kawamori@public.cloak) (Ping timeout: 180 seconds)
- # [03:09] * Quits: mdas (~mdas@public.cloak) (Client closed connection)
- # [03:16] * Joins: mdas (~mdas@public.cloak)
- # [03:25] * Joins: MK (kawamori@public.cloak)
- # [03:31] * Quits: mdas (~mdas@public.cloak) ("Leaving...")
- # [03:33] * Quits: MK (kawamori@public.cloak) (Ping timeout: 180 seconds)
- # [03:33] * Joins: MK (kawamori@public.cloak)
- # [03:53] * Quits: MK (kawamori@public.cloak) (Ping timeout: 180 seconds)
- # [04:04] * Joins: MK (kawamori@public.cloak)
- # [04:34] * Quits: MK (kawamori@public.cloak) (Ping timeout: 180 seconds)
- # [04:39] * Joins: MK (kawamori@public.cloak)
- # [05:59] * Quits: MK (kawamori@public.cloak) (Ping timeout: 180 seconds)
- # [07:29] * Joins: bhill2 (~bhill2@public.cloak)
- # [07:31] * Joins: MK (kawamori@public.cloak)
- # [08:06] * Joins: MK_ (kawamori@public.cloak)
- # [08:06] * Quits: MK (kawamori@public.cloak) (Ping timeout: 180 seconds)
- # [08:15] * Quits: MK_ (kawamori@public.cloak) (Ping timeout: 180 seconds)
- # [08:27] * Quits: bhill2 (~bhill2@public.cloak) (Client closed connection)
- # [08:30] * Joins: MK (kawamori@public.cloak)
- # [10:27] * Joins: bhill2 (~bhill2@public.cloak)
- # [10:37] * Quits: bhill2 (~bhill2@public.cloak) (Ping timeout: 180 seconds)
- # [10:55] * Quits: Lachy (~Lachy@public.cloak) ("My MacBook Pro has gone to sleep. ZZZzzz…")
- # [11:07] * Quits: MK (kawamori@public.cloak) (Ping timeout: 180 seconds)
- # [11:16] * Joins: Lachy (~Lachy@public.cloak)
- # [11:24] * Joins: Ms2ger (~Ms2ger@public.cloak)
- # [11:24] * Quits: Lachy (~Lachy@public.cloak) ("My MacBook Pro has gone to sleep. ZZZzzz…")
- # [11:24] * Joins: Lachy (~Lachy@public.cloak)
- # [11:25] * Quits: Lachy (~Lachy@public.cloak) ("Textual IRC Client: www.textualapp.com")
- # [11:33] * Joins: AutomatedTester (~AutomatedTester@public.cloak)
- # [11:34] * Quits: Ms2ger (~Ms2ger@public.cloak) (Ping timeout: 180 seconds)
- # [11:42] * Joins: darobin (rberjon@public.cloak)
- # [12:04] * Quits: AutomatedTester (~AutomatedTester@public.cloak) (Client closed connection)
- # [12:13] * Quits: darobin (rberjon@public.cloak) (Client closed connection)
- # [12:13] * Joins: darobin (rberjon@public.cloak)
- # [12:22] * Joins: abarsto (~abarsto@public.cloak)
- # [12:22] * abarsto is now known as ArtB
- # [12:33] * Joins: Lachy (~Lachy@public.cloak)
- # [12:36] * Quits: Lachy (~Lachy@public.cloak) ("Textual IRC Client: www.textualapp.com")
- # [12:36] * Joins: Lachy (~Lachy@public.cloak)
- # [12:55] * Joins: Lachy_ (~Lachy@public.cloak)
- # [12:58] * Quits: Lachy (~Lachy@public.cloak) (Client closed connection)
- # [13:14] * Quits: Lachy_ (~Lachy@public.cloak) (Ping timeout: 180 seconds)
- # [13:56] * Joins: Lachy (~Lachy@public.cloak)
- # [14:03] * Joins: plh (plehegar@public.cloak)
- # [14:05] * Joins: AutomatedTester (~AutomatedTester@public.cloak)
- # [14:27] * Quits: Lachy (~Lachy@public.cloak) (Client closed connection)
- # [14:27] * Joins: Lachy (~Lachy@public.cloak)
- # [14:28] <jgraham> MikeSmith: Thanks for the review; pushed a small fixup
- # [14:28] * Joins: darobin_ (rberjon@public.cloak)
- # [14:28] * Quits: darobin (rberjon@public.cloak) (Client closed connection)
- # [14:29] <jgraham> ato: Thanks
- # [14:30] * Joins: Ms2ger (~Ms2ger@public.cloak)
- # [14:35] <MikeSmith> jgraham: cheers & glad to help
- # [14:36] <MikeSmith> I plan to start spending more time again to help out with reviews and also writing up PRs
- # [14:41] <jgraham> Sounds good :)
- # [14:44] * Joins: mdas (~mdas@public.cloak)
- # [14:52] * Quits: Lachy (~Lachy@public.cloak) ("My MacBook Pro has gone to sleep. ZZZzzz…")
- # [14:53] * Joins: Lachy (~Lachy@public.cloak)
- # [15:06] <MikeSmith> jgraham: btw since you aren't going to be at the tech plenary day at TPAC to do a breakout session about the status of the wpt work, I will be channeling you there and attempting to lead such a session
- # [15:06] <jgraham> MikeSmith: OK
- # [15:07] <jgraham> Apologies for missing the plenary day; I would have come but I didn't know if my GTAC talk was Tuesday or Wednesday until very recently
- # [15:07] <jgraham> By which time I'd already booked flights
- # [15:11] <MikeSmith> jgraham: yeah I know
- # [15:13] <MikeSmith> jgraham: if it's OK with you what I'd like to do is just use some subset of your GTAC slides to talk about wptrunner and wptserve at the breakout session
- # [15:39] * plh is now known as plh-away
- # [15:41] * Joins: Lachy_ (~Lachy@public.cloak)
- # [15:45] * Quits: shepazu (schepers@public.cloak) ("is sleepy")
- # [15:48] * Quits: Lachy (~Lachy@public.cloak) (Ping timeout: 180 seconds)
- # [16:00] <jgraham> MikeSmith: I'm not too sure how useful my slides will be; they are pretty minimalist
- # [16:05] <MikeSmith> jgraham: I'll only have about 20 minute to present anyway
- # [16:05] <MikeSmith> or I'm only going to take 20 minutes
- # [16:05] <jgraham> OK
- # [16:05] * Joins: darobin (rberjon@public.cloak)
- # [16:05] * Quits: darobin_ (rberjon@public.cloak) (Client closed connection)
- # [16:06] <MikeSmith> I think the sessions are 45 minutes but I want to leave lots of time to actual discussion
- # [16:06] <jgraham> Yup, that makes sense in that scenario
- # [16:07] <MikeSmith> I was thinking mainly to talk about wptrunner and where that's currently at, and then plans for automation of "manual" tests with WebDriver
- # [16:08] <jgraham> Makes sense. I don't think the TPAC audience really care so much about the implementation
- # [16:15] <MikeSmith> right
- # [16:32] * Quits: Lachy_ (~Lachy@public.cloak) ("Textual IRC Client: www.textualapp.com")
- # [16:34] * Joins: Lachy (~Lachy@public.cloak)
- # [16:41] <MikeSmith> jgraham: what's the set of command-line args I need to invoke wptrunner with?
- # [16:41] <MikeSmith> I'm trying this:
- # [16:42] <MikeSmith> wptrunner --metadata /opt/workspace/web-platform-tests/ --tests /opt/workspace/web-platform-tests/ --binary /opt/workspace/mozilla-central/obj-x86_64-apple-darwin14.0.0/dist/Nightly.app/Contents/MacOS/firefox
- # [16:47] * Quits: darobin (rberjon@public.cloak) (Client closed connection)
- # [16:48] * Joins: darobin (rberjon@public.cloak)
- # [16:48] <jgraham> MikeSmith: So for gecko it's kind of much easier if you have a source tree, because it turns on crash-if-you-access-external-resources and a default gecko profile pings lots of services (e.g. the malware thing)
- # [16:49] <jgraham> So you need a prefs file that disables all the relevant things
- # [16:49] <MikeSmith> I have a source tree
- # [16:49] <jgraham> MikeSmith: |mach web-platform-tests|
- # [16:49] <MikeSmith> oh
- # [16:50] * Quits: Lachy (~Lachy@public.cloak) ("Textual IRC Client: www.textualapp.com")
- # [16:50] <MikeSmith> well hot damn
- # [16:50] <MikeSmith> that's pretty cool
- # [16:51] * Joins: Lachy (~Lachy@public.cloak)
- # [16:51] <MikeSmith> and fast
- # [16:52] <jgraham> I would rate it as "could be faster" :)
- # [16:52] <jgraham> Closing and reopening the window all the time is pretty slow, but I decided it probably helped with stability
- # [16:53] <jgraham> And marionette can be slow (e.g. there's a 5s pause on startup)
- # [16:54] <MikeSmith> ok
- # [16:55] <MikeSmith> so yeah why is it closing and reopening the window?
- # [16:55] <MikeSmith> how does it determine at which point to do that?
- # [16:55] * Quits: darobin (rberjon@public.cloak) (Ping timeout: 180 seconds)
- # [16:56] <jgraham> Well, what do you mean?
- # [16:56] <jgraham> The tab gets closed after each test
- # [16:57] <jgraham> The whole window gets closed if you got an unexpected result
- # [16:59] <MikeSmith> ah
- # [16:59] <MikeSmith> that's what I meant I guess
- # [17:00] <MikeSmith> I just notice that it was closing the whole window several times while runnign the 2dcontext tests
- # [17:01] <jgraham> So that suggests you got an unexpected result. That can happen if your os isn't exactly the same as the one we have on the test machines
- # [17:01] <MikeSmith> ah ok
- # [17:01] <MikeSmith> well I'm running OS X 10.10
- # [17:02] <jgraham> Yeah, that's not something we have
- # [17:02] <MikeSmith> ok
- # [17:02] <jgraham> The problem is that the automatic "update the metadata" script isn't clever enough to know when something applies to all versions of OSX (for example) or when it's version specific
- # [17:02] <MikeSmith> I see
- # [17:03] <jgraham> So it assumes that everything is specific to the exact configuration it was using, which isn't right
- # [17:03] <MikeSmith> ok
- # [17:03] <jgraham> (but it's hard to do really well without adding knowledge of all the different configurations in use)
- # [17:03] <MikeSmith> right
- # [17:06] <MikeSmith> so another question, is wptrunner usable with chrome now? have you tried it?
- # [17:06] <ato> It worked fine last time I tried it.
- # [17:06] <ato> Which is a few weeks ago.
- # [17:07] <ato> It also works with Servo \o/
- # [17:09] <jgraham> MikeSmith: Yeah, works with Chrome. There's even a video of that in the talk
- # [17:10] <jgraham> Currently Gecko+Servo+FirefoxOS+Chrome work. IE had some patches that have likely bitrotted a little (we should talk to Erika about that). Safari-via-Selenium would likely to be trivial to support.
- # [17:10] <jgraham> Opera doesn't have any useful remote control protocol at the moment afaik
- # [17:10] <ato> Note that Selenium support for Safari isn't great.
- # [17:11] <ato> But for the limitations of what wpt does, it should be fine.
- # [17:11] <jgraham> Yeah, wpt really doesn't do too much advanced stuff
- # [17:11] <ato> jgraham: Opera submitted a patch to Selenium for some changes to the Python client yesterday, so presumably they have something in the works.
- # [17:12] <MikeSmith> ok
- # [17:18] <MikeSmith> so to run Chromium what wptrunner invocation should I use
- # [17:18] <MikeSmith> I'm trying:
- # [17:18] <MikeSmith> wptrunner --product=chrome --binary=/opt/workspace/chromium/src/out/Release/Chromium.app/Contents/MacOS/Chromium
- # [17:18] <jgraham> You need chromedriver
- # [17:19] <jgraham> http://chromedriver.storage.googleapis.com/index.html
- # [17:19] * Quits: Lachy (~Lachy@public.cloak) ("My MacBook Pro has gone to sleep. ZZZzzz…")
- # [17:19] <MikeSmith> oh
- # [17:20] <MikeSmith> you mean I have it point it at my chromedriver binary?
- # [17:20] <jgraham> Yeah it might not be terribly well documented
- # [17:20] <MikeSmith> not the browser binary
- # [17:20] <jgraham> Right
- # [17:20] <jgraham> Chromedriver has to launch the browser
- # [17:20] <MikeSmith> nah yeah it makes perfect sense
- # [17:20] <MikeSmith> yeah of course
- # [17:20] <MikeSmith> I just wasn't thinking about it
- # [17:20] <jgraham> Well not really "of course" but that's how Google designed it :)
- # [17:21] <ato> We should change that so that it picks chromedriver up from PATH and sets the browser binary capability with the value from --binary instead.
- # [17:22] <jgraham> That would make some sense
- # [17:22] <ato> Actually let me just go do that.
- # [17:24] * Quits: Ms2ger (~Ms2ger@public.cloak) (Ping timeout: 180 seconds)
- # [18:01] * Joins: bhill2 (~bhill2@public.cloak)
- # [18:01] * Quits: bhill2 (~bhill2@public.cloak) (Client closed connection)
- # [18:01] * Joins: bhill2 (~bhill2@public.cloak)
- # [18:15] <ato> jgraham: Any idea where this might come from? https://gist.github.com/andreastt/c44ffd58b82629a5a849
- # [18:15] <ato> wptrunner and wpt master.
- # [18:21] <jgraham> ato: Delete your MANIFEST.json
- # [18:22] <ato> Okay, now I get this: https://gist.github.com/andreastt/d38867adfec777c73a20
- # [18:22] <jgraham> Oh, I thought I fixed that
- # [18:23] * ato realises he should run it more often
- # [18:23] <ato> Also we need tests.
- # [18:26] * Joins: gitbot (~gitbot@public.cloak)
- # [18:26] -gitbot:#testing- [web-platform-tests] jgraham created jgraham/changes_clean_directory from master (+0 new commits): https://github.com/w3c/web-platform-tests/commits/jgraham/changes_clean_directory
- # [18:26] * Parts: gitbot (~gitbot@public.cloak) (gitbot)
- # [18:27] * Joins: gitbot (~gitbot@public.cloak)
- # [18:27] -gitbot:#testing- [web-platform-tests] jgraham pushed 1 new commit to jgraham/changes_clean_directory: https://github.com/w3c/web-platform-tests/commit/02e1b618a8a80f23f5b22cddec81a71f63a4418e
- # [18:27] -gitbot:#testing- web-platform-tests/jgraham/changes_clean_directory 02e1b61 James Graham: Fix crash creating manifest when working directory doesn't have any changes
- # [18:27] * Parts: gitbot (~gitbot@public.cloak) (gitbot)
- # [18:27] * Joins: gitbot (~gitbot@public.cloak)
- # [18:27] -gitbot:#testing- [web-platform-tests] jgraham opened pull request #1301: Fix crash creating manifest when working directory doesn't have any changes (master...jgraham/changes_clean_directory) https://github.com/w3c/web-platform-tests/pull/1301
- # [18:27] * Parts: gitbot (~gitbot@public.cloak) (gitbot)
- # [18:27] <jgraham> Try the branch in https://github.com/w3c/web-platform-tests/pull/1301
- # [18:28] <jgraham> And yeah, tests would help but this would be pretty nasty to write a test for, I think
- # [18:28] <jgraham> It depends on the state of the working directory
- # [18:29] <ato> That seems to work fine.
- # [18:29] <ato> I'll go ahead and review.
- # [18:29] * Quits: bhill2 (~bhill2@public.cloak) (Client closed connection)
- # [18:30] * Joins: gitbot (~gitbot@public.cloak)
- # [18:30] -gitbot:#testing- [web-platform-tests] andreastt closed pull request #1301: Fix crash creating manifest when working directory doesn't have any changes (master...jgraham/changes_clean_directory) https://github.com/w3c/web-platform-tests/pull/1301
- # [18:30] * Parts: gitbot (~gitbot@public.cloak) (gitbot)
- # [18:30] <jgraham> Thanks, and sorry for missing it
- # [18:30] <ato> We should have caught it in review.
- # [18:33] * Joins: bhill2 (~bhill2@public.cloak)
- # [18:36] * Joins: Ms2ger (~Ms2ger@public.cloak)
- # [18:39] <ato> jgraham: There's no way to do an init_fail from the context of a browser class right?
- # [18:42] * Joins: Lachy (~Lachy@public.cloak)
- # [18:43] <jgraham> No
- # [18:43] <jgraham> Well maybe, I would have to look at the code. But iirc it's not trivial
- # [18:45] <ato> In testrunner.py:335 I'm wondering if we should special case an Exception object to call init_failed.
- # [18:45] <jgraham> ato: I have to go in a minute but I will have a look later
- # [18:46] <ato> What happens now is that it'll loop three times, which is unnecessary if you know that you can't recover from something.
- # [18:46] <ato> But I guess there's no harm in retrying either.
- # [18:46] <jgraham> Yeah, that could be better
- # [19:01] * Quits: bhill2 (~bhill2@public.cloak) (Client closed connection)
- # [19:05] * plh-away is now known as plh
- # [19:07] * Joins: bhill2 (~bhill2@public.cloak)
- # [19:19] * Joins: Lachy_ (~Lachy@public.cloak)
- # [19:22] * Joins: Lachy__ (~Lachy@public.cloak)
- # [19:25] * Quits: Lachy (~Lachy@public.cloak) (Ping timeout: 180 seconds)
- # [19:28] * Quits: Lachy_ (~Lachy@public.cloak) (Ping timeout: 180 seconds)
- # [19:52] * Quits: Lachy__ (~Lachy@public.cloak) ("My MacBook Pro has gone to sleep. ZZZzzz…")
- # [20:01] * Quits: bhill2 (~bhill2@public.cloak) (Client closed connection)
- # [20:09] * Joins: bhill2 (~bhill2@public.cloak)
- # [20:09] * Quits: bhill2 (~bhill2@public.cloak) (Client closed connection)
- # [20:11] * Joins: bhill2 (~bhill2@public.cloak)
- # [20:15] * Quits: ArtB (~abarsto@public.cloak) ("Leaving.")
- # [20:17] * Quits: bhill2 (~bhill2@public.cloak) (Client closed connection)
- # [20:17] * Joins: bhill2 (~bhill2@public.cloak)
- # [20:18] * Joins: Lachy (~Lachy@public.cloak)
- # [20:19] * Quits: Lachy (~Lachy@public.cloak) ("My MacBook Pro has gone to sleep. ZZZzzz…")
- # [20:27] * Joins: bhill2_ (~bhill2@public.cloak)
- # [20:27] * Quits: bhill2 (~bhill2@public.cloak) (Client closed connection)
- # [20:28] * Quits: bhill2_ (~bhill2@public.cloak) (Client closed connection)
- # [20:28] * Joins: bhill2 (~bhill2@public.cloak)
- # [20:29] * Quits: bhill2 (~bhill2@public.cloak) (Client closed connection)
- # [20:29] * Joins: bhill2 (~bhill2@public.cloak)
- # [20:40] * Joins: abarsto (~abarsto@public.cloak)
- # [20:40] * abarsto is now known as ArtB
- # [20:47] * Joins: Lachy (~Lachy@public.cloak)
- # [21:01] * Joins: bhill2_ (~bhill2@public.cloak)
- # [21:01] * Quits: bhill2 (~bhill2@public.cloak) (Client closed connection)
- # [21:22] <ato> jgraham: Will wptrunner stop the browser between each test?
- # [21:23] * Quits: bhill2_ (~bhill2@public.cloak) (Client closed connection)
- # [21:24] * Joins: bhill2 (~bhill2@public.cloak)
- # [21:30] * Quits: bhill2 (~bhill2@public.cloak) (Client closed connection)
- # [21:30] <jgraham> ato: No, it stops the browser after each test that gets an unexpected result
- # [21:31] <ato> So fail for an expected pass for example?
- # [21:31] <ato> And by default there are no expected results in the wpt repo, at least not for Chrome?
- # [21:39] * Joins: bhill2 (~bhill2@public.cloak)
- # [21:42] * Quits: bhill2 (~bhill2@public.cloak) (Client closed connection)
- # [21:44] <Ms2ger> ato, correct
- # [21:45] <ato> Ms2ger: Thanks
- # [21:47] * Joins: bhill2 (~bhill2@public.cloak)
- # [21:54] * ato curses mozprocess
- # [22:08] * Quits: scott_gonzalez (~scott_gonzalez@public.cloak) (scott_gonzalez)
- # [22:22] <ato> MikeSmith: It's not perfect: https://critic.hoppipolla.co.uk/r/2949
- # [22:36] * Joins: scott_gonzalez (~scott_gonzalez@public.cloak)
- # [22:54] * Quits: Lachy (~Lachy@public.cloak) ("My MacBook Pro has gone to sleep. ZZZzzz…")
- # [22:57] * Joins: Lachy (~Lachy@public.cloak)
- # [23:01] * Quits: bhill2 (~bhill2@public.cloak) (Client closed connection)
- # [23:12] * Joins: bhill2 (~bhill2@public.cloak)
- # [23:15] * Joins: Lachy_ (~Lachy@public.cloak)
- # [23:19] * Joins: Lachy__ (~Lachy@public.cloak)
- # [23:20] * Quits: Lachy (~Lachy@public.cloak) (Client closed connection)
- # [23:22] * Quits: scott_gonzalez (~scott_gonzalez@public.cloak) (scott_gonzalez)
- # [23:24] * Quits: Lachy_ (~Lachy@public.cloak) (Ping timeout: 180 seconds)
- # [23:58] * Quits: bhill2 (~bhill2@public.cloak) (Ping timeout: 180 seconds)
- # [23:59] * Joins: bhill2 (~bhill2@public.cloak)
- # [23:59] * Joins: MK (kawamori@public.cloak)
- # Session Close: Thu Oct 23 00:00:00 2014
The end :)