Options:
- # Session Start: Mon May 19 00:00:00 2014
- # Session Ident: #testing
- # [00:07] * Quits: AutomatedTester (~AutomatedTester@public.cloak) (Client closed connection)
- # [01:17] * Joins: AutomatedTester (~AutomatedTester@public.cloak)
- # [01:24] * Quits: AutomatedTester (~AutomatedTester@public.cloak) (Ping timeout: 180 seconds)
- # [02:12] * Joins: zcorpan (~zcorpan@public.cloak)
- # [02:15] * Joins: glenn_ (~glenn@public.cloak)
- # [02:21] * glenn_ is now known as glenn
- # [02:24] * Joins: plh (~plh@public.cloak)
- # [02:47] * astearns_ is now known as astearns
- # [02:52] * Joins: plinss_ (~plinss@public.cloak)
- # [02:53] * Quits: plinss_ (~plinss@public.cloak) (plinss_)
- # [03:32] * Joins: lmclister (~lmclister@public.cloak)
- # [03:55] * Joins: gitbot (~gitbot@public.cloak)
- # [03:55] -gitbot:#testing- [web-platform-tests] sideshowbarker pushed 24 new commits to master: https://github.com/w3c/web-platform-tests/compare/2911db9ba6c4...5adbfa2aba26
- # [03:55] -gitbot:#testing- web-platform-tests/master 92b954f Denis Ah-Kang: already tested
- # [03:55] -gitbot:#testing- web-platform-tests/master 83cf850 Denis Ah-Kang: type hidden is barred from constraint validation
- # [03:55] -gitbot:#testing- web-platform-tests/master 5d4cbe5 Denis Ah-Kang: rewrite tests on type email
- # [03:55] * Parts: gitbot (~gitbot@public.cloak) (gitbot)
- # [03:59] * Joins: gitbot (~gitbot@public.cloak)
- # [03:59] -gitbot:#testing- [web-platform-tests] sideshowbarker closed pull request #773: tests on input types {email,password,number,checkbox,radio,file,image,reset,button} (master...input-element) https://github.com/w3c/web-platform-tests/pull/773
- # [03:59] * Parts: gitbot (~gitbot@public.cloak) (gitbot)
- # [05:56] * Quits: lmclister (~lmclister@public.cloak) ("")
- # [06:02] * Joins: lmclister (~lmclister@public.cloak)
- # [07:25] * Quits: lmclister (~lmclister@public.cloak) ("")
- # [07:32] * Joins: lmclister (~lmclister@public.cloak)
- # [07:52] * Quits: lmclister (~lmclister@public.cloak) ("")
- # [08:43] * Quits: Lachy (~Lachy@public.cloak) ("My MacBook Pro has gone to sleep. ZZZzzz…")
- # [08:54] * Joins: Lachy (~Lachy@public.cloak)
- # [09:11] * Quits: Lachy (~Lachy@public.cloak) ("My MacBook Pro has gone to sleep. ZZZzzz…")
- # [09:20] * Joins: Ms2ger (~Ms2ger@public.cloak)
- # [09:23] * Joins: Lachy (~Lachy@public.cloak)
- # [09:33] * Joins: darobin (rberjon@public.cloak)
- # [09:41] * Quits: Lachy (~Lachy@public.cloak) ("My MacBook Pro has gone to sleep. ZZZzzz…")
- # [09:42] * Joins: Lachy (~Lachy@public.cloak)
- # [10:14] * Quits: Lachy (~Lachy@public.cloak) ("My MacBook Pro has gone to sleep. ZZZzzz…")
- # [10:14] * Joins: Lachy (~Lachy@public.cloak)
- # [10:22] * Quits: Lachy (~Lachy@public.cloak) ("My MacBook Pro has gone to sleep. ZZZzzz…")
- # [10:25] * Quits: plh (~plh@public.cloak) ("Page closed")
- # [10:27] * Quits: glenn (~glenn@public.cloak) ("Page closed")
- # [10:35] * Joins: anssik (~uid10742@public.cloak)
- # [10:45] * Quits: zcorpan (~zcorpan@public.cloak) (Ping timeout: 180 seconds)
- # [11:04] * Joins: Lachy (~Lachy@public.cloak)
- # [11:16] * Joins: gitbot (~gitbot@public.cloak)
- # [11:16] -gitbot:#testing- [web-platform-tests] sgrekhov opened pull request #995: Pull request for Animation timeline tests (master...submission/unipro-web-animations-01) https://github.com/w3c/web-platform-tests/pull/995
- # [11:16] * Parts: gitbot (~gitbot@public.cloak) (gitbot)
- # [11:31] * Joins: AutomatedTester (~AutomatedTester@public.cloak)
- # [11:44] * Joins: gitbot (~gitbot@public.cloak)
- # [11:44] -gitbot:#testing- [web-platform-tests] AutomatedTester pushed 1 new commit to master: https://github.com/w3c/web-platform-tests/commit/a0b890471897dc4e47595c1e6c277d58022d3b6a
- # [11:44] -gitbot:#testing- web-platform-tests/master a0b8904 Seva Lotoshnikov: BODY element visibility: must always be displayed no matter what...
- # [11:44] * Parts: gitbot (~gitbot@public.cloak) (gitbot)
- # [11:48] * Joins: gitbot (~gitbot@public.cloak)
- # [11:48] -gitbot:#testing- [web-platform-tests] AutomatedTester pushed 1 new commit to master: https://github.com/w3c/web-platform-tests/commit/4ae33e059cdc2d6fc1ef3017dc056067a60cb56f
- # [11:48] -gitbot:#testing- web-platform-tests/master 4ae33e0 Seva Lotoshnikov: Replacing use of find_element_by_tag_name that doesn't exist in the spec...
- # [11:48] * Parts: gitbot (~gitbot@public.cloak) (gitbot)
- # [12:11] * Quits: AutomatedTester (~AutomatedTester@public.cloak) (Client closed connection)
- # [12:32] * Joins: abarsto (~abarsto@public.cloak)
- # [12:32] * abarsto is now known as ArtB
- # [13:31] * Joins: scott_gonzalez (~scott_gonzalez@public.cloak)
- # [13:46] * Joins: zcorpan (~zcorpan@public.cloak)
- # [14:24] * Quits: Lachy (~Lachy@public.cloak) ("My MacBook Pro has gone to sleep. ZZZzzz…")
- # [14:28] * Joins: Lachy (~Lachy@public.cloak)
- # [14:37] * Quits: Lachy (~Lachy@public.cloak) ("My MacBook Pro has gone to sleep. ZZZzzz…")
- # [14:42] * Joins: Lachy (~Lachy@public.cloak)
- # [15:31] * Joins: AutomatedTester (~AutomatedTester@public.cloak)
- # [15:38] <jgraham> Does http://hoppipolla.co.uk/410/wptrunner.svg make the smallest bit of sense to anyone?
- # [15:38] <jgraham> Ms2ger, darobin, AutomatedTester ^
- # [15:39] <darobin> jgraham: I never understand diagrams, at all
- # [15:40] <Ms2ger> Fancy
- # [15:40] <Ms2ger> DUnno
- # [15:41] <AutomatedTester> jgraham: what do the different arrow types mean?
- # [15:42] <Ms2ger> See, there's the manager
- # [15:42] <AutomatedTester> and the different line types
- # [15:43] <darobin> they point at things!
- # [15:43] <jgraham> darobin: I have a feeling there is some secret code I am supposed to use for this kind of diagram that is passed down to people in a ritual trance-like state by their Computer Engineering course. But since I never got indoctrinated into that special club, I don't actually know how to make "proper" diagrams. So your lack of understanding is matched neatly by my own.
- # [15:43] <darobin> jgraham: ah, so it isn't just me
- # [15:44] <darobin> jgraham: well, from my point of view, if it's any reassurance, it looked a lot like the other diagrams that I don't understand :)
- # [15:44] <jgraham> AutomatedTester: the dashed lines are things that happen cross-process in some way or another
- # [15:44] <jgraham> The dotted line is something that wasn't a normal line or a dashed line
- # [15:45] <jgraham> The open arrow was the closest thing to a SQL 1:many type arrow this website had
- # [15:45] <jgraham> Except it is very unlike that and doesn't communicate the right thing at all
- # [15:45] <darobin> lol
- # [15:46] <darobin> jgraham: may I ask why you are dragging yourself through this?
- # [15:46] <jgraham> darobin: Because the alternative was 1000 words of the form "the ManagerGroup bone is connected to the TestRunnerManager bone"
- # [15:47] <darobin> I guess that's the proximate cause; I was asking about a more ulimate one
- # [15:47] <darobin> *ultimate
- # [15:50] <jgraham> Well partly because one piece of review feedback was basically "I don't understand how this all fits together". And partly because the amount of review feedback per line of code strongly suggests that was the case.
- # [15:50] <jgraham> (i.e. it suffered from the nuclear power plant problem)
- # [15:50] <darobin> okay, fair enough
- # [15:53] * Joins: gitbot (~gitbot@public.cloak)
- # [15:53] -gitbot:#testing- [web-platform-tests] zcorpan opened pull request #996: Test img loading. WIP (master...img-bug-24711) https://github.com/w3c/web-platform-tests/pull/996
- # [15:53] * Parts: gitbot (~gitbot@public.cloak) (gitbot)
- # [15:55] <jgraham> So the range of opinions there were "dunno", "are you crazy?" and "this needs a key".
- # [16:04] <darobin> I think the last one is the important one
- # [16:05] <jgraham> Yes, I think that's true
- # [16:05] <jgraham> Maybe I should have done that before asking for feedback
- # [16:09] * Joins: MK (kawamori@public.cloak)
- # [16:12] * Quits: AutomatedTester (~AutomatedTester@public.cloak) (Client closed connection)
- # [16:33] * Quits: zcorpan (~zcorpan@public.cloak) (Client closed connection)
- # [16:34] * Joins: zcorpan (~zcorpan@public.cloak)
- # [16:41] * Quits: zcorpan (~zcorpan@public.cloak) (Ping timeout: 180 seconds)
- # [16:42] * Quits: MK (kawamori@public.cloak) (Ping timeout: 180 seconds)
- # [16:45] * Joins: AutomatedTester (~AutomatedTester@public.cloak)
- # [16:46] * Quits: anssik (~uid10742@public.cloak) ("Connection closed for inactivity")
- # [17:02] <tobie_> jgraham: what the BrowserExecutor and Executor?
- # [17:03] <tobie_> * what are
- # [17:03] <tobie_> What's the difference between Browser and Browser under test?
- # [17:06] <jgraham> tobie_: I have a slightly updated diagram!
- # [17:06] <jgraham> tobie_: Reload?
- # [17:08] <jgraham> (and again)
- # [17:08] <tobie_> COLOR
- # [17:08] <tobie_> or should I say: COLOUR?
- # [17:09] <jgraham> Yes, you should
- # [17:09] * Joins: gitbot (~gitbot@public.cloak)
- # [17:09] -gitbot:#testing- [web-platform-tests] AutomatedTester closed pull request #988: Some tests for BODY element visibility: must always be displayed no matter what. (master...v1) https://github.com/w3c/web-platform-tests/pull/988
- # [17:09] * Parts: gitbot (~gitbot@public.cloak) (gitbot)
- # [17:09] <jgraham> :)
- # [17:09] * Joins: gitbot (~gitbot@public.cloak)
- # [17:09] -gitbot:#testing- [web-platform-tests] AutomatedTester closed pull request #987: Replacing use of find_element_by_tag_name that doesn't exist in the spec (master...by_tag_name) https://github.com/w3c/web-platform-tests/pull/987
- # [17:09] * Parts: gitbot (~gitbot@public.cloak) (gitbot)
- # [17:09] <jgraham> An Executor is something that implements the testing protocol for a specific test type. So there's a WebDriverTestharnessExecutor, for example
- # [17:10] <tobie_> I thought you were writing python.
- # [17:10] <jgraham> A BrowserExecutor is just a way of holding some information about the Browser in the same process as the Executor
- # [17:10] <jgraham> I know, I know, it sounds like Java
- # [17:11] <Ms2ger> "find_element_by_css"
- # [17:11] <Ms2ger> Who invented this?
- # [17:11] <jgraham> I couldn't work out what the non-Java-sounding solution to that was
- # [17:12] <tobie_> Ms2ger: querySelector sounds so much clearer.
- # [17:12] <tobie_> :D
- # [17:12] <Ms2ger> More accurate, definitely
- # [17:12] <tobie_> mouarf.
- # [17:12] <jgraham> I'm not sure "query" is accurate
- # [17:12] <Ms2ger> At least Selector is
- # [17:13] <jgraham> "selectSelector"
- # [17:13] <tobie_> oh, Ms2ger wasn't trolling?
- # [17:13] <Ms2ger> For once
- # [17:15] <jgraham> AutomatedTester: Does the key help?
- # [17:15] <tobie_> jgraham: what problem is this arch trying to solve?
- # [17:16] <jgraham> tobie_: I have a list! But it's also descriptive i.e. this is what the architecture is
- # [17:16] <tobie_> ok
- # [17:17] <jgraham> tobie_: The list is basically 1) Fast 2) Fault tolerant 3) Portable to multiple products
- # [17:17] <tobie_> that high level
- # [17:18] <jgraham> In particular one sub-goal of 2) was to be able to run multiple browser processes in parallel
- # [17:18] <jgraham> s/2/1/
- # [17:19] <tobie_> k
- # [17:20] <tobie_> Think the diagram makes sense.
- # [17:20] <jgraham> Hopefully when I finish the documentation this will all make sense :) But the point of the diagram is to help understand how the pieces fit together so that people have some idea where to start looking for issues.
- # [17:20] <tobie_> I see.
- # [17:21] <tobie_> You might want to describe the itinerary of what happens for a test run
- # [17:23] <jgraham> Yeah.
- # [17:23] <jgraham> tobie_: Thanks
- # [17:23] * Quits: darobin (rberjon@public.cloak) (Client closed connection)
- # [17:23] <tobie_> e.g. http://coremob.github.io/coremob-test-approach/#host-the-test-runner-directly-on-the-w3c-test-server
- # [17:23] <tobie_> (I hope you can do better than that, but you get the idea)
- # [17:26] <Ms2ger> Heh, coremob
- # [17:29] <tobie_> jgraham: I also find representing 1:many is better done by duplicating stuff (eg http://i.msdn.microsoft.com/dynimg/IC164178.gif)
- # [17:30] <tobie_> …than by using fancy arrows
- # [17:38] <jgraham> tobie_: So, I agree, but it's not really clear to me how to make that work on this diagram
- # [17:44] * Joins: zcorpan (~zcorpan@public.cloak)
- # [17:46] * Joins: lmclister (~lmclister@public.cloak)
- # [17:49] <tobie_> jgraham: are you hand authoring the svg?
- # [17:50] <jgraham> tobie_: I am using something called draw.io
- # [17:51] <jgraham> tobie_: See if you like the new version more
- # [17:51] * Joins: rhauck (~Adium@public.cloak)
- # [17:51] * Quits: zcorpan (~zcorpan@public.cloak) (Ping timeout: 180 seconds)
- # [17:52] <tobie_> I do.
- # [17:52] <tobie_> I'm still confused as to what "Product under Test" means.
- # [17:52] <tobie_> Compared to Browser
- # [17:52] <jgraham> It's the brower process, typically
- # [17:53] <jgraham> (although with Firefox OS it can be a whole device)
- # [17:57] * Quits: Lachy (~Lachy@public.cloak) ("My MacBook Pro has gone to sleep. ZZZzzz…")
- # [17:59] <jgraham> tobie_: If you have a better term, I'm happy to change it.
- # [18:03] <tobie_> It's like a group of browsing contexts, no?
- # [18:06] * Quits: rhauck (~Adium@public.cloak) ("Leaving.")
- # [18:08] <jgraham> Typically it's a single browsing context that's actually running the tests
- # [18:11] <tobie_> Why don't you call "Product under Test" a "browsing context" then?
- # [18:11] <jgraham> That seems pretty weird
- # [18:12] <jgraham> For example it makes it sound like you could have multiple tests in different tabs of the same browser, which you can't
- # [18:12] <jgraham> +instance
- # [18:13] <jgraham> I could call it "Browser Instance" I guess, but that's a bit confusing with the Browser class
- # [18:15] <tobie_> think it's not super clear from your diagram, whether something is a class of your program or a "thing"
- # [18:19] <jgraham> Do you have any suggestions on how to make it clearer?
- # [18:20] <tobie_> it's kind of a chiken and egg problem; I would if I had a better sense of how the whole thing is organized, which I'd get from a clearer diagram.
- # [18:20] <tobie_> * chicken
- # [18:21] <tobie_> maybe use a browser window or some such for the Product under test thing.
- # [18:22] <jgraham> Everything in purple is a class (or function in the case of run_tests) in the program. As is everything in blue or orange. The things in green are external pieces at higher levels than classes. The red thing is whatever browser it is that you're trying to test (a firefox process, or a device running Firefox OS, or a servo process or a Chrome process, or whatever)
- # [18:26] <tobie_> What does the TestLoader do?
- # [18:26] <tobie_> (Sorry, I think I'm waisting more of your time than being helpful)
- # [18:27] <jgraham> tobie_: No, it's helpful
- # [18:28] <jgraham> tobie_: The TestLoader reads the manifest file and any options limiting the tests to include and returns a Queue of test objects ready to be run.
- # [18:29] <tobie_> ok.
- # [18:29] <jgraham> It doesn't *directly* talk to the Executor, but the Queue gets passed through to the — well actually to the TestRunner I think, which is a mistake in the diagram
- # [18:29] <tobie_> So yeah, you'd really need a list of what's happening
- # [18:29] <tobie_> and in what order.
- # [18:30] <tobie_> Also, it's somewhat of a misnomer given the Web context.
- # [18:31] * Joins: dom (dom@public.cloak)
- # [18:32] <jgraham> I don't understand why you think it's a misnomer
- # [18:32] <jgraham> (I'm not saying you're wrong)
- # [18:32] <Ms2ger> It doesn't really "load", does it?
- # [18:33] <jgraham> Well it queues
- # [18:33] <tobie_> yeah, TestCollectionBuilder or something
- # [18:33] <jgraham> But TestQueuer sounds a bit strange
- # [18:34] <Ms2ger> I can't come up with something nice either
- # [18:35] <tobie_> ManifestReader?
- # [18:35] <tobie_> Impl. ves encaps and all
- # [18:36] <tobie_> s/ves/vs/
- # [18:37] <jgraham> Well it's also used by the code for reducing a list of unstable tests to a minimal set required to reproduce the instability and that usage depends on internal state, so it's not like it gets called to read the manifest and then disappears (in that context)
- # [18:38] <Ms2ger> Should that be the same class? :)
- # [18:38] <tobie_> ^
- # [18:38] <jgraham> Well sure, I mean it's just a superset of the functionality
- # [18:39] <tobie_> TestEnqueuer
- # [18:39] <Ms2ger> TestQueueReducer
- # [18:39] <tobie_> Fancy
- # [18:39] <tobie_> What does Executor do?
- # [18:40] <jgraham> http://www.classnamer.com/ suggests ConfigurableNodeReader :p
- # [18:41] <Ms2ger> wfm
- # [18:41] <jgraham> tobie_: It actually executes the tests. So for example when using WebDriver, it send the WebDriver commands to the browser to start a test and listens for the result
- # [18:41] <tobie_> And what does testrunner do?
- # [18:42] <tobie_> jgraham: that's not what classnamer gave me (StableRobotVector)
- # [18:43] <jgraham> It is the generic interface between executors (which are specific to test running protocol and test type) and the rest of the code (which is not). It does things like pulls the next test off the queue of tests and passes it to the executor to run
- # [18:46] <tobie_> and BrowserExecutor?
- # [18:48] <jgraham> That is just a way of packaging some information about the Browser (e.g. the port on which it will accept a WebDriver connection) so that it can be used in the same process as the Executor.
- # [18:49] <jgraham> (passing the Browser instance cross-process doesn't work and sometimes there is a little bit of extra behaviour in the BrowserExecutor e.g. in B2G it has code to ensure that the device is configured for running tests)
- # [18:50] * Joins: rhauck (~Adium@public.cloak)
- # [18:52] <tobie_> Not sure whether that's a design issue or a diagram one, but I feel the whole thing would gain in clarity if the Executor and ExecutorBrowser were brokered by the test runner (and thus absent from the diagram)
- # [18:53] <jgraham> They're kind of important, because if you want to implement support for a new browser you might have to implement a new Executor (depending on whether it supports one of the existing ones)
- # [18:54] <jgraham> I'm not sure what you mean "brokered by"
- # [18:54] <tobie_> Agreed, but they're not important to understand the mechanics of the whole thing
- # [18:54] <tobie_> yeah, poor choice of words, sorry,
- # [18:55] <tobie_> I feel like the TestRunner should be the one controlling the test queue, and using the executor to actually get the test running
- # [18:57] <tobie_> In pseudo code: t = new TestRunner(new Executor("webdriver"))
- # [18:57] <tobie_> t.tests = queue;
- # [18:57] <tobie_> t.next();
- # [19:03] <jgraham> Yes, well that's basically what happens
- # [19:04] <jgraham> The signature for TestRunner is (self, test_queue, command_queue, result_queue, executor)
- # [19:04] <jgraham> command_queue and result_queue are implementation details for getting messages back and forward from the child process
- # [19:08] <jgraham> tobie_: Uploaded a new diagram (it also seems like there was a bug in the key, which might have been confusing)
- # [19:19] <tobie_> thats much clearer already
- # [20:46] * Joins: darobin (rberjon@public.cloak)
- # [21:22] * Quits: ArtB (~abarsto@public.cloak) ("Leaving.")
- # [21:43] * Quits: darobin (rberjon@public.cloak) (Client closed connection)
- # [21:47] * Joins: abarsto (~abarsto@public.cloak)
- # [21:47] * abarsto is now known as ArtB
- # [21:48] * Joins: gitbot (~gitbot@public.cloak)
- # [21:48] -gitbot:#testing- [web-platform-tests] AutomatedTester pushed 1 new commit to master: https://github.com/w3c/web-platform-tests/commit/4a70874c4a757e730d86938c4a94d2b33d6fc0ed
- # [21:48] -gitbot:#testing- web-platform-tests/master 4a70874 Jason Leyba: tests for WebDriver executeScript...
- # [21:48] * Parts: gitbot (~gitbot@public.cloak) (gitbot)
- # [21:51] * Joins: gitbot (~gitbot@public.cloak)
- # [21:51] -gitbot:#testing- [web-platform-tests] AutomatedTester closed pull request #704: Add some tests for WebDriver executeScript (section 11.2.1) (master...master) https://github.com/w3c/web-platform-tests/pull/704
- # [21:51] * Parts: gitbot (~gitbot@public.cloak) (gitbot)
- # [21:53] * Quits: schuki (~quassel@public.cloak) ("No Ping reply in 180 seconds.")
- # [21:56] * Joins: schuki (~quassel@public.cloak)
- # [22:00] * Quits: schuki (~quassel@public.cloak) ("No Ping reply in 180 seconds.")
- # [22:00] * Joins: schuki (~quassel@public.cloak)
- # [22:03] * Quits: schuki (~quassel@public.cloak) ("No Ping reply in 180 seconds.")
- # [22:04] * Joins: schuki (~quassel@public.cloak)
- # [22:08] * Quits: schuki (~quassel@public.cloak) ("No Ping reply in 180 seconds.")
- # [22:16] * Joins: schuki (~quassel@public.cloak)
- # [22:19] * Quits: schuki (~quassel@public.cloak) ("No Ping reply in 180 seconds.")
- # [22:24] * Joins: schuki (~quassel@public.cloak)
- # [22:28] * Quits: schuki (~quassel@public.cloak) ("No Ping reply in 180 seconds.")
- # [22:36] * Joins: schuki (~quassel@public.cloak)
- # [22:55] <jgraham> http://wptrunner.readthedocs.org/en/latest/design.html
- # [22:55] <jgraham> tobie_: ^
- # [22:55] <jgraham> I seem to have used italics a lot where I meant code
- # [23:29] * Quits: lmclister (~lmclister@public.cloak) ("")
- # [23:32] * Quits: Ms2ger (~Ms2ger@public.cloak) ("nn")
- # [23:37] * Quits: dom (dom@public.cloak) (Ping timeout: 180 seconds)
- # [23:42] * Quits: ArtB (~abarsto@public.cloak) (Ping timeout: 180 seconds)
- # [23:44] * Joins: abarsto (~abarsto@public.cloak)
- # [23:44] * abarsto is now known as ArtB
- # Session Close: Tue May 20 00:00:01 2014
The end :)