/irc-logs / w3c / #testing / 2015-10-14 / end

Options:

Previous day, Next day

  1. # Session Start: Wed Oct 14 00:00:00 2015
  2. # Session Ident: #testing
  3. # [00:14] * Quits: rego (~smuxi@public.cloak) (Client closed connection)
  4. # [00:19] * Joins: rego (~smuxi@public.cloak)
  5. # [00:36] * Joins: gitbot (~gitbot@public.cloak)
  6. # [00:36] -gitbot:#testing- [csswg-test] cvrebert created take-2 (+1 new commit): https://github.com/w3c/csswg-test/commit/6d53125d80f8
  7. # [00:36] -gitbot:#testing- csswg-test/take-2 6d53125 Chris Rebert: Try checking for the existence of .hg subdirectory...
  8. # [00:36] * Parts: gitbot (~gitbot@public.cloak)
  9. # [00:48] * Joins: gitbot (~gitbot@public.cloak)
  10. # [00:48] -gitbot:#testing- [csswg-test] cvrebert pushed 1 new commit to master: https://github.com/w3c/csswg-test/commit/0e53d2b2d9259c3bc2b9c611ff0976c412aafa9a
  11. # [00:48] -gitbot:#testing- csswg-test/master 0e53d2b Chris Rebert: Try checking for the existence of .hg subdirectory...
  12. # [00:48] * Parts: gitbot (~gitbot@public.cloak)
  13. # [00:55] * Joins: admin (~admin@public.cloak)
  14. # [00:55] * admin slaps Yves around a bit with a large fishbot
  15. # [00:55] * Quits: admin (~admin@public.cloak) ("Page closed")
  16. # [01:17] * Joins: gitbot (~gitbot@public.cloak)
  17. # [01:17] -gitbot:#testing- [csswg-test] cvrebert created robust-get_new_commits (+1 new commit): https://github.com/w3c/csswg-test/commit/d5069a30dace
  18. # [01:17] -gitbot:#testing- csswg-test/robust-get_new_commits d5069a3 Chris Rebert: Fix tools/travis/build.py:get_new_commits() in the case where there are no new commits...
  19. # [01:17] * Parts: gitbot (~gitbot@public.cloak)
  20. # [01:17] * Joins: gitbot (~gitbot@public.cloak)
  21. # [01:17] -gitbot:#testing- [csswg-test] cvrebert deleted take-2 at 6d53125: https://github.com/w3c/csswg-test/commit/6d53125
  22. # [01:17] * Parts: gitbot (~gitbot@public.cloak)
  23. # [01:21] * Joins: gitbot (~gitbot@public.cloak)
  24. # [01:21] -gitbot:#testing- [csswg-test] cvrebert opened pull request #918: Fix tools/travis/build.py:get_new_commits() when there aren't any new commits (master...robust-get_new_commits) https://github.com/w3c/csswg-test/pull/918
  25. # [01:21] * Parts: gitbot (~gitbot@public.cloak)
  26. # [02:12] * Quits: bhill2 (~bhill2@public.cloak) (Client closed connection)
  27. # [03:10] * Quits: ArtB (~ArtB@public.cloak) ("Leaving.")
  28. # [03:20] * Joins: Halfbrite (~Halfbrite@public.cloak)
  29. # [03:22] * Quits: Halfbrite (~Halfbrite@public.cloak) ("Page closed")
  30. # [03:22] * Joins: halfbrite (~halfbrite@public.cloak)
  31. # [03:27] <halfbrite> I was expecting to see a /css/ directory under web-platform-tests, but didn't find one. Are css tests in a separate repo? (I was interested in border-collapse tests, per http://www.w3.org/TR/CSS21/tables.html#propdef-border-collapse)
  32. # [04:14] * Parts: kochi1 (~kochi@public.cloak)
  33. # [04:17] * Quits: kochi (~kochi@public.cloak) (Client closed connection)
  34. # [04:20] * Joins: kochi (~kochi@public.cloak)
  35. # [04:24] * Joins: kochi1 (~kochi@public.cloak)
  36. # [04:51] * Joins: jyasskin (~textual@public.cloak)
  37. # [05:08] * Quits: rego (~smuxi@public.cloak) (Ping timeout: 180 seconds)
  38. # [05:08] * Quits: jyasskin (~textual@public.cloak) ("My computer has gone to sleep. ZZZzzz…")
  39. # [05:10] * Joins: rego (~smuxi@public.cloak)
  40. # [06:15] * Joins: jyasskin (~textual@public.cloak)
  41. # [06:16] * Joins: bhill2 (~bhill2@public.cloak)
  42. # [06:41] * Joins: bhill2_ (~bhill2@public.cloak)
  43. # [06:41] * Quits: bhill2 (~bhill2@public.cloak) (Client closed connection)
  44. # [06:43] * Quits: bhill2_ (~bhill2@public.cloak) (Client closed connection)
  45. # [07:14] * Quits: jyasskin (~textual@public.cloak) ("My computer has gone to sleep. ZZZzzz…")
  46. # [07:18] * Joins: jyasskin (~textual@public.cloak)
  47. # [07:24] * Quits: rego (~smuxi@public.cloak) (Client closed connection)
  48. # [07:28] * Quits: jyasskin (~textual@public.cloak) ("My computer has gone to sleep. ZZZzzz…")
  49. # [07:31] * Joins: rego (~smuxi@public.cloak)
  50. # [07:39] * Quits: rego (~smuxi@public.cloak) (Client closed connection)
  51. # [07:46] * Joins: rego (~smuxi@public.cloak)
  52. # [07:56] * Quits: rego (~smuxi@public.cloak) (Client closed connection)
  53. # [08:03] * Joins: rego (~smuxi@public.cloak)
  54. # [08:21] * Quits: rego (~smuxi@public.cloak) (Client closed connection)
  55. # [08:23] * Joins: Ms2ger (~Ms2ger@public.cloak)
  56. # [08:28] * Joins: rego (~smuxi@public.cloak)
  57. # [08:38] * Quits: rego (~smuxi@public.cloak) (Client closed connection)
  58. # [08:45] * Joins: rego (~smuxi@public.cloak)
  59. # [09:28] * Quits: halfbrite (~halfbrite@public.cloak) (Ping timeout: 180 seconds)
  60. # [09:34] * Quits: Ms2ger (~Ms2ger@public.cloak) (Ping timeout: 180 seconds)
  61. # [09:45] * Joins: Ms2ger (~Ms2ger@public.cloak)
  62. # [09:47] * Joins: dom (dom@public.cloak)
  63. # [11:51] * Joins: ArtB (~ArtB@public.cloak)
  64. # [12:50] <jgraham> WTAF
  65. # [12:51] <jgraham> How is https://github.com/w3c/web-platform-tests/blob/master/html/dom/elements/elements-in-the-dom/unknown-element.html timing out‽
  66. # [12:51] <jgraham> Ms2ger: ^
  67. # [12:51] <jgraham> (intermittently on Mac and Windows)
  68. # [12:56] <jgraham> Hmm, the test before is html/dom/dynamic-markup-insertion/opening-the-input-stream/document.open-02.html
  69. # [13:15] * Joins: simonstewart (~textual@public.cloak)
  70. # [13:31] * Joins: plh (plehegar@public.cloak)
  71. # [13:49] * Joins: gitbot (~gitbot@public.cloak)
  72. # [13:49] -gitbot:#testing- [csswg-test] Ms2ger closed pull request #918: Fix tools/travis/build.py:get_new_commits() when there aren't any new commits (master...robust-get_new_commits) https://github.com/w3c/csswg-test/pull/918
  73. # [13:49] * Parts: gitbot (~gitbot@public.cloak)
  74. # [13:49] * Joins: gitbot (~gitbot@public.cloak)
  75. # [13:49] -gitbot:#testing- [csswg-test] Ms2ger deleted robust-get_new_commits at d5069a3: https://github.com/w3c/csswg-test/commit/d5069a3
  76. # [13:49] * Parts: gitbot (~gitbot@public.cloak)
  77. # [14:19] <gsnedders> am I right in believing we shouldn't have testharness.css linked from any test?
  78. # [14:19] <gsnedders> because testharness.js adds it?
  79. # [14:20] <jgraham> You are
  80. # [14:21] * gsnedders wonders if we should have a lint check for this
  81. # [14:21] <gsnedders> And remove it from existing testsing
  82. # [14:21] <gsnedders> *tests
  83. # [14:21] <jgraham> Makes sense
  84. # [14:29] <Ms2ger> I imagine there might be cases where you don't want th.js to mess with your DOM
  85. # [14:31] <jgraham> It only inserts it after the tests are complete
  86. # [14:32] <Ms2ger> Oh, hm
  87. # [14:32] <Ms2ger> Go for it
  88. # [15:04] <gsnedders> Ms2ger: testharness.js has done this for years
  89. # [15:05] <Ms2ger> Yeah, I know
  90. # [15:05] <gsnedders> I don't quite think the /original/ version did.
  91. # [15:05] <Ms2ger> I meant that being able to avoid it could be useful
  92. # [15:05] <gsnedders> yeah
  93. # [15:05] <gsnedders> I remember discussions about that from years back
  94. # [16:21] * Quits: rego (~smuxi@public.cloak) (Client closed connection)
  95. # [16:28] * Joins: rego (~smuxi@public.cloak)
  96. # [16:32] * Quits: rego (~smuxi@public.cloak) (Client closed connection)
  97. # [16:33] * Joins: jyasskin (~textual@public.cloak)
  98. # [16:39] * Joins: rego (~smuxi@public.cloak)
  99. # [16:43] * Quits: rego (~smuxi@public.cloak) (Client closed connection)
  100. # [16:50] * Joins: rego (~smuxi@public.cloak)
  101. # [16:58] * Quits: rego (~smuxi@public.cloak) (Client closed connection)
  102. # [17:08] * Joins: rego (~smuxi@public.cloak)
  103. # [17:16] * Quits: rego (~smuxi@public.cloak) (Client closed connection)
  104. # [17:23] * Joins: rego (~smuxi@public.cloak)
  105. # [17:23] * Quits: rego (~smuxi@public.cloak) (Client closed connection)
  106. # [17:32] * Joins: rego (~smuxi@public.cloak)
  107. # [18:20] * Joins: w3c (~w3c@public.cloak)
  108. # [18:31] * Quits: w3c (~w3c@public.cloak) (Ping timeout: 180 seconds)
  109. # [18:51] * Quits: Ms2ger (~Ms2ger@public.cloak) ("nn")
  110. # [19:05] * Quits: jyasskin (~textual@public.cloak) ("My computer has gone to sleep. ZZZzzz…")
  111. # [19:06] * Joins: bhill2 (~bhill2@public.cloak)
  112. # [19:22] * Joins: halfbrite (~halfbrite@public.cloak)
  113. # [19:23] * Quits: dom (dom@public.cloak) ("")
  114. # [19:23] <halfbrite> Any guidance on how to think about the relationship between web-platform-test and csswg-test? web-platform-tests seems incomplete without css tests. Is the expectation that a vendor runs both suites?
  115. # [19:24] <halfbrite> ...or is adding css tests to web-platform-test a todo?
  116. # [19:24] * Quits: rego (~smuxi@public.cloak) (Client closed connection)
  117. # [19:27] <gsnedders> halfbrite: both are run
  118. # [19:27] <gsnedders> halfbrite: there's currently no real effort to merge them
  119. # [19:31] * Joins: rego (~smuxi@public.cloak)
  120. # [19:39] <gsnedders> halfbrite: basically csswg-test should be CSS and CSSOM, web-platform-tests should be the rest of the web platform
  121. # [19:39] <gsnedders> where the web platform is roughly defined by "what browsers implement"
  122. # [19:47] * Quits: rego (~smuxi@public.cloak) (Client closed connection)
  123. # [19:54] * Joins: rego (~smuxi@public.cloak)
  124. # [20:03] <halfbrite> With the tests organized by spec, is there much overhead in maintaining the correct directory structure as specs evolve? (Within the Edge team, we're doing a review of our internal tests. It's been suggested we reorganize to match w3c/wpt's structure to make it easy to opportunistically contribute back. However, there is concern that a directory structure derived from tests is a moving target.)
  125. # [20:03] * Quits: rego (~smuxi@public.cloak) (Client closed connection)
  126. # [20:04] <halfbrite> *directory structure derived from *specs*
  127. # [20:07] <gsnedders> I've only really just reappeared after mostly vanishing years ago, so I'm not the best person to answer. :)
  128. # [20:07] <gsnedders> AFAIK there just hasn't been much rearranging of tests (or of sections in most of the specs)
  129. # [20:09] <gsnedders> Looking through the repo's history tends to suggest we don't move stuff much
  130. # [20:10] <halfbrite> (I was getting the same impression looking through the commits as well.)
  131. # [20:11] * Joins: rego (~smuxi@public.cloak)
  132. # [20:15] * Quits: rego (~smuxi@public.cloak) (Client closed connection)
  133. # [20:16] <gsnedders> halfbrite: are you guys running any of the tests, BTW?
  134. # [20:17] <jgraham> halfbrite: Yeah, at least moving tests around would also be a pain for Mozilla because we'd have to update all our internal metadata to match
  135. # [20:17] <jgraham> Well probably not *much* of a pain, since the update system is automatic, but it would still be suboptimal
  136. # [20:18] <jgraham> The history of why CSS tests aren't in wpt is long and not a litle sad
  137. # [20:18] <gsnedders> Much to do with CSS WG politics, really.
  138. # [20:19] <jgraham> But basically at this point the main constraint is that they want to use their custom build and metadata system that depends on hg and a database and all this stuff, and there's no easy way to reconcile that with wpt
  139. # [20:19] <jgraham> where there is no build system
  140. # [20:21] <jgraham> For runnng CSS tests in servo we have https://github.com/jgraham/css-test-built which is an automatically updated build of the CSS tests
  141. # [20:21] <jgraham> In theory that could be a submodule of wpt, but it's unclear that's not more trouble than it's worth
  142. # [20:21] <gsnedders> tbf, the metadata system seems less and less important to the WG
  143. # [20:22] <halfbrite> I see, thanks for the pointers. (Are you using WPTRunner in Mozilla, or have you built your own solution there as well?)
  144. # [20:22] <gsnedders> (The former.)
  145. # [20:22] * Joins: rego (~smuxi@public.cloak)
  146. # [20:23] <jgraham> Well both are true. In that wptrunner (i.e. https://github.com/w3c/wptrunner is something that I wrote with Mozilla as a specific use case and the hope it would get a wider audience)
  147. # [20:23] <jgraham> That ) ended up in the wrong place
  148. # [20:24] <gsnedders> well, it's essentially a wrapper around wptrunner, no?
  149. # [20:24] * Quits: rego (~smuxi@public.cloak) (Client closed connection)
  150. # [20:26] <halfbrite> I see. Looking at the commit history, it seems Mozilla is fairly active. I'm curious, are you the primary contributors at this point and is WPT your primary means of testing? (In Edge, we run WPT regularily, but it's currently more to track which way we're trending than it is a means to detect defects.)
  151. # [20:27] <gsnedders> (I'm not affiliated with Mozilla, FWIW. Or any browser vendor, any more.)
  152. # [20:27] <jgraham> It isn't our primary means of testing, except for the Servo project, but it is something that we run on every commit along with numerous other testsuites
  153. # [20:28] <gsnedders> I thought wpt (+ a few things not yet upstreamed) were the largest part of a number of more recent features
  154. # [20:28] <gsnedders> *?
  155. # [20:29] <jgraham> https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&revision=833c3c37daa6 is a typical Mozilla CI run; the things that look like W(+4) and Wr and similar are web-platform-tests
  156. # [20:29] <jgraham> gsnedders: Service-workers perhaps? I'm not sure how many other tests we wrote for that, but I think more than I would have liked in the end
  157. # [20:31] * Joins: rego (~smuxi@public.cloak)
  158. # [20:31] * Quits: rego (~smuxi@public.cloak) (Client closed connection)
  159. # [20:33] <gsnedders> halfbrite: I don't know if you or anyone else involved in this are going be at TPAC, but myself and fantasai were planning on trying to work out what people need to run more tests (both wpt and csswg-test) more regularly, in some proposed session for the plenary day
  160. # [20:34] <gsnedders> halfbrite: https://www.w3.org/wiki/TPAC/2015/SessionIdeas#Web_Platform_Testing
  161. # [20:36] <halfbrite> I'm sure someone from Edge will be there. I'll try to send them your way.
  162. # [20:37] <gsnedders> We'll probably try and grab someone from the CSS WG if nothing else :)
  163. # [20:43] * Joins: rego (~smuxi@public.cloak)
  164. # [20:47] * Quits: rego (~smuxi@public.cloak) (Client closed connection)
  165. # [20:54] * Joins: rego (~smuxi@public.cloak)
  166. # [21:00] * Quits: rego (~smuxi@public.cloak) (Client closed connection)
  167. # [21:07] * Joins: rego (~smuxi@public.cloak)
  168. # [21:17] * Quits: rego (~smuxi@public.cloak) (Client closed connection)
  169. # [21:24] * Joins: rego (~smuxi@public.cloak)
  170. # [21:26] <halfbrite> I noticed service-worker\specgen.json. It looks like there is a tool to import a spec and stub out tests pages?
  171. # [22:01] * Joins: jyasskin (~textual@public.cloak)
  172. # [22:19] <gsnedders> halfbrite: some specs do magic things.
  173. # [22:20] <gsnedders> halfbrite: oft times, the tools are specific to that spec
  174. # [22:24] <halfbrite> gsnedders: I see. I think it's quite a slick idea. I was particularily impressed w/using IdlHarness.js to automatically test everything that can be inferred by the webidl.
  175. # [22:25] <gsnedders> The one disadvantage of idlharness.js is that if you have one minor bug you end up with hundreds of errors all over the place so it can be hard to find bigger issues.
  176. # [22:26] <gsnedders> But yeah, it certainly reduces the number of tests that need written by hand.
  177. # [22:26] * Quits: rego (~smuxi@public.cloak) (Client closed connection)
  178. # [22:26] <gsnedders> (And it can certainly be improved upon to test more!)
  179. # [22:28] <halfbrite> I could see that how a small bug could send IdlHarness.js off the rails. Do you know if there is a tool keep the idl tests up to date?
  180. # [22:29] <gsnedders> No, there isn't. Not least because they need a fair bit of hand-holding to work.
  181. # [22:29] <halfbrite> It seems like that could be a chore (esp. tracking down the dependencies.)
  182. # [22:29] <gsnedders> If it changes incompatibly, hopefully it'll get noticed when the impls change :)
  183. # [22:30] <gsnedders> if it gets extended, hopefully someone notices eventually…
  184. # [22:31] <halfbrite> That's an interesting pholisophical question... In general, does WPT try to track the implementations or the latest specs?
  185. # [22:31] <gsnedders> latest specs
  186. # [22:31] <gsnedders> except, uh, when there's a decision to change the spec with the support of all vendors
  187. # [22:32] <gsnedders> but in general the spec
  188. # [22:32] <gsnedders> (the more interesting stuff is when there are multiple editions of a spec, be it ED v. REC, or W3C v. WHATWG)
  189. # [22:33] <halfbrite> Heh. I can imagine. BTW, thank you, this has been an illuminating conversation in general.
  190. # [22:33] * Joins: rego (~smuxi@public.cloak)
  191. # [22:34] * Quits: rego (~smuxi@public.cloak) (Client closed connection)
  192. # [22:42] * Joins: rego (~smuxi@public.cloak)
  193. # [22:48] * Quits: rego (~smuxi@public.cloak) (Client closed connection)
  194. # [22:55] * Joins: rego (~smuxi@public.cloak)
  195. # [23:01] * Quits: rego (~smuxi@public.cloak) (Client closed connection)
  196. # [23:02] <gsnedders> halfbrite: feel free to shout out here any time (though especially at this time of day there may well be quite a lag, given so many of us here are based in Europe), or throw stuff to public-test-infra@w3.org
  197. # [23:05] <gsnedders> halfbrite: in general most of us are happy to do whatever to get the tests used as widely as possible (and ideally in CI)!
  198. # [23:07] <gsnedders> (really this is what I care about primarily, more than anything else)
  199. # [23:08] * Joins: rego (~smuxi@public.cloak)
  200. # [23:12] * Quits: rego (~smuxi@public.cloak) (Client closed connection)
  201. # [23:13] <jgraham> The specgen thing is pretty useless
  202. # [23:13] <jgraham> Or at least no one ever actually followed through and wrote service-worker tests using it
  203. # [23:14] <jgraham> I guess it could help if you had some setup where you wanted to precisely define tasks for people to get shallow but broad coverage of something
  204. # [23:15] * Joins: zcorpan (~zcorpan@public.cloak)
  205. # [23:15] <halfbrite> gsnedders: (Thanks! I will reach out as I keep exploring.)
  206. # [23:18] <halfbrite> jgraham: I was thinking of using the tool (if I can find it) to stub out an empty directory structure to help developers reorganize our internal tests to match WPT as a first step towards merging/contributing back.
  207. # [23:19] * Joins: rego (~smuxi@public.cloak)
  208. # [23:21] <halfbrite> We have over a decade of accumulated tests that we're reviewing, organized first around the release in which the feature was implemented, then usually by the feature team that implemented it.
  209. # [23:26] <jgraham> halfbrite: Interesting, I guess it could work, although I think that tool is more about stubbing out tests from a spec
  210. # [23:27] <jgraham> FWIW Mozilla didn't try to reorganise existing tests but run the new tests in parallel, in the expectation that over time we will write more wpt tests and fewer tests that can't be shared
  211. # [23:29] <halfbrite> That's our plan B. :) ...but we happen to have other work going on that creates an opportunity to do this "opportunistically", provided we can do it quickly.
  212. # [23:30] <halfbrite> (or more accurately, we are touching/moving the tests anyway. If we can easily map them to the right location, it's low cost.)
  213. # [23:49] <gsnedders> organised by original release is certainly an interesting structure :)
  214. # [23:57] <gsnedders> (there again, Opera's "by bug ID" was also nice)
  215. # [23:59] <gsnedders> ("so where are the [xxx] tests? let's try and find what bug ID originally implemented them, and then start looking for any bug fixes ever made on them")
  216. # Session Close: Thu Oct 15 00:00:00 2015

Previous day, Next day

Think these logs are useful? Then please donate to show your gratitude (and keep them up, of course). Thanks! — Krijn