/irc-logs / w3c / #testing / 2014-06-27 / end

Options:

  1. # Session Start: Fri Jun 27 00:00:00 2014
  2. # Session Ident: #testing
  3. # [00:09] * Quits: Lachy (~Lachy@public.cloak) ("My MacBook Pro has gone to sleep. ZZZzzz…")
  4. # [00:19] * Joins: MK (kawamori@public.cloak)
  5. # [00:37] * Joins: scott_gonzalez (~scott_gonzalez@public.cloak)
  6. # [01:11] * heycam|away is now known as heycam
  7. # [01:36] * Joins: Lachy (~Lachy@public.cloak)
  8. # [01:41] * Quits: MK (kawamori@public.cloak) (Ping timeout: 180 seconds)
  9. # [01:43] * Quits: lmclister (~lmclister@public.cloak) ("")
  10. # [02:06] * Quits: ArtB (~abarsto@public.cloak) ("Leaving.")
  11. # [02:08] * Quits: AutomatedTester (~AutomatedTester@public.cloak) (Client closed connection)
  12. # [04:45] * Quits: glenn (~gadams@public.cloak) (Client closed connection)
  13. # [04:45] * Joins: glenn (~gadams@public.cloak)
  14. # [04:52] * Quits: glenn (~gadams@public.cloak) (Ping timeout: 180 seconds)
  15. # [05:18] * heycam is now known as heycam|away
  16. # [06:05] * heycam|away is now known as heycam
  17. # [06:43] * Joins: glenn (~gadams@public.cloak)
  18. # [09:00] * Joins: dom (dom@public.cloak)
  19. # [09:03] * Joins: gitbot (~gitbot@public.cloak)
  20. # [09:03] -gitbot:#testing- [web-platform-tests] dontcallmedom pushed 5 new commits to master: https://github.com/w3c/web-platform-tests/compare/7e9339dd8eaf...95a037c31763
  21. # [09:03] -gitbot:#testing- web-platform-tests/master 5bb0464 Zhiqiang Zhang: Import Geolocation API Test Suite...
  22. # [09:03] -gitbot:#testing- web-platform-tests/master 1c923b7 Zhiqiang Zhang: Remove tests covered by geo IDL tests in #348
  23. # [09:03] -gitbot:#testing- web-platform-tests/master 51f0807 Zhiqiang Zhang: Rewrite other geo tests using testharness.js as...
  24. # [09:03] * Parts: gitbot (~gitbot@public.cloak) (gitbot)
  25. # [09:27] * Quits: glenn (~gadams@public.cloak) (Client closed connection)
  26. # [09:44] * Quits: ptressel (~chatzilla@public.cloak) ("zzz")
  27. # [09:48] * Quits: Lachy (~Lachy@public.cloak) ("My MacBook Pro has gone to sleep. ZZZzzz…")
  28. # [10:02] * Joins: zcorpan (~zcorpan@public.cloak)
  29. # [10:04] * Quits: zcorpan (~zcorpan@public.cloak) ("Leaving...")
  30. # [10:29] * Joins: Lachy (~Lachy@public.cloak)
  31. # [10:39] * Joins: Ms2ger (~Ms2ger@public.cloak)
  32. # [11:20] <jgraham> I have a bikeshed. Anyone got a paintbrush?
  33. # [11:20] <wilhelm> Is lead paint okay?
  34. # [11:21] <jgraham> Depends if you want to kill the idea :p
  35. # [11:22] <jgraham> So forCSP testing, it seems like there is a desire for substitutions in .headers files
  36. # [11:22] <jgraham> So far so fine
  37. # [11:22] <jgraham> But there is a requirement to do something like put the *same* random data in multiple places
  38. # [11:23] <jgraham> So random data is easy to add. Something like {{uuid()}}
  39. # [11:24] <jgraham> But something is needed for variable binding. So far I have {{$foo:uuid()}} and {{$foo}}, where the $ is required
  40. # [11:24] <jgraham> Those will both be replaced with the same uuid
  41. # [11:25] <jgraham> But I am open to other suggestions for syntax
  42. # [11:25] <jgraham> (the above is implemented and working, but it's not hard to change)
  43. # [11:25] * Joins: gitbot (~gitbot@public.cloak)
  44. # [11:25] -gitbot:#testing- [web-platform-tests] sizuhiko opened pull request #1079: remove createSR wrapper of createShadowRoot (master...submission/sizuhiko/remove_wrapper_of_create_shadow_root_separated) https://github.com/w3c/web-platform-tests/pull/1079
  45. # [11:25] * Parts: gitbot (~gitbot@public.cloak) (gitbot)
  46. # [11:26] <Ms2ger> Seems somewhat ugly, but I guess it's fine
  47. # [11:28] <Ms2ger> Btw, I've been working on cleaning up Philip`'s tests: https://github.com/Ms2ger/web-platform-tests/compare/philip
  48. # [11:28] <Ms2ger> Will probably PR tonight
  49. # [11:31] <jgraham> Yeah, I wasn't sure about requiring the $, but it at least means it's clear that {{host:uuid()}} {{host}} doesn't work
  50. # [11:33] <jgraham> (I guess the next bit of syntax will be {{$foo:uuid()|js}} which will js escape rather than html escape)
  51. # [11:53] * Joins: AutomatedTester (~AutomatedTester@public.cloak)
  52. # [12:02] <ato> Great, so MS is pushing for NodeJS in wpt.
  53. # [12:02] * ato goes to rewrite all his code
  54. # [12:03] <gsnedders> MS are V8 fanboys now? :)
  55. # [12:10] * Quits: AutomatedTester (~AutomatedTester@public.cloak) (Client closed connection)
  56. # [12:13] * Quits: Lachy (~Lachy@public.cloak) ("My MacBook Pro has gone to sleep. ZZZzzz…")
  57. # [12:24] <MikeSmith> ato: where are they asking? w3c list?
  58. # [12:25] <ato> public-browser-tools-testing@ specifically.
  59. # [12:25] <gsnedders> wait, we have *another* ML now?
  60. # [12:25] * Joins: abarsto (~abarsto@public.cloak)
  61. # [12:25] * abarsto is now known as ArtB
  62. # [12:27] <wilhelm> gsnedders: That's for the WebDriver work. And has been for several years.
  63. # [12:27] <gsnedders> oh
  64. # [12:27] <gsnedders> did I used to be on this, before I got unsubscribed from everything by changing to no affiliation?
  65. # [12:30] <MikeSmith> gsnedders: maybe
  66. # [12:31] <MikeSmith> gsnedders: I can re-subscribe you if you want
  67. # [12:31] <gsnedders> don't bother
  68. # [12:31] <MikeSmith> k
  69. # [12:31] <gsnedders> I haven't even bothered trying to get myself back on public-html
  70. # [12:31] <wilhelm> Why would you want to subscribe to that? :D
  71. # [12:33] * Quits: scott_gonzalez (~scott_gonzalez@public.cloak) (scott_gonzalez)
  72. # [12:33] <MikeSmith> for the GT
  73. # [12:33] <gsnedders> to tell them they're doing it wrong when appropriate
  74. # [12:34] <gsnedders> but mostly to ignore it like I do all mailing lists
  75. # [12:38] <MikeSmith> funny thing you probably didn't know, when you go to confession at church, priests can assign you to read mailing lists as penance
  76. # [12:38] <MikeSmith> instead of repeating hail marys or our fathers
  77. # [12:38] <jgraham> MikeSmith: Wow, you must have done something *terrible*
  78. # [12:39] <MikeSmith> jgraham: no for that they assign you to read IETF mailing lists
  79. # [12:39] <wilhelm> I just got the prayer thing. I lied about my sin, though. Maybe that would have qualified for the mailing list penance.
  80. # [12:40] <gsnedders> MikeSmith: Thankfully, i wasn't brought up Catholic
  81. # [12:42] <MikeSmith> I heard that so far only some Google PMs have done anything bad enough to merit the you-must-read-an-IETF-mailing-list penance assignment
  82. # [12:42] <MikeSmith> but that's just a rumore
  83. # [12:44] <MikeSmith> wilhelm: speaking of penance we are now officially re-bound to the Browser Tools and Testing WG yoke through December 2015
  84. # [12:45] <MikeSmith> https://lists.w3.org/Archives/Member/w3c-ac-members/2014AprJun/0058.html
  85. # [12:45] <MikeSmith> "The Browser Testing and Tools Working Group charter [1] is hereby extended through December 2015."
  86. # [12:45] <wilhelm> MikeSmith: I just forwarded that one to our list.
  87. # [12:46] <wilhelm> A little bondage is good for you.
  88. # [12:46] <MikeSmith> wilhelm: thanks
  89. # [12:46] <MikeSmith> wilhelm: btw about the "The first version is expected to be completed by March 2015" that is non-binding of course
  90. # [12:46] <wilhelm> Yes, but worth a shot. (c;
  91. # [12:46] <MikeSmith> yeah
  92. # [12:54] * Joins: Lachy (~Lachy@public.cloak)
  93. # [12:58] <jgraham> Ms2ger: https://critic.hoppipolla.co.uk/r/1915 if you have time to review
  94. # [13:07] * Joins: scott_gonzalez (~scott_gonzalez@public.cloak)
  95. # [13:07] <Ms2ger> jgraham, perhaps later
  96. # [13:19] * Joins: MK (kawamori@public.cloak)
  97. # [13:30] <jgraham> Ms2ger: np
  98. # [13:31] <Ms2ger> Also, planning to do some parsing+serialization work this weekend
  99. # [13:32] <jgraham> Nice
  100. # [13:38] * Joins: zcorpan (~zcorpan@public.cloak)
  101. # [13:43] * Quits: zcorpan (~zcorpan@public.cloak) ("Leaving...")
  102. # [14:10] * Quits: Ms2ger (~Ms2ger@public.cloak) ("bbl")
  103. # [14:33] * Quits: MarkS (msadecki@public.cloak) (Ping timeout: 180 seconds)
  104. # [14:40] * Joins: anssik (~uid10742@public.cloak)
  105. # [14:47] * Quits: MK (kawamori@public.cloak) (Ping timeout: 180 seconds)
  106. # [15:07] * Joins: plh (plehegar@public.cloak)
  107. # [15:45] * Quits: plh (plehegar@public.cloak) ("Leaving")
  108. # [16:11] * Joins: gitbot (~gitbot@public.cloak)
  109. # [16:11] -gitbot:#testing- [web-platform-tests] dontcallmedom closed pull request #348: Import IDL tests for Geolocation API (master...submission/zqzhang/geo-idl-test-suite) https://github.com/w3c/web-platform-tests/pull/348
  110. # [16:11] * Parts: gitbot (~gitbot@public.cloak) (gitbot)
  111. # [16:23] * heycam is now known as heycam|away
  112. # [16:45] * Joins: glenn (~gadams@public.cloak)
  113. # [16:48] * Quits: botie (botie@public.cloak) (Client closed connection)
  114. # [16:49] * Joins: plh (plehegar@public.cloak)
  115. # [17:00] * Quits: plh (plehegar@public.cloak) ("Leaving")
  116. # [17:01] * Joins: plh (plehegar@public.cloak)
  117. # [17:17] * Joins: gitbot (~gitbot@public.cloak)
  118. # [17:17] -gitbot:#testing- [web-platform-tests] andreastt opened pull request #1080: Make tools/scripts/manifest.py executable (master...ato/manifest_executable) https://github.com/w3c/web-platform-tests/pull/1080
  119. # [17:17] * Parts: gitbot (~gitbot@public.cloak) (gitbot)
  120. # [17:18] <ato> Are there any docs on writing manifests anywhere?
  121. # [17:21] * Joins: Ms2ger (~Ms2ger@public.cloak)
  122. # [17:23] <jgraham> Writing what kind of manifest?
  123. # [17:27] <ato> I think I may have misunderstood. The manifest is deduced from filename?
  124. # [17:27] * Ms2ger curses
  125. # [17:27] <ato> So I need to patch manifest.py to treat .py files specially if I want to add webdriver tests to the generated manifest?
  126. # [17:28] * jgraham blesses
  127. # [17:29] <jgraham> ato: Yeah, you need something in manifest.py to detect .py files in the webdriver repo as WebDriverTests
  128. # [17:29] <jgraham> (you will also need to add WebDriver to the list of test types)
  129. # [17:29] <Ms2ger> jgraham, so there's a bunch of Philip's tests for which the generated code seems checked in, but the source is gone
  130. # [17:29] <Ms2ger> Hmm
  131. # [17:30] * Ms2ger checks something
  132. # [17:30] <ato> jgraham: Thanks
  133. # [17:30] <jgraham> http://canvex.lazyilluminati.com/misc/tests/ ?
  134. # [17:33] * Quits: dom (dom@public.cloak) ("")
  135. # [17:35] * Ms2ger tries and fails to undelete a file
  136. # [17:35] <ato> So I'm wondering if “webdriver tests” is the best name. My specific use case right now is that I want to include the webdriver spec tests in wptrunner, but presumably it will in the future be used for converting manual tests into automated, interactive tests.
  137. # [17:37] <Ms2ger> Okay, at least one is a dupe
  138. # [17:37] <Ms2ger> ato, webdriven tests?
  139. # [17:38] <ato> I just don't want to add something by the wrong name and sticking with it “because it's hard to change”. (-:
  140. # [17:39] <ato> Also I notice there are no manifest.py tests. )-:
  141. # [17:43] <jgraham> ato: WebdriverProtocolTest?
  142. # [17:49] <jgraham> ato: And yeah, manifest.py isn't my finest moment
  143. # [17:57] * Quits: Lachy (~Lachy@public.cloak) ("My MacBook Pro has gone to sleep. ZZZzzz…")
  144. # [17:58] * Quits: plh (plehegar@public.cloak) ("Leaving")
  145. # [17:58] * Joins: plh (plehegar@public.cloak)
  146. # [18:03] <Ms2ger> Hm
  147. # [18:03] <Ms2ger> What happened to the img.complete tests?
  148. # [18:04] <jgraham> 303 zcorpan
  149. # [18:06] <Ms2ger> !summon zcorpan
  150. # [18:08] <jgraham> 404 not found
  151. # [18:10] <Ms2ger> Dammit you
  152. # [18:10] <wilhelm> 412
  153. # [18:12] * Ms2ger fails wilhelm
  154. # [18:13] * jgraham thought consultants always responded with 402
  155. # [18:17] <Ms2ger> Bah
  156. # [18:17] * Ms2ger pokes plh
  157. # [18:17] * Joins: lmclister (~lmclister@public.cloak)
  158. # [18:20] <jgraham> 100
  159. # [18:24] * Ms2ger grumbles
  160. # [18:24] <Ms2ger> Surely plh should be up by now
  161. # [18:33] <plh> back
  162. # [18:34] <plh> I was having a bite :)
  163. # [18:35] <plh> I don't understand the problem yet
  164. # [18:37] <ato> Hah.
  165. # [18:38] <ato> So I patched manifest.py but the Python test name rule I put in place didn't match a single file in the repo.
  166. # [18:43] * Joins: Lachy (~Lachy@public.cloak)
  167. # [18:45] <Ms2ger> plh, my problem is that cloning https://dvcs.w3.org/hg/html/ asks for a password
  168. # [18:45] <plh> hu
  169. # [18:45] * plh looks
  170. # [18:46] * Quits: anssik (~uid10742@public.cloak) ("Connection closed for inactivity")
  171. # [18:49] <plh> ms2ger, I don't know why there is a password yet but but isn't that the repo we used to have for html related tests? We don't use it anymore
  172. # [18:49] <Ms2ger> plh, yeah, but I want the history :)
  173. # [18:50] <Ms2ger> Because zcorpan isn't around to tell me where Philip's img.complete tests went
  174. # [18:50] <plh> ok
  175. # [18:52] <plh> I'll need to one of our system guys. I get a password as well
  176. # [18:52] <plh> s//ask/
  177. # [18:52] <plh> give me 30 minutes
  178. # [18:53] <Ms2ger> Thanks!
  179. # [19:03] <plh> ah, found a way to access that repo in the meantime. I made a zip of it: http://jay.w3.org/~plehegar/htmlhg.zip
  180. # [19:03] <plh> I'll work on why that stuff is being a password
  181. # [19:05] <plh> s/being/behind/
  182. # [19:08] <ato> jgraham: Should the manifest be an exhaustive list of child tests also if possible?
  183. # [19:08] <Ms2ger> plh, thanks :)
  184. # [19:08] <jgraham> ato: No, no child tests
  185. # [19:08] <Ms2ger> (I wonder, should we add a lint for references to obsolete drafts?)
  186. # [19:09] <jgraham> Or just "You appear to be adding metadata. Statistically this metadat will be out of date. Please stop"
  187. # [19:10] <Ms2ger> Do spec links often get out of date?
  188. # [19:10] <Ms2ger> I think most of the ones in dom/ are accurate, at least
  189. # [19:10] <jgraham> No idea. Ask me again in 5 years
  190. # [19:10] <Ms2ger> Perhaps not terribly useful, but not wrong
  191. # [19:11] <jgraham> Certianly for some specs (HTML) I wouldn't rely on them
  192. # [19:11] <Ms2ger> Yeah, I guess HTML moves things around more than average
  193. # [19:12] <ato> What happens if a test file contains no tests?
  194. # [19:12] <jgraham> ato: Nothing much
  195. # [19:12] <Ms2ger> Is it still a test?
  196. # [19:12] <Ms2ger> How about if it falls in the forest?
  197. # [19:12] <jgraham> *fails
  198. # [19:13] <Ms2ger> Ha
  199. # [19:13] <Ms2ger> Man, the old html tests repo is hard to navigaet
  200. # [19:13] <Ms2ger> gate
  201. # [19:14] <jgraham> ato: Specifically in that case I would expect that you would get a test status of OK and no subtest statuses
  202. # [19:15] <ato> Is there a test results document that I can look at on the WWW?
  203. # [19:16] <ato> Showing results from wptrunner.
  204. # [19:16] <jgraham> ato: What kind of results?
  205. # [19:16] <jgraham> I mean it just outputs structured logs
  206. # [19:16] <jgraham> I can point you at one
  207. # [19:16] <ato> Yeah, that works.
  208. # [19:17] <jgraham> http://mozilla-releng-blobs.s3.amazonaws.com/blobs/cedar/sha512/196fefc04c933117d3723d45a6b23a0b1cdf7e66bdef1ff32fdedbcc7289ee4669978082858310987c7106553b439892538c764ff54d810b139cb4c088fd132b
  209. # [19:17] <jgraham> That file is a little large, be warned
  210. # [19:17] <ato> Thanks, that's perfect.
  211. # [19:18] * Joins: glenn_ (~gadams@public.cloak)
  212. # [19:20] <ato> So a parent test in wpt is constituted by _one file_, right? And it would be problematic if a test type used something other than that, say a Python module's full name for parent test.
  213. # [19:20] <jgraham> Well
  214. # [19:20] <jgraham> I suppose it's possible to use anything
  215. # [19:21] <jgraham> So far testharness tests use the url and reftests use (url, type, ref_url)
  216. # [19:22] <jgraham> So actually nothing *strictly* uses filename
  217. # [19:22] <jgraham> *But*
  218. # [19:22] <jgraham> The expected data stuff does assume a mapping between a testname and a path which might not work so well if you don't use filenames.
  219. # [19:23] <jgraham> Or rather urls
  220. # [19:23] <jgraham> So that may have to be updated
  221. # [19:23] <ato> My concern here is that using unittest imposes certain constraints on how its test runner is designed to run tests.
  222. # [19:24] * Quits: glenn (~gadams@public.cloak) (Ping timeout: 180 seconds)
  223. # [19:25] <jgraham> So can't you just take a path to a python file and tell unittest to run that path worth of tests?
  224. # [19:25] <ato> Not directly.
  225. # [19:26] <ato> You can with a few workarounds, so we *could* make it work.
  226. # [19:26] <jgraham> You can somehow because it supports that from the command line
  227. # [19:27] <ato> Yes, so what it does from command line is to go on a test discovery hunt through the path or file you give it.
  228. # [19:27] <ato> Which means traversal into sub directories, and then into files matching a pattern to look for classes extending unittest.TestCase.
  229. # [19:28] <ato> Then it takes what it finds and puts it in multiple unittest.TestSuite's (which the test runner later conflates into an array) which it passes to the TextTestRunner.
  230. # [19:29] <jgraham> But if you give it a full path to a .py file it will only look in that file, surely?
  231. # [19:31] <ato> Yes, if the path is a single file it will do only the latter part about inspecting the file's classes for certain special traits.
  232. # [19:32] <jgraham> Right, so one possible implementation strategy would be to build a list of .py files containing tests, call each of those a "test", run them in turn, and call all the results from each file a "subtest"
  233. # [19:32] <jgraham> (another strategy would be to not use unittest ;)
  234. # [19:35] <ato> So I agree with that approach. I think files are conceptually a lot easier for humans to reason about in this case.
  235. # [19:36] <ato> Although I'm taking the exact opposite approach with a test harness I'm developing at Mozilla.
  236. # [19:37] * Quits: ArtB (~abarsto@public.cloak) ("Leaving.")
  237. # [19:51] * Joins: ptressel (~chatzilla@public.cloak)
  238. # [20:06] <Ms2ger> Aha
  239. # [20:10] * Ms2ger traces this test back to revision 5bfdbef872e3 (Wed Feb 23 18:39:20 2011 +0000)
  240. # [20:11] <Ms2ger> Oh wait
  241. # [20:11] <Ms2ger> That commit removed 2d.drawImage.incomplete.html
  242. # [20:13] <Ms2ger> Oh nooooooo
  243. # [20:13] <Ms2ger> approved/ strikes again
  244. # [20:15] <Ms2ger> Oh, and the reason I can't find 2d.drawImage.svg is that some Daniel Davis imitated Philip's code, but didn't add his test to the yaml
  245. # [20:23] * Quits: lmclister (~lmclister@public.cloak) ("")
  246. # [20:24] * Joins: lmclister (~lmclister@public.cloak)
  247. # [20:27] * Quits: glenn_ (~gadams@public.cloak) (Client closed connection)
  248. # [20:27] * Joins: glenn (~gadams@public.cloak)
  249. # [20:28] * Quits: glenn (~gadams@public.cloak) (Client closed connection)
  250. # [20:34] * Joins: rhauck (~Adium@public.cloak)
  251. # [20:34] * Joins: glenn (~gadams@public.cloak)
  252. # [20:35] * Quits: glenn (~gadams@public.cloak) (Client closed connection)
  253. # [20:40] <Ms2ger> Spec references: <a href="../annotated-spec/canvas.html#testrefs.2d.drawImage.svg">2d.drawImage.svg</a>
  254. # [20:40] <Ms2ger> Dead link.
  255. # [20:44] * Ms2ger kicks people wrapping tests in (function() { ... })();
  256. # [21:19] <plh> ms2ger, I don't think we ever had the annotated spec in that repo
  257. # [21:26] * Joins: AutomatedTester (~AutomatedTester@public.cloak)
  258. # [22:04] * Quits: ptressel (~chatzilla@public.cloak) ("bbl")
  259. # [22:22] * Quits: rhauck (~Adium@public.cloak) ("Leaving.")
  260. # [22:25] <AutomatedTester> Yves: did you ever get the REST api endpoint?
  261. # [22:26] <Ms2ger> plh, the canvas one? Sure did :)
  262. # [22:27] <Yves> not yet no :(
  263. # [23:17] * Quits: AutomatedTester (~AutomatedTester@public.cloak) (Client closed connection)
  264. # [23:26] * Quits: plh (plehegar@public.cloak) ("Leaving")
  265. # [23:48] * Joins: rhauck (~Adium@public.cloak)
  266. # [23:53] * Quits: Ms2ger (~Ms2ger@public.cloak) ("nn")
  267. # Session Close: Sat Jun 28 00:00:00 2014

The end :)