/irc-logs / w3c / #testing / 2014-10-20 / end

Options:

  1. # Session Start: Mon Oct 20 00:00:00 2014
  2. # Session Ident: #testing
  3. # [00:32] * Quits: shepazu (schepers@public.cloak) ("is sleepy")
  4. # [01:55] * Quits: ptressel (~chatzilla@public.cloak) ("bbl")
  5. # [04:53] * Joins: mdas (~mdas@public.cloak)
  6. # [04:54] * Quits: mdas (~mdas@public.cloak) ("Leaving...")
  7. # [04:59] * Joins: mdas (~mdas@public.cloak)
  8. # [05:00] * Quits: mdas (~mdas@public.cloak) ("Leaving...")
  9. # [08:26] * Joins: Ms2ger (~Ms2ger@public.cloak)
  10. # [09:28] * Joins: ptressel (~chatzilla@public.cloak)
  11. # [10:29] * Quits: Ms2ger (~Ms2ger@public.cloak) (Ping timeout: 180 seconds)
  12. # [10:48] * Quits: Lachy (~Lachy@public.cloak) ("My MacBook Pro has gone to sleep. ZZZzzz…")
  13. # [11:16] * Joins: AutomatedTester (~AutomatedTester@public.cloak)
  14. # [11:18] * Joins: Lachy (~Lachy@public.cloak)
  15. # [11:31] * Joins: Ms2ger (~Ms2ger@public.cloak)
  16. # [12:25] * Quits: AutomatedTester (~AutomatedTester@public.cloak) (Client closed connection)
  17. # [12:25] * Joins: abarsto (~abarsto@public.cloak)
  18. # [12:25] * abarsto is now known as ArtB
  19. # [12:48] * Quits: Ms2ger (~Ms2ger@public.cloak) (Ping timeout: 180 seconds)
  20. # [13:24] * Quits: ptressel (~chatzilla@public.cloak) ("ZZZzzz...")
  21. # [14:15] * Joins: scott_gonzalez (~scott_gonzalez@public.cloak)
  22. # [14:27] * Joins: Ms2ger (~Ms2ger@public.cloak)
  23. # [14:35] <ArtB> ms2ger, jgraham, all - the Pointer Events WG will soon start interop testing. Since almost all of the tests are manual, seems like tools/runner won't be very useful for obtaining results. As such, do you have any recommendations on how to gather test results when a test suite is just about all manual?
  24. # [14:37] <Ms2ger> http://w3c-test.org/tools/runner/index.html?
  25. # [14:40] <Ms2ger> JavaScript error: http://w3c-test.org/tools/runner/runner.js, line 489: this.test_results is null
  26. # [14:51] <jgraham> I guess there isn't a MANIFEST.json on w3c-test.org
  27. # [14:51] <ArtB> (yeah, no worky for me today)
  28. # [14:52] <jgraham> And it can't generate one because there isn't a html5lib
  29. # [14:56] <jgraham> Which we should just add as a submodule I guess
  30. # [14:57] <jgraham> ArtB: I don't think I have any recommendations other than "sweat, blood, and tears (usually in that order)"
  31. # [14:57] <jgraham> The test runner will allow you to run manual tests and generate a results.json file, if that helps
  32. # [14:57] <ArtB> jgraham (yeah, I kinda figured that but wanted to check you folks first)
  33. # [14:58] <ArtB> that indeed could/should be helpful
  34. # [14:58] <jgraham> I'm not sure anyone has ever actually used the manual codepath in anger, so there might be issues
  35. # [14:58] <ArtB> but not sure sure
  36. # [14:59] <jgraham> (in case you're wondering, anger usually comes after sweat but before blood when running manual tests)
  37. # [14:59] <ArtB> LOL
  38. # [15:00] <jgraham> (on an entirely unrelated topic r? 12:57 < jgraham> The test runner will allow you to run manual tests and generate a results.json file, if that helps
  39. # [15:00] <jgraham> Uh, not that
  40. # [15:01] <jgraham> (on an entirely unrelated topic r? https://critic.hoppipolla.co.uk/r/2890 )
  41. # [15:02] * Joins: AutomatedTester (~AutomatedTester@public.cloak)
  42. # [15:09] * Quits: AutomatedTester (~AutomatedTester@public.cloak) (Client closed connection)
  43. # [15:12] <ato> jgraham: Is self.tests always populated with the test types?
  44. # [15:13] <ato> if not self.tests[self.test_type]
  45. # [15:36] <jgraham> Yes
  46. # [15:36] <ato> r+ then
  47. # [15:37] <jgraham> It could perhaps just take test_list = tests[test_type] instead of taking the full dict and the key seperately
  48. # [15:37] <jgraham> I'm not sure how much of a win that would be
  49. # [15:37] <ato> Both is fine I think. I won't make you change anything (-:
  50. # [15:39] <jgraham> Thanks for the review :)
  51. # [15:44] * Joins: mdas (~mdas@public.cloak)
  52. # [15:57] * Joins: shepazu (schepers@public.cloak)
  53. # [16:03] * Joins: plh (plehegar@public.cloak)
  54. # [16:13] * Quits: plh (plehegar@public.cloak) ("Leaving")
  55. # [16:16] * Joins: plh (plehegar@public.cloak)
  56. # [17:04] * Joins: AutomatedTester (~AutomatedTester@public.cloak)
  57. # [17:08] * Joins: gitbot (~gitbot@public.cloak)
  58. # [17:08] -gitbot:#testing- [web-platform-tests] jgraham created jgraham/chunked_fixup (+1 new commit): https://github.com/w3c/web-platform-tests/commit/d9bc182af2e1
  59. # [17:08] -gitbot:#testing- web-platform-tests/jgraham/chunked_fixup d9bc182 James Graham: Fix XHR server side to actually write a response for chunked output.
  60. # [17:08] * Parts: gitbot (~gitbot@public.cloak) (gitbot)
  61. # [17:08] * Joins: gitbot (~gitbot@public.cloak)
  62. # [17:08] -gitbot:#testing- [web-platform-tests] jgraham opened pull request #1297: Fix XHR server side to actually write a response for chunked output. (master...jgraham/chunked_fixup) https://github.com/w3c/web-platform-tests/pull/1297
  63. # [17:08] * Parts: gitbot (~gitbot@public.cloak) (gitbot)
  64. # [17:20] * Quits: AutomatedTester (~AutomatedTester@public.cloak) (Client closed connection)
  65. # [17:22] * Joins: AutomatedTester (~AutomatedTester@public.cloak)
  66. # [17:27] <ArtB> plh, shepazu - the http://w3c-test.org/tools/runner/index.html service isn't working properly (see irc://irc.w3.org:6665/#testing from earlier today). Would you please see that it is fixed before the Oct 21 PEWG call (11:00 Boston)?
  67. # [17:28] <jgraham> I expect darobin or MikeSmith will be able to help
  68. # [17:28] <ArtB> ok; thanks jgraham
  69. # [17:28] <plh> Art, what is broken with it
  70. # [17:28] <plh> ?
  71. # [17:28] <plh> the page loads at least
  72. # [17:28] <jgraham> plh: There's no MANIFEST.json file
  73. # [17:28] <ArtB> plh, try running a test site
  74. # [17:28] <plh> ah
  75. # [17:28] <plh> that's a problem
  76. # [17:28] <jgraham> So it doesn't know what tests there are
  77. # [17:29] <plh> ok, will fix
  78. # [17:29] <plh> thanks for the report
  79. # [17:29] <jgraham> I guess the server should have a html5lib installed somewhere so it can be autogenerated if it's missing, and the sync script should be updated to pregenerate it
  80. # [17:31] <MikeSmith> so, python is using 99.4% of the CPU on that machine
  81. # [17:31] <MikeSmith> and it was doing that earlier today before I restarted wptserve
  82. # [17:31] <MikeSmith> so I guess I need to restart wptserve again
  83. # [17:32] <plh> I'm regenerating the manifest btw
  84. # [17:32] <plh> so, I guess this is using a bit of the cpu as well
  85. # [17:33] <plh> and we should probably expect folks trying to use w3c-test.org during tpac
  86. # [17:33] <jgraham> Yeah, so generating the manifest involves parsing every file in the repo
  87. # [17:33] <plh> so a daily restart would be good during that week
  88. # [17:33] <jgraham> So it could well be using the whole cpu
  89. # [17:34] <MikeSmith> I thought the Web runner was supposed to generate the manifest on demand now
  90. # [17:35] <MikeSmith> automatically I mean
  91. # [17:35] <jgraham> "Error: Manifest generation failed"
  92. # [17:35] <MikeSmith> also, it was hoggin all the cpu even before regenerated the manifest
  93. # [17:35] <jgraham> (from the js console when loading the runner)
  94. # [17:36] <plh> ah, we get a 500
  95. # [17:36] <jgraham> Hence my supposition that it doesn't have access to html5lib
  96. # [17:36] <plh> nope
  97. # [17:36] <plh> it does, since I ran the script
  98. # [17:36] <plh> on the command line
  99. # [17:36] <plh> http://w3c-test.org/tools/runner/update_manifest.py returns 500
  100. # [17:37] <plh> that's what makes the runner fails
  101. # [17:37] <jgraham> Well I don't know how the server is set up but the update_manifest script doesn't necessarily have the same sys.path as the command line python
  102. # [17:38] * plh dives into logs
  103. # [17:38] <jgraham> Oh
  104. # [17:38] <jgraham> It's all my fault
  105. # [17:38] <jgraham> Sorry :)
  106. # [17:38] <plh> http://w3c-test.org/tools/runner/index.html
  107. # [17:38] <plh> oops
  108. # [17:39] <plh> 'module' object has no attribute 'update manifest'
  109. # [17:39] <jgraham> AttributeError: 'module' object has no attribute 'update_manifest'
  110. # [17:39] <jgraham> Yeah
  111. # [17:39] <jgraham> The API of manifest.py changed slightly and I forgot it was used here
  112. # [17:40] <plh> got time to do a patch?
  113. # [17:40] <jgraham> Yeah
  114. # [17:40] <jgraham> Working on i
  115. # [17:40] <jgraham> t
  116. # [17:40] <plh> thank you!
  117. # [17:53] * Joins: gitbot (~gitbot@public.cloak)
  118. # [17:53] -gitbot:#testing- [web-platform-tests] jgraham created jgraham/runner_manifest_fix (+1 new commit): https://github.com/w3c/web-platform-tests/commit/418ec21501c6
  119. # [17:53] -gitbot:#testing- web-platform-tests/jgraham/runner_manifest_fix 418ec21 James Graham: Update test runner for changes in the manifest API.
  120. # [17:53] * Parts: gitbot (~gitbot@public.cloak) (gitbot)
  121. # [17:53] * Joins: gitbot (~gitbot@public.cloak)
  122. # [17:53] -gitbot:#testing- [web-platform-tests] jgraham opened pull request #1298: Update test runner for changes in the manifest API. (master...jgraham/runner_manifest_fix) https://github.com/w3c/web-platform-tests/pull/1298
  123. # [17:53] * Parts: gitbot (~gitbot@public.cloak) (gitbot)
  124. # [17:58] <jgraham> r? on that
  125. # [17:58] <plh> looking
  126. # [17:58] * Joins: gitbot (~gitbot@public.cloak)
  127. # [17:58] -gitbot:#testing- [web-platform-tests] plehegar pushed 1 new commit to master: https://github.com/w3c/web-platform-tests/commit/81006fc24f5dcb929eb2773f1da26a27c610321c
  128. # [17:58] -gitbot:#testing- web-platform-tests/master 81006fc Philippe Le Hegaret: Merge pull request #1298 from w3c/jgraham/runner_manifest_fix...
  129. # [17:58] * Parts: gitbot (~gitbot@public.cloak) (gitbot)
  130. # [17:59] <jgraham> Thanks
  131. # [17:59] <jgraham> Sorry for breaking the world
  132. # [17:59] * Quits: Lachy (~Lachy@public.cloak) ("My MacBook Pro has gone to sleep. ZZZzzz…")
  133. # [17:59] <plh> 'no pb
  134. # [18:00] <plh> I guess not many people uses w3c-test.org nowadays
  135. # [18:00] <plh> Art, you're back in business
  136. # [18:01] <plh> (I only uses it because I'm too lazy to properly configure my virtual machine with IE)
  137. # [18:03] <jgraham> I find it useful to link to on bugs
  138. # [18:04] <plh> true
  139. # [18:07] <Ms2ger> jgraham, http://w3c-test.org/tools/ does something weird
  140. # [18:07] <jgraham> Ms2ger: That's expected behaviour, assuming you get a 404
  141. # [18:08] <Ms2ger> I do
  142. # [18:08] <jgraham> That doesn't mean it's not weird
  143. # [18:08] <jgraham> But it's not unanticipated
  144. # [18:09] <jgraham> /tools/ is all 404 except /tools/runner/
  145. # [18:09] <ArtB> thanks jgraham and plh!
  146. # [18:37] * Joins: bhill2 (~bhill2@public.cloak)
  147. # [18:41] * Quits: bhill2 (~bhill2@public.cloak) (Client closed connection)
  148. # [18:49] * Joins: bhill2 (~bhill2@public.cloak)
  149. # [19:13] * Quits: bhill2 (~bhill2@public.cloak) (Client closed connection)
  150. # [19:15] * Joins: bhill2 (~bhill2@public.cloak)
  151. # [19:34] * Joins: Lachy (~Lachy@public.cloak)
  152. # [19:40] * Joins: Lachy_ (~Lachy@public.cloak)
  153. # [19:45] * Quits: Lachy (~Lachy@public.cloak) (Ping timeout: 180 seconds)
  154. # [19:46] * Quits: bhill2 (~bhill2@public.cloak) (Client closed connection)
  155. # [19:50] * Joins: bhill2 (~bhill2@public.cloak)
  156. # [19:51] * Joins: Lachy (~Lachy@public.cloak)
  157. # [19:51] * Quits: Lachy_ (~Lachy@public.cloak) (Client closed connection)
  158. # [19:53] * Quits: bhill2 (~bhill2@public.cloak) (Client closed connection)
  159. # [19:57] * Joins: bhill2 (~bhill2@public.cloak)
  160. # [20:04] * Joins: bhill2_ (~bhill2@public.cloak)
  161. # [20:06] * Quits: bhill2 (~bhill2@public.cloak) (Ping timeout: 180 seconds)
  162. # [20:08] * Quits: plh (plehegar@public.cloak) ("Leaving")
  163. # [20:39] * Quits: bhill2_ (~bhill2@public.cloak) (Client closed connection)
  164. # [20:41] * Joins: bhill2 (~bhill2@public.cloak)
  165. # [20:55] * Joins: plh (plehegar@public.cloak)
  166. # [20:58] * Quits: plh (plehegar@public.cloak) ("Leaving")
  167. # [20:58] * Quits: bhill2 (~bhill2@public.cloak) (Client closed connection)
  168. # [20:58] * Joins: bhill2 (~bhill2@public.cloak)
  169. # [20:59] * Quits: bhill2 (~bhill2@public.cloak) (Client closed connection)
  170. # [20:59] * Joins: bhill2 (~bhill2@public.cloak)
  171. # [21:00] * Quits: bhill2 (~bhill2@public.cloak) (Client closed connection)
  172. # [21:00] * Joins: bhill2 (~bhill2@public.cloak)
  173. # [21:00] * Joins: plh (plehegar@public.cloak)
  174. # [21:29] * Quits: ArtB (~abarsto@public.cloak) ("Leaving.")
  175. # [21:40] * Quits: bhill2 (~bhill2@public.cloak) (Client closed connection)
  176. # [21:41] * Joins: bhill2 (~bhill2@public.cloak)
  177. # [21:42] * Quits: bhill2 (~bhill2@public.cloak) (Client closed connection)
  178. # [21:50] * Joins: bhill2 (~bhill2@public.cloak)
  179. # [22:00] * Quits: bhill2 (~bhill2@public.cloak) (Client closed connection)
  180. # [22:03] * Joins: bhill2 (~bhill2@public.cloak)
  181. # [22:10] * Quits: bhill2 (~bhill2@public.cloak) (Ping timeout: 180 seconds)
  182. # [22:26] * Joins: bhill2 (~bhill2@public.cloak)
  183. # [22:51] * Quits: plh (plehegar@public.cloak) ("Leaving")
  184. # [23:17] * Quits: bhill2 (~bhill2@public.cloak) ("Leaving...")
  185. # [23:19] * Joins: bhill2 (~bhill2@public.cloak)
  186. # [23:20] * Quits: mdas (~mdas@public.cloak) (Client closed connection)
  187. # [23:20] * Joins: mdas (~mdas@public.cloak)
  188. # [23:20] * Quits: mdas (~mdas@public.cloak) ("Leaving...")
  189. # [23:31] * Quits: scott_gonzalez (~scott_gonzalez@public.cloak) (scott_gonzalez)
  190. # [23:45] * Joins: ptressel (~chatzilla@public.cloak)
  191. # [23:57] * Quits: Ms2ger (~Ms2ger@public.cloak) (Ping timeout: 180 seconds)
  192. # Session Close: Tue Oct 21 00:00:00 2014

The end :)