/irc-logs / w3c / #testing / 2013-09-20 / end

Options:

  1. # Session Start: Fri Sep 20 14:27:53 2013
  2. # Session Ident: #testing
  3. # [14:27] * Now talking in #testing
  4. # [14:27] <krijnh> jgraham: yep!
  5. # [14:27] <jgraham> darobin: also, 11:55 < jgraham> So, I wonder if there should be a single repo that contains both wptserve and web-platform-tests as submodules
  6. # [14:28] <jgraham> krijnh: Thanks!
  7. # [14:30] <tobie> jgraham: why not make it a submodule of web-platform-tests?
  8. # [14:30] <jgraham> tobie: Well making the server a submodule of w-p-t is another possibility, but then I will need to add (more?) rules so that you don't try and serve the server code
  9. # [14:31] <jgraham> I guess I need to exclude /tools anyway
  10. # [14:31] <tobie> hadn't thought about that.
  11. # [14:31] <tobie> but yeah, a config file with excluded repos sounds like a reasonable idea
  12. # [14:31] <tobie> s/repos/dirs/
  13. # [14:32] <jgraham> And if you want to have a single script that does "start the server with all the right config" in the root directory, that needs to be excluded too
  14. # [14:32] <jgraham> I guess this isn't an insurmountable problem. I hadn't realised that I would have to solve it anyway
  15. # [14:33] <jgraham> So I think I retract my suggestion
  16. # [14:34] <darobin> jgraham: yeah, I reckon that just putting wptserve inside wpt would make sense
  17. # [14:34] <darobin> the more self-contained the better
  18. # [14:36] <jgraham> OK, I will do that on the branch
  19. # [14:37] <jgraham> But first lunch
  20. # [15:00] * Joins: AutomatedTester (~AutomatedTester@public.cloak)
  21. # [15:12] * Joins: gitbot (~gitbot@public.cloak)
  22. # [15:12] -gitbot:#testing- [web-platform-tests] hallvors pushed 1 new commit to hallvors/XHRbugFix3: https://github.com/w3c/web-platform-tests/commit/d8e47ea2f318d0f2f0fd6001f04d9f1d9aa6a3dd
  23. # [15:12] -gitbot:#testing- web-platform-tests/hallvors/XHRbugFix3 d8e47ea Hallvord R. M. Steen: dropping invalid tests for data: and about:blank, adding a data: URL that won't parse
  24. # [15:12] * Parts: gitbot (~gitbot@public.cloak) (gitbot)
  25. # [15:41] <jgraham> So, more random thoughts:
  26. # [15:42] <jgraham> What if we only supported running the tests on some unpriviledged port, like 8000. Or rather 8000, 8001 and 4430, so something
  27. # [15:42] <jgraham> s/so/or/
  28. # [15:42] <darobin> the value being?
  29. # [15:42] <jgraham> YOu wouldn't need to sudo to get the "real" configuration
  30. # [15:43] <jgraham> Otherwise I think we need to support both and find some clever way to deal with tests that try to hardcode port numbers
  31. # [15:44] <jgraham> (going totally the other way, we could support running on *random* ports and have some magical substitution syntax so that tests could find out the ports in case they need to know
  32. # [15:44] <jgraham> )
  33. # [15:45] <AutomatedTester> is this for the webserver?
  34. # [15:45] <jgraham> Yes
  35. # [15:45] <AutomatedTester> the python one?
  36. # [15:45] <jgraham> Yes
  37. # [15:45] <AutomatedTester> ask the OS for a free port and then share that information to the tests
  38. # [15:45] <jgraham> That sounds good, but it's not that easy to do
  39. # [15:46] <AutomatedTester> which part is not easy?
  40. # [15:46] <jgraham> The second part
  41. # [15:47] <AutomatedTester> how are the tests started/loaded?
  42. # [15:47] <darobin> it may generally be useful to have the test be able to load configuration information from a reliable source
  43. # [15:47] <jgraham> HTTP GET to the test URL
  44. # [15:47] <darobin> e.g. for domains as well
  45. # [15:47] <jgraham> Well domains aren't so hard
  46. # [15:47] <darobin> jgraham: if we run this on the web on a common site I'd rather we did that on port 80
  47. # [15:48] <darobin> at least, I think I'd rather that
  48. # [15:48] * darobin thinks
  49. # [15:48] <jgraham> Because you always know that the domain that you loaded the test from is the "main" domain
  50. # [15:48] <jgraham> and there are subdomains
  51. # [15:48] <AutomatedTester> darobin: on a common site you would probably have a port forwarding mechansim like a loadbalancer or tables
  52. # [15:49] <jgraham> and then we are good right up to certificates and other such things
  53. # [15:49] <AutomatedTester> or wsgi
  54. # [15:49] <jgraham> AutomatedTester: That doesn't really help
  55. # [15:49] <jgraham> AFAICT
  56. # [15:49] <AutomatedTester> if wsgi?
  57. # [15:49] <jgraham> I guess one other problem is that ports 80 and 443 are blessed
  58. # [15:49] <AutomatedTester> helps if it MUST be on a common site for people to use
  59. # [15:50] <jgraham> So URL serialization happens differently there
  60. # [15:51] <jgraham> Which kind of suggests that you want to always run on those ports
  61. # [15:52] <jgraham> But I wouldn't be super happy making people run the server as root (and work out how to disable their other web server)
  62. # [15:52] <jgraham> And it might not even be allowed on automation
  63. # [15:52] <jgraham> AutomatedTester: ^?
  64. # [15:52] <darobin> AutomatedTester: on a normal site, yes, on this one I would really rather avoid fronting it with anything
  65. # [15:52] <darobin> we've had too many problems with servers rewriting stuff
  66. # [15:53] <AutomatedTester> jgraham: I dont think we can run on root on our test infra
  67. # [15:53] <AutomatedTester> jgraham: a number of things internally at Mozilla start the webserver on a random port and pass to the tests
  68. # [15:53] <darobin> I agree we need to support running on arbitrary ports; I'm just not sure that we want to enforce specific ones
  69. # [15:54] <jgraham> OK
  70. # [15:54] <jgraham> So
  71. # [15:54] <darobin> also, re running as root, can't you just grab the port and then drop privilege?
  72. # [15:54] <AutomatedTester> random port == port that we given by OS
  73. # [15:54] <AutomatedTester> not sure
  74. # [15:55] <jgraham> This doesn't solve the problem that location.href on port 80 looks very different from location.href for the same site on port 17890
  75. # [15:55] <jgraham> I *suppose* we can just expect the tests to deal with that
  76. # [15:56] <jgraham> It also doesn't suggest a good way of passing this port information to the tests
  77. # [15:56] <tobie> We should just document these assumptions
  78. # [15:56] <jgraham> I don't really fancy doing string substitution on every file we serve
  79. # [15:57] <tobie> no?
  80. # [15:57] <tobie> :D
  81. # [15:58] <jgraham> (but just doing some magic so that it is avaliable through js isn't always good enough e.g. a CSS file might use background-image with a different port url())
  82. # [15:58] <darobin> jgraham: I would hope the tests really deal with that, but if it's going to work they'll need someone to tell them what the port is, no?
  83. # [15:59] <jgraham> darobin: The questions are "who" and "how", I think
  84. # [15:59] <darobin> well for JS, just GET /config and receive some JSON with all you need
  85. # [15:59] <jgraham> (also means we can't *test* url serialization rules for default ports)
  86. # [15:59] <darobin> for CSS, well, that sucks
  87. # [15:59] <tobie> are there a valid use cases for CSS needing that info?
  88. # [16:00] <jgraham> tobie: 13:58 < jgraham> (but just doing some magic so that it is avaliable through js isn't always good enough e.g. a CSS file might use background-image with a different port url())
  89. # [16:00] <tobie> Is this currently an issue in the CSS WG?
  90. # [16:00] * Quits: zcorpan (~zcorpan@public.cloak) (Client closed connection)
  91. # [16:01] * Joins: zcorpan (~zcorpan@public.cloak)
  92. # [16:01] <jgraham> I don't know. I don't know what setup they use
  93. # [16:01] <tobie> plinss: should be able to help here. He's due to write-up a set of reqs for this anyway.
  94. # [16:01] <jgraham> So the CSS problem we can solve
  95. # [16:02] <jgraham> at the cost that every file that wants to know the port has to be a .py file
  96. # [16:02] <jgraham> The "how do I test rules that depend on whether we are running on port 80 or not" problem strikes me an unsolvable without running on port 80
  97. # [16:03] <tobie> Curious if this is actually an issue for the CSS WG now.
  98. # [16:03] <AutomatedTester> if everything just used webdriver™
  99. # [16:03] <AutomatedTester> :P
  100. # [16:03] <jgraham> I don't see how this is related to WebDriver
  101. # [16:03] <tobie> nwither do I actually
  102. # [16:04] <jgraham> the problem is
  103. # [16:04] <AutomatedTester> the test setup in python could start the server and then get the port number
  104. # [16:05] <jgraham> AutomatedTester: That part isn't the problem
  105. # [16:05] <jgraham> I can fix that
  106. # [16:05] <jgraham> I can't write test(function() {assert_equals("http://server-name/foo", window.location.href)}
  107. # [16:05] <jgraham> Even if I know server-name
  108. # [16:05] <jgraham> I can only write
  109. # [16:06] <jgraham> test(function() {assert_equals("http://server-name:port/foo", window.location.href)}
  110. # [16:06] <jgraham> Except that's wrong if port is 80
  111. # [16:06] <jgraham> So I have to test a different codepath depending on whether the port is 80 or not
  112. # [16:06] <jgraham> i.e.
  113. # [16:07] <tobie> assert_url_equals
  114. # [16:07] <jgraham> test(function() {assert_equals(port === 80 ? "http://server-name/foo" : "http://server-name:port/foo", window.location.href)}
  115. # [16:07] <jgraham> Which is bad
  116. # [16:07] <jgraham> Because whether the test passes depends on which server you run it on
  117. # [16:08] * Quits: zcorpan (~zcorpan@public.cloak) (Ping timeout: 180 seconds)
  118. # [16:08] <tobie> Let me try to understand this better
  119. # [16:09] <tobie> Are you concerned with not being able to test that port 80 doesn't show in the window.location.href unless you're running on port 80?
  120. # [16:09] <jgraham> Yes
  121. # [16:11] <tobie> Well, that's a bit like running a reftest on a setup that can't take screenshots: you're screwed.
  122. # [16:11] <darobin> have I missed the bit where that's a part of the platform where we're worried about interop?
  123. # [16:11] <jgraham> I think the concern boils down to "not being on both port 80 and some other port makes it impossible to test some simple features. Worse, it makes it easy to write a test where the pass condition depends on the server"
  124. # [16:11] <darobin> I mean, sure, we want to test that
  125. # [16:11] <darobin> but maybe it shouldn't condition everything?
  126. # [16:11] <jgraham> darobin: You haven't been working on servo, I take it :)
  127. # [16:12] <darobin> (I guess there are places where it matters more though)
  128. # [16:12] <darobin> jgraham: heh, point taken
  129. # [16:12] * darobin would actually like to work on Servo, if he happened to be competent for that, which he isn't
  130. # [16:13] <tobie> I think I'm missing the big picture here.
  131. # [16:13] <tobie> For how many tests is this an issue?
  132. # [16:14] <jgraham> I have no idea
  133. # [16:15] <tobie> Well, what API's outside of window.loc are affected by this?
  134. # [16:15] <jgraham> Possibly URL?
  135. # [16:16] <tobie> Depending on how the tests are run, this is true
  136. # [16:17] <tobie> But that's really an implementation detail of the test, is it not?
  137. # [16:17] <jgraham> I don't understand the question
  138. # [16:18] <tobie> What I'm clumsily trying to get at is that there are two kinds of tests affected by this
  139. # [16:19] <tobie> 1) those that test whether :80 doesn't show up in URLs
  140. # [16:19] <tobie> for these, there are probably little we can do if not running on port 80.
  141. # [16:20] * Joins: mdas_ (~mdas@public.cloak)
  142. # [16:20] <tobie> 2) tests where we're not really concerned about :80 being implicit, but it so happens that we're relying on this behavior (somewhat accidentally)
  143. # [16:20] <jgraham> Yes, I agree
  144. # [16:20] <tobie> These tests should be fixed.
  145. # [16:20] <jgraham> So we can fix type 2 tests
  146. # [16:22] <jgraham> I guess I can add that yak to my shaving pile
  147. # [16:24] <tobie> re type 1 test I guess we should just not run the tests when not on port 80 (and report a NOTRUN)
  148. # [16:26] * Quits: mdas (~mdas@public.cloak) (Ping timeout: 180 seconds)
  149. # [16:27] <jgraham> Hmm, interesting idea
  150. # [16:28] <jgraham> But it doesn't help that much if only w3c-test.org uses port 80
  151. # [16:32] * Joins: zcorpan (~zcorpan@public.cloak)
  152. # [16:51] <jgraham> AutomatedTester: Do you have a good python function to find a free port somewhere? I can write a simple one ofc but there might be a better way
  153. # [16:53] <AutomatedTester> jgraham: https://code.google.com/p/selenium/source/browse/py/selenium/webdriver/common/utils.py#22
  154. # [16:55] <jgraham> So, wait, what? Binding to port 0 selects a free port?
  155. # [16:57] <jgraham> That seems to be the case
  156. # [16:58] <jgraham> Cool, I learnt my one thing for today
  157. # [16:58] <jgraham> Just need to make sure not to do it again
  158. # [17:01] <darobin> that's a pretty cool default
  159. # [17:16] * Joins: jhammel (~jhammel@public.cloak)
  160. # [17:51] <jgraham> So, anyone for websockets?
  161. # [17:52] <jgraham> I think we need to host our own clone of pywebsocket
  162. # [17:52] <jgraham> In git
  163. # [17:52] <jgraham> Then we can submodule it
  164. # [17:52] <jgraham> Then we can hope that it doesn't really care which port its on
  165. # [17:52] <jgraham> +'
  166. # [17:55] <AutomatedTester> yea, selecting port 0 means the OS will give you a free port
  167. # [17:56] <AutomatedTester> so you then disconnect and *hopefully* its still free
  168. # [17:56] <AutomatedTester> neat little trick :)
  169. # [17:57] <jgraham> Yeah, kind of obviously racy
  170. # [17:57] <jgraham> But good none-the-less
  171. # [17:58] <jgraham> (you presumably could pass the actual socket back if whatever you want to use the port for supported passing sockets in)
  172. # [17:58] <AutomatedTester> its Good Enough™
  173. # [17:58] <jgraham> (which typically they don't)
  174. # [17:58] * Quits: Lachy (~Lachy@public.cloak) ("My MacBook Pro has gone to sleep. ZZZzzz…")
  175. # [18:01] * darobin wonders if it's really intended that jgraham has the right to clone repos to the w3c org :)
  176. # [18:04] <jgraham> It does seem strange that I can't add a user to an existing repo, but I can add whole new repos
  177. # [18:04] <darobin> jgraham: wait... so you can fork an existing repo to w3c, but can't create a w3c repo from scract?
  178. # [18:04] <darobin> scratch
  179. # [18:04] <jgraham> I can create one from scratch
  180. # [18:05] <darobin> ah, I read that backwards
  181. # [18:05] <darobin> sorry
  182. # [18:05] <jgraham> I can't give others commit access to the existing repos
  183. # [18:06] <darobin> well, maybe we trust you to come into the sheep house but not to be discerning as to which wolves you invite in?
  184. # [18:06] <jgraham> (or, potentially, I can't work out how in the UI, but I don't think it's that)
  185. # [18:30] * Quits: darobin (rberjon@public.cloak) (Client closed connection)
  186. # [18:36] * Joins: scott_gonzalez (~scott_gonzalez@public.cloak)
  187. # [18:37] <tobie> jgraham: I checked: you have admin privileges on wpt
  188. # [18:37] <jgraham> tobie: Oh. OK
  189. # [18:37] <tobie> but that doesn't give you the right to invite folks
  190. # [18:37] <jgraham> Then I guess I just don't understand the github ui
  191. # [18:37] <jgraham> Oh
  192. # [18:37] <tobie> as that's an org level thing
  193. # [18:37] <tobie> (afaik)
  194. # [18:37] <jgraham> ThenI just don't understand the permissions model
  195. # [18:38] <tobie> yeah, it's funky
  196. # [18:39] * Joins: Automate_ (~AutomatedTester@public.cloak)
  197. # [18:39] * Quits: AutomatedTester (~AutomatedTester@public.cloak) (Client closed connection)
  198. # [18:41] <andreastt> Automate_, jgraham: Did it use to be the default in Python that 0 did that?
  199. # [18:41] <andreastt> I distinctly remember some more complicated code (à la the Java implementation in Selenium) for doing that in Python.
  200. # [18:43] * Quits: tobie (tobie@public.cloak)
  201. # [18:45] * Quits: dom (dom@public.cloak) ("")
  202. # [18:58] <Automate_> andreastt: the selenium code uses 0 in python
  203. # [18:58] <Automate_> andreastt: its ~5 lines of code
  204. # [18:59] <Automate_> https://code.google.com/p/selenium/source/browse/py/selenium/webdriver/common/utils.py#22
  205. # [18:59] <andreastt> Certainly delegating it to the library is a neat trick.
  206. # [19:04] * Quits: zcorpan (~zcorpan@public.cloak) (Client closed connection)
  207. # [19:06] * Joins: Lachy (~Lachy@public.cloak)
  208. # [19:08] * Joins: zcorpan (~zcorpan@public.cloak)
  209. # [19:08] * Quits: zcorpan (~zcorpan@public.cloak) (Client closed connection)
  210. # [19:08] * Joins: zcorpan (~zcorpan@public.cloak)
  211. # [19:15] * Quits: zcorpan (~zcorpan@public.cloak) (Ping timeout: 180 seconds)
  212. # [19:15] * Joins: glenn (~gadams@public.cloak)
  213. # [19:39] * Joins: zcorpan (~zcorpan@public.cloak)
  214. # [20:16] * Joins: AutomatedTester (~AutomatedTester@public.cloak)
  215. # [20:16] * Quits: Automate_ (~AutomatedTester@public.cloak) (Client closed connection)
  216. # [20:28] * Quits: zcorpan (~zcorpan@public.cloak) (Client closed connection)
  217. # [20:28] * Joins: zcorpan (~zcorpan@public.cloak)
  218. # [21:08] * Quits: AutomatedTester (~AutomatedTester@public.cloak) (Client closed connection)
  219. # [21:08] * Joins: AutomatedTester (~AutomatedTester@public.cloak)
  220. # [21:49] * Quits: zcorpan (~zcorpan@public.cloak) (Client closed connection)
  221. # [21:49] * Joins: zcorpan (~zcorpan@public.cloak)
  222. # [21:56] * Quits: zcorpan (~zcorpan@public.cloak) (Ping timeout: 180 seconds)
  223. # [21:58] * Joins: zcorpan (~zcorpan@public.cloak)
  224. # [22:21] * Quits: glenn (~gadams@public.cloak) (Client closed connection)
  225. # [22:22] * Quits: ArtB (~abarsto@public.cloak) (Ping timeout: 180 seconds)
  226. # [22:51] * Joins: Automate_ (~AutomatedTester@public.cloak)
  227. # [22:51] * Quits: AutomatedTester (~AutomatedTester@public.cloak) (Client closed connection)
  228. # [23:42] * Joins: tobie (tobie@public.cloak)
  229. # [23:48] * Joins: gitbot (~gitbot@public.cloak)
  230. # [23:48] -gitbot:#testing- [web-platform-tests] tobie pushed 2 new commits to master: https://github.com/w3c/web-platform-tests/compare/069ff216c8ba...a3fb594ee776
  231. # [23:48] -gitbot:#testing- web-platform-tests/master 691f05c Tobie Langel: Make the coverage tool rely on the specref db.
  232. # [23:48] -gitbot:#testing- web-platform-tests/master a3fb594 Tobie Langel: Merge pull request #344 from tobie/coverage-tool-updates...
  233. # [23:48] * Parts: gitbot (~gitbot@public.cloak) (gitbot)
  234. # [23:57] * Joins: AutomatedTester (~AutomatedTester@public.cloak)
  235. # [23:57] * Quits: Automate_ (~AutomatedTester@public.cloak) (Client closed connection)
  236. # Session Close: Sat Sep 21 00:00:00 2013

The end :)