13:30:02 <mmcgrath> #startmeeting The release
13:30:02 <zodbot> Meeting started Tue May 25 13:30:02 2010 UTC.  The chair is mmcgrath. Information about MeetBot at http://wiki.debian.org/MeetBot.
13:30:02 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
13:30:20 <mdomsch> #action add task to hide -Alpha and -Beta releases from the mirrorlist on release day
13:30:22 <ricky> I'm going to push spins.fp.o ahead of time
13:30:40 <mmcgrath> #actiln getting more seeders ready
13:30:40 <smooge> but I don't want to do rewriterules
13:30:46 <mmcgrath> nirik: how's your torrent stuff looking?
13:30:48 <ricky> smooge: Redirect 301 /originalpath /newpath
13:30:53 * mdomsch has a seeder running
13:30:58 <smooge> oh
13:31:01 <ricky> No mod_rewrite needed :-)
13:38:48 <stickster> #info Announcement for mailing list is queued, checked all links
13:42:07 <ricky> Heh, a syncStatic takes far longer in production than in staging
13:42:37 <Oxf13> morning
13:43:08 <stickster> Hiya Oxf13
13:43:09 <Oxf13> oh, let me get some ini files in place for torrents
13:43:28 * stickster has PR folks on hold for the Red Hat press release
13:43:36 <ricky> Maybe it has to do something with load average: 37.93, 35.87, 32.98 :-)
13:43:46 <mmcgrath> ricky: where you seeing that load average at?
13:43:48 <stickster> wowsers
13:43:53 <ricky> bapp01
13:44:08 <ricky> (The one place where it's normal, I guess)
13:44:28 <mmcgrath> eh, that's still a bit high, it's all cpu time too
13:44:47 <ricky> Maybe time for a bapp02 at some pint
13:44:48 <ricky> **point
13:45:17 <ricky> There's crawling and transifex pulling going on
13:45:35 <ricky> Website build took almost 13 minutes there, takes less than 5 on app01.stg
13:45:47 <smooge> ricky, mmcgrath look at http://fpaste.org/L7qM/ and let me know what I need to do
13:46:07 <ricky> smooge: Did you add that file in fedora-docs/manifests/init.pp?
13:46:15 <smooge> not yet
13:46:23 <Oxf13> ok, torrent inis in place
13:46:27 <ricky> (Just copy the fedora-docs.conf block from above and replace it with redirects.conf)
13:47:01 <mmcgrath> get that redirects pushed out asap, we don't want that syncing seconds after the announcement goes out.
13:47:35 <mdomsch> we can kill crawlers if we need
13:47:36 * stickster can hold announcement until things get confirmed
13:47:48 <smooge> ricky, thanks for catching that
13:47:50 <mdomsch> tx looks like the killer right now
13:47:55 <stickster> Should I give PR folks a target time after 10:00, like 10:15 or 10:30?
13:48:05 <ricky> mdomsch: Nah, not a problem, the websites build is finished now
13:48:31 <ricky> smooge: I sent my +1
13:48:41 <ricky> smooge: I imagine mmcgrath is happy to +1 too
13:48:50 <mmcgrath> +1
13:49:12 <mmcgrath> ricky: Oxf13: we on pace for a 10:00 release?
13:49:20 <Oxf13> I am
13:49:21 <ricky> Websites are ready to go
13:49:46 <smooge> ricky, pushed.. needs a puppetd -t on the proxy servers
13:50:11 * smooge wonders if a func call would work that way
13:50:17 <ricky> smooge: I just ran this:
13:50:18 <ricky> make HOSTS="proxy1 proxy3 proxy4 proxy5 proxy6 proxy7" push
13:50:21 <stickster> Oxf13: ricky: mmcgrath: smooge: OK, shall I wave the checkered flag for the PR folks to push promo stuff to redhat.com and the press release for a 10am go?
13:50:24 <ricky> Handy to know for future reference
13:50:28 <ricky> In puppet/
13:50:50 <smooge> interesting
13:50:51 <Oxf13> stickster: as far as I'm concerned, yes.  Most of my stuff was done last night.
13:50:58 <Oxf13> how are we looking on amount of mirrors open?
13:50:59 <mmcgrath> make push HOSTS :)
13:51:10 <ricky> smooge: It works via snmp, might not work for some machines due to firewall rules, but it's very handy for doing all the proxies
13:51:18 <mmcgrath> Oxf13: I'll re check, it was 90% earlier
13:51:39 <ricky> Hm, proxy7 isn't in func
13:51:44 <ricky> But it isn't live either, so no problem
13:52:03 <ricky> Er, I take that back
13:52:03 <ricky> Adding it now
13:52:10 <smooge> hmmm I thought I got it in func
13:53:20 <ricky> Aaaand I have no idea how to add it.
13:53:20 <mmcgrath> Oxf13: yeah still sitting around 90%
13:53:26 <ricky> I can just run the commands manually
13:54:51 <Oxf13> so I think we're just waiting for the ricky vote on pulling the trigger
13:55:00 <ricky> Oh, nothing blocking from here
13:55:16 <ricky> Just let me know when you want the website pushed
13:55:48 <mdomsch> mmcgrath, who is that last 10%?
13:55:52 <mdomsch> Oxf13, when was the bitflip?
13:56:01 <mmcgrath> we're starting to see an up-tick in proxy requests - http://mmcgrath.fedorapeople.org/monitoring/proxy-requests.html
13:56:18 <mmcgrath> mdomsch: http://fpaste.org/cN3t/]
13:56:21 <mmcgrath> oops - http://fpaste.org/cN3t/
13:56:24 <Oxf13> mdomsch: I did it around midnight pacific time
13:56:33 <Oxf13> nearly 7 hours ago
13:56:45 <Oxf13> ricky: lets push the website now
13:56:55 <ricky> OK, running
13:57:19 <stickster> And with that, I'm going to have Red Hat move their bits out too -- it will take 10-15 minutes for them to show up as well.
13:57:44 * stickster will save ML announcement until websites are ready
13:58:31 <ricky> OK, website's out, doing spins now
13:58:42 * mmcgrath does a link check
13:59:42 <mmcgrath> ricky: FYI, http://stg.fedoraproject.org/en/get-fedora != http://fedoraproject.org/en/get-fedora
14:00:04 <ricky> They look the same to me
14:00:04 <mmcgrath> ricky: nm, there it goes.
14:00:12 <ricky> Ah, maybe you hit proxy6
14:00:21 <ricky> I'm syncing manually there now as well, func didn't hit it either.
14:00:36 <ricky> It hit it fine for spins though
14:00:48 <ricky> Weird?
14:01:36 <stickster> mmcgrath: ricky: OK to release ML announcement now?
14:01:49 * ricky is fine with it
14:02:38 <mmcgrath> stickster: looks good.  I got 100% link coverage
14:02:53 * mmcgrath finds the clickstream of the new design a bit confusing
14:04:17 <ricky> A result of trying to include everything :-(  At least it's not on the main download page though
14:04:35 <mmcgrath> ricky: yeah, it's still much better.  Don't get me wrong
14:04:51 <mmcgrath> https://admin.fedoraproject.org/collectd/bin/graph.cgi?hostname=proxy01;plugin=apache;type=apache_bytes;begin=-3600 <-- weird
14:05:02 <ricky> Wow :-)
14:06:24 <rsc> stickster: why "http://get.fedoraproject.org?F13an" instead of "http://get.fedoraproject.org"?
14:06:32 <ricky> For link tracking purposes
14:06:36 <ricky> So we know where people came from
14:06:41 <rsc> lol
14:07:18 <ke4zvu3> rsc: http://lists.fedoraproject.org/pipermail/marketing/2010-May/012918.html
14:07:19 <herlo> rsc: feel free to use a tinyurl or bit.ly or soemthing
14:07:23 <Oxf13> Date: 	Tue, 25 May 2010 10:03:40 -0400
14:07:26 <ricky> Have we collected any data from thos eyet?
14:07:28 <ricky> **yet
14:07:33 <Oxf13> that might be the closest to 10:00 am release we've ever done (:
14:07:48 <stickster> herlo: rsc: There are links prepared at https://fedoraproject.org/wiki/Link_tracking
14:08:04 <ricky> Heh, running the website sync half an hour in advance helped :-)
14:08:04 <stickster> herlo: rsc: Using 100% free software ur1.ca site, no less :-)
14:09:05 <stickster> ricky: Not yet -- we need a more solid Marketing SOP for them to make sure we're measuring apples to apples
14:09:16 <ricky> OK
14:09:22 <herlo> stickster: yeah, figured :)
14:09:29 <ricky> Anybody have a .announce prepared?
14:10:03 <Oxf13> $ rm -rf 13.RC3/
14:10:03 <Oxf13> rm: cannot remove `13.RC3//Fedora/i386/iso/.nfs000000000094f46100000012': Device or resource busy
14:10:15 <Oxf13> mmcgrath: more wackyness with that secondary1 storage
14:10:28 <stickster> ricky: No, but I can paste my recent dent
14:11:01 <ricky> Sounds good, go for it
14:11:02 <stickster> ricky: Sound good?
14:11:07 <stickster> ah, ok :-D
14:11:42 <stickster> done
14:11:45 <mmcgrath> Oxf13: looking
14:13:09 <mmcgrath> Oxf13: that's interesting.  I assume that's not somethign you actually created?
14:13:17 <Oxf13> that is correct
14:13:30 <mmcgrath> marek: ping
14:13:50 <zodbot> Announcement from my owner (stickster): Happy Fedora 13 Release Day! http://ur1.ca/03bfg -- to download: http://ur1.ca/03ajw
14:15:05 <marek> mmcgrath: pong
14:15:08 <Oxf13> mdomsch: just to confirm, redirects for releases/13/ are disabled correct?
14:16:00 <mdomsch> Oxf13, they still point at development/13 but the content is the same
14:16:05 <mdomsch> I can drop them now
14:16:10 <mmcgrath> marek: hey, just created a weird ticket.  I'll give you the ID if you want to look (when I get it back)
14:16:10 <ricky> Note that there are pretty much 0 torrent links at fp.o
14:16:17 <Oxf13> please do, I just closed the ticket to make that happen
14:16:25 <ricky> So there may be more via mirrors this time.
14:17:54 <mdomsch> np
14:18:16 <mmcgrath> we're serving around 380 requests / second at the moment
14:20:36 <ricky> I'm getting mirrors.txt
14:20:42 <ricky> Er, releases.txt
14:21:31 <mmcgrath> ricky: thanks I was just about to do that :)
14:23:15 <mdomsch> Oxf13, done
14:23:22 <mdomsch> will take effect top of the hour
14:23:27 <mmcgrath> marek:  - INC000000198176
14:24:18 <ricky> Thanks to whoever got func on proxy7
14:25:20 <marek> mmcgrath: let me find someone to have a look at it
14:29:15 <mmcgrath> marek: thanks, appreciated.  not a huge major deal or anything but pretty weird :_/
14:29:26 <marek> :)
14:30:39 <smooge> what can I help with at the moment
14:31:13 <mmcgrath> smooge: right now we're just trying to pay attention to everything :)
14:33:11 <mmcgrath> mdomsch: barf, the anaconda redirect as method didn't get fixed :(
14:33:25 <mdomsch> mmcgrath, ?
14:33:37 <mdomsch> which redirect?
14:33:39 <mmcgrath> BFO is still using - method=http://download.fedora.redhat.com/pub/fedora/linux/releases/13/Fedora/i386/os/
14:33:49 <mmcgrath> because download.fedoraproject.org doesn't work
14:33:50 <mdomsch> ouch
14:34:00 <mdomsch> oh, anaconda doesn't handle the redirect
14:34:04 <mmcgrath> yeah
14:35:00 <mdomsch> so is BFO getting used so much taht d.f.r.c can't handle it?
14:35:16 <mmcgrath> not yet
14:35:16 <mdomsch> of so, yea!, and "oh"
14:35:26 <mmcgrath> and I suspect it we'll be ok.
14:35:54 <mmcgrath> if we need to get real fancy we can have it serve that as a server side script that changes method on download.
14:36:27 <nirik> mmcgrath: torrents are humming along here. ;)
14:36:55 * mdomsch seeding at only 200KB/sec
14:37:09 <mdomsch> cap is above 750
14:37:18 <Jeff_S> hrm, my rsync from warthog9 is failing... should I just manually bit-flip for now, or are there new things being pushed as well?
14:37:32 <mdomsch> Jeff_S, manual please
14:37:38 <Jeff_S> mdomsch: ok, thanks
14:37:59 <heffer> mdomsch, how can I get the metalink files for the isos from mm? and do they include the torrent information?
14:38:39 <mdomsch> heffer, the metalink files for isos are per-ISO, not grouped, and do not inlcude torrent info
14:38:56 <ricky> http://fedoraproject.org/get-fedora-all
14:39:04 <zodbot> Ticket notification - infrastructure: Ticket #2144 (bug closed): Template modification (FedoraVersion <https://fedorahosted.org/fedora-infrastructure/ticket/2144#comment:2>
14:39:07 <ricky> Not linked from anywhere at the moment, but it has everything
14:39:25 <mdomsch> heffer, http://mirrors.fedoraproject.org/metalink?path=pub/fedora/linux/releases/13/Fedora/x86_64/iso/Fedora-13-x86_64-DVD.iso
14:39:33 <heffer> mdomsch, too bad. because using them with aria2 in http/ftp/torrent mode would be really nice.
14:39:47 <heffer> mdomsch, thanks for the link
14:39:51 <mdomsch> heffer, just a small matter of code I haven't had time to write
14:40:01 <heffer> mdomsch, okay :)
14:44:11 <mmcgrath> ricky: ping
14:44:18 <ricky> mmcgrath: pong
14:44:34 <mmcgrath> ricky: can you fix the F13 release.txt name.  It says (Branch) :)
14:44:55 * mmcgrath is going through and double checking everything
14:45:08 <ricky> Blah, I really should have slept last night
14:45:11 <ricky> Good thing you checked
14:46:29 <CodeBlock> ricky: hehe, you were too excited for the release ;)
14:49:01 <mmcgrath> https://admin.fedoraproject.org/collectd/bin/index.cgi?hostname=proxy3&plugin=apache&timespan=3600&action=show_selection&ok_button=OK
14:49:15 <mmcgrath> keepalives are taking up a lot of our scoreboard at the moment.
14:51:55 <mmcgrath> hmm, we're seeing a lot of docs errors
14:52:23 <mmcgrath> stickster: who's someone good to talk about wrt docs errors?
14:52:49 <mmcgrath> ricky: hey got a minute?
14:53:31 <ricky> mmcgrath: Sure
14:53:44 <mmcgrath> ricky: ssh to proxy3 and tail -f *error.log
14:54:02 <stickster> mmcgrath: Probably best is rudi, but he's in BNE so he may have finally dropped off for some shuteye
14:54:10 <mmcgrath> :-/
14:54:10 <stickster> I can try to help as much as I can
14:54:26 <stickster> We can make sure sparks is here too, as Docs team lead.
14:54:27 <mmcgrath> stickster: I'm not sure what is including this - http://docs.fedoraproject.org/es-ES/Fedora/13/html/Release_Notes/Common_Content/css/default.css
14:54:34 <mmcgrath> but it's refering to css docs that don't exist
14:54:46 <mmcgrath> so browsers try to request them and they don't return
14:55:14 <ricky> Hm, I'm guessing that needs to be fixed on the publican side
14:55:32 <stickster> mmcgrath: Should we put empty docs in for those? ricky is right, they're generated by the Publican brand bits
14:55:50 <mmcgrath> heh, that's a fun one - [Tue May 25 14:55:32 2010] [error] [client 67.207.201.165] (36)File name too long: access to /get-fedora to download a LiveCD, regardless of what operating system youre running. This will give you a working version of Fedora, complete with common applications, all running off your CD drive - your hard drive wont be touched at all.And when youre ready, installation is just a click away.\nWant an even more enjoyable way to use F
14:56:03 <ricky> Hehe
14:56:11 <stickster> wow
14:56:14 <ricky> Mispaste, happens to me all the time
14:56:20 * stickster too :-)
14:56:33 <ricky> Good thing my URLs usually don't have rm -rfs in them :-)
14:57:14 <stickster> mmcgrath: I can generate empty files for those CSS as a temporary fix and put them in our docs/web git repo
14:57:15 <ke4qqq> mmcgrath: so the 404 on the docs site is pushing them to the error/welcome page? which means the docs are basically inaccessible I assume
14:57:31 <mmcgrath> ke4qqq: beats me, you'll have to talk to the docs guys.
14:57:34 <stickster> ke4qqq: I wouldn't think so, just generating lots of other 404s in the logs.
14:57:42 <stickster> I just loaded a bunch of docs with no problem.
14:57:57 <ke4qqq> stickster: lang.css is the problem -
14:58:04 <ricky> The error pages should redirect to /
14:58:09 <ricky> So nothing should be inaccessible
14:58:28 <ricky> Well, except for that langs.css file :-)
14:58:47 <stickster> http://docs.fedoraproject.org/es-ES/Fedora/13/html/Release_Notes/index.html <-- seems to work OK
15:01:37 <stickster> mmcgrath: Talking on #fedora-docs about this now
15:01:48 <mmcgrath> ricky: yeah, I can see in yslow, it's actually downloading and trying to run the main page stuff for all css docs that are missing
15:01:59 <mmcgrath> which isn't breaking the page, but is very inefficient
15:02:03 <mmcgrath> stickster: thanks
15:02:10 <stickster> If we have a testing doc for staging sites, I recommend adding this to the test measurements
15:02:14 <CodeBlock> mmcgrath: anything I can help out with as far as release-day heckticness?
15:02:32 <ricky> mmcgrath: Shall I whip up a rewriterule?
15:03:25 <mmcgrath> ricky: I'm not sure.
15:03:39 <ricky> It would fix the extra useless downloading
15:03:45 <mmcgrath> stickster: this is something that could be tested before it gets to staging
15:03:46 <ke4qqq> ricky: empty file wouldn't be easier? (/me doesn't claim to know the answer, just asking)
15:03:47 <ricky> Any downsides?
15:04:02 <ricky> ke4qqq: It's under a bunch of paths :-/
15:04:09 <mmcgrath> ricky: I say hold off for now and let the docs team see if they can fix it and push it out.
15:04:23 <ricky> Publican is designed to have each language have its own files and be self-contained
15:04:26 <ricky> OK
15:04:29 <mmcgrath> it doesn't seem to be causing anything horrible.
15:04:58 <mmcgrath> ricky: here's one you could fix - [Tue May 25 15:04:42 2010] [error] [client 59.172.157.158] File does not exist: /srv/web/spins.fedoraproject.org/favicon.ico
15:05:03 <ricky> This is a good lesson to avoid last-minute stuff next time.  Hopefully we've mitigated most of the issues
15:05:07 <ricky> OK
15:05:16 <mmcgrath> [Tue May 25 15:04:47 2010] [error] [client 67.207.201.163] (36)File name too long: access to /get-fedora to download a LiveCD, regardless of what operating system youre running. This will give you a working version of Fedora, complete with common applications, all running off your CD drive - your hard drive wont be touched at all.And when youre ready, installation is just a click away.\nWant an even more enjoyable way to use Fedora, risk-free failed
15:05:22 <mmcgrath> that one just popped up again :)
15:05:43 <CodeBlock> hah
15:05:48 <ricky> Haha, from a slightly different IP too
15:05:51 <ricky> Same guy, I guess
15:06:10 <ke4qqq> mmcgrath: twice??? is that a bad link somewhere?
15:06:38 <ricky> The IPs are 2 apart, for what it's worth
15:06:54 <mmcgrath> ah, it's a bot.
15:06:57 * ricky would hope our link checking would have caught that - did a lot of that last night
15:06:59 <mmcgrath> +http://www.chainn.com/mxbot.html
15:07:01 <mmcgrath> whatever that is
15:07:06 <ricky> Hah, smart bot
15:07:37 <ke4qqq> interesting
15:07:45 <CodeBlock> mmcgrath: Anything I can help out with, or are we good? I have a few minutes, then I'll be gone for ~45 minutes, then I'll have quite a while if we need anything
15:08:00 <mmcgrath> CodeBlock: aside from more torrent seeders I think we're in good shape at the moment
15:08:16 <CodeBlock> mmcgrath: I'm already seeding, grabbed it last night, heh ;)
15:08:27 * nirik is pushing about 4800KB/sec for torrents now.
15:08:32 <CodeBlock> I'll bump up the throttle on my vps a bit for it
15:09:06 <rigeld2> I'd seed at SHSU but we block torrenting in and out, so... :-/
15:09:22 * rigeld2 stabs his security guy
15:09:36 <mmcgrath> yeah torrent is dangerous
15:09:36 <ricky> Whoever designed the idea of /favicon.ico..
15:09:41 <CodeBlock> mmcgrath: actually I'm only seeding -live, I'll grab some of the others I guess
15:09:51 <rigeld2> ricky++
15:13:23 <CodeBlock> mmcgrath: Which torrents really need some seeders atm, any particular ones?
15:14:01 <mmcgrath> It's hard to tell, you can look at http://torrent.fedoraproject.org:6969/ to see whichh ones are downloading the most of
15:14:19 <mmcgrath> Fedora-13-x86_64-DVD looks pretty popular.
15:14:22 <mmcgrath> by an order of magnitude
15:14:36 <mmcgrath> Fedora-13-i386-DVD same with that one
15:14:49 <ricky> Can I get two +1s for this: http://dpaste.com/199000/
15:15:00 <mmcgrath> +1
15:15:02 <daMaestro> umm.. so even i'm confused on where to get the torrents
15:15:22 <daMaestro> rather, i don't see the links anywhere on get-fedora
15:15:36 <mmcgrath> ricky: ^^
15:16:34 <ricky> Yeah, they aren't linked on fp.o at all, it was hard to find space with all the things we had to include :-(
15:16:50 <daMaestro> gah
15:17:24 * ricky wishes we could just list 4 ISOs, desktop/DVD i686 and x86_64.
15:18:22 <mmcgrath> daMaestro: maybe make a note on the websites list
15:18:29 <mmcgrath> probably worth another discussion
15:18:45 * ricky is discussing with sijis in #fedora-websites
15:19:01 <mmcgrath> ah
15:19:15 <ricky> He's in California, so times are a bit weird for him :-)
15:27:35 <CodeBlock> Alright, seeding both arch's -live and -DVD
15:27:37 * CodeBlock goes to get lunch
15:27:55 <mmcgrath> CodeBlock: thanks
15:28:14 <jokajak> i'm currently uploading at 2500KB/s
15:28:19 <jokajak> stupid box won't go any faster
15:39:39 <CodeBlock> mmcgrath: no problem, let me know if you come up with anything for me to do :)
15:40:43 <skvidal> dgilmore, jds2001: ping - mailman puppet config
15:40:59 <skvidal> we have manifests/services/mailman.pp and modules/mailman/
15:41:11 <skvidal> and it sure looks like modules/mailman is NOT being used but the other is
15:41:29 <skvidal> so I'm just trying to figure out which one is the 'right' one
15:44:18 <dgilmore> skvidal: i dont think we got it finshed ported to a module
15:44:33 <skvidal> okay
15:44:38 <skvidal> so my changes from yesterday are moot
15:44:41 <skvidal> I'll go fix them up
15:45:07 <smooge> stickster, do you have more redirects you want me to do?
15:46:28 <inode0> wow, best release day download experience ever
15:47:07 <stickster> smooge: Still investigating
15:47:19 <stickster> smooge: Weirdly, our docs-web repo appears to show that we *have* those CSS files
15:48:36 <stickster> smooge: mmcgrath: http://git.fedorahosted.org/git/?p=docs/web.git;a=tree;f=public_html/es-ES/Common_Content/css;h=f2396150050d46d5e5167f3c9893ade3d99fe450;hb=master
15:50:57 * stickster wishes nb were around to ask
15:54:42 <mmcgrath> stickster: can you build your copy of that and send me the output of a "find" so I can see where all the files are?
15:55:46 <stickster> mmcgrath: http://pfrields.fedorapeople.org/temp/css-list.txt
15:56:39 <gladiac> Hi
15:57:04 <gladiac> has someone ever looked at http://www.mirrorbrain.org/
15:57:22 <ricky> I think mdomsch has
15:57:23 <mdomsch> gladiac, yes
15:57:31 <mdomsch> I've spoken with peter poeml quite a bit
15:57:38 <gladiac> ah ok, fine
15:57:49 <gladiac> then I just hope Fedora will use it one day
15:57:52 <mdomsch> MB is quite nice, MM accomplishes similar things with a different implementation
15:58:05 <mmcgrath> stickster: yeah, do a search on that page for "Release_Notes/Common_Content/" note that lang.css is gone
15:58:19 <mmcgrath> strange prediction :)
15:58:45 <stickster> Oh bugger
15:58:54 <stickster> mmcgrath: Good call. I'll put those in place and push them.
15:59:05 <stickster> mmcgrath: A zero-byte file should be fine there, right?
15:59:33 <mmcgrath> stickster: it'd be even better if it didn't get requested at all but if that's a docbook convention that's hackish to fix a 0byte is fine too
15:59:46 <mmcgrath> unless that file should actually be there?
15:59:55 * mmcgrath isn't sure the mechanism that's actually creating the request and why
16:00:28 <stickster> mmcgrath: It's to accommodate specific language CSS, i.e. RTL in some languages
16:01:06 <stickster> When you build a document for publication in publican, it generates the default.css with those lines
16:01:35 <mmcgrath> <nod>
16:01:36 <stickster> It simply doesn't add a lang.css unless there's one it already knows it needs, like RTL for the he-IL (Hebrew) locale
16:01:43 <mmcgrath> so this could more be a publican bug?
16:01:48 <stickster> mmcgrath: It definitely is
16:01:56 <stickster> But we won't be able to get it fixed and pushed for at least most of a day
16:02:00 <stickster> Since the maintainer is in BNE
16:02:21 <stickster> So the fastest fix that the site's git pull won't overwrite (AFAIK) is putting in 0-byte lang.css for now
16:02:31 <mmcgrath> <nod>
16:05:25 * stickster files Publican bug first to reference in the git changelog
16:07:14 <stickster> mmcgrath: There's a good chance the Publican maintainer will say, "Fix your 404 redirects."
16:07:43 <mmcgrath> sites shouldn't have 404's anyway so even if we fix the redirects, we're still trying to serve content that doesn't exist.
16:09:04 * mmcgrath sees http codes 103 and wonders wtf thats about
16:09:40 <smooge> 103?
16:10:31 <stickster> .bug 595799
16:10:32 <zodbot> stickster: Bug 595799 Missing lang.css file when publishing to web to avoid 404s - https://bugzilla.redhat.com/show_bug.cgi?id=595799
16:10:36 <mmcgrath> smooge: yeah
16:11:35 <smooge> mmcgrath, you probably found this already http://jw.n--tree.net/blog/2009/9/23/http-status-code-103
16:11:39 <smooge> mod cache
16:12:00 <mmcgrath> yeah it's cache related, I just don't get what is causing it and what the client might be seeing.
16:12:10 <stickster> mmcgrath: :-( no access to push my change, apparently
16:12:38 <mmcgrath> ah hah, that's what's going on.
16:12:43 <mmcgrath> the wiki does something similar.
16:12:56 <mmcgrath> stickster: you can't push to the docs site?  O_o
16:13:25 <ricky> Make sure you get in docs-writers and docs-publishers
16:13:44 <ricky> Looks like you're missing the second one
16:13:47 <ricky> sparks: ^
16:14:01 <stickster> Yup, just saw that, fixing
16:22:32 <ricky> If anybody's interested, here are spins stats for F12 (including both i686 and x86_64)
16:22:36 <ricky> http://dpaste.com/199031/
16:23:09 <ricky> Do we have total download.fp.o stats for the DVDs and Desktop/KDE Live media?
16:24:16 <ricky> /wiki/Statistics has combined stats: 1486587 unique IPs
16:25:07 <nirik> those dammed LXDErs beating out Xfce! :)
16:27:55 <ricky> Oxf13: By the way - we were talking about hardlinks and timestamp changes yesterday - if people sync with -H as suggested, then they shouldn't end up pulling more data
16:30:00 <mdomsch> mmcgrath, which proxy is on ipv6?  proxy4?
16:30:06 <ricky> Yup
16:30:16 <mmcgrath> mdomsch: yeah, and shortly proxy2
16:30:28 <ricky> Woah, PHX2 has ipv6?  Cool
16:30:30 <mdomsch> its response time is a lot slower than the others, at least for me
16:30:38 <mmcgrath> ricky: nope, it'll be in the UK
16:30:40 <mdomsch> ricky, no
16:30:48 <ricky> Ohh, that proxy2 :-)
16:31:04 <mdomsch> $ time wget -6 -S -O - 'https://mirrors.fedoraproject.org/mirrorlist?repo=fedora-13&arch=x86_64&country=global' | wc -l
16:32:01 <mdomsch> https://admin.fedoraproject.org/collectd/bin/index.cgi?hostname=proxy4&plugin=apache&timespan=3600&action=show_selection&ok_button=OK
16:33:08 <mmcgrath> mdomsch: it's ipv6 specific, not proxy4 specific -
16:33:10 <mmcgrath> time wget -SO- --header="Host: mirrors.fedoraproject.org" 'http://152.46.7.222/mirrorlist?repo=fedora-13&arch=x86_64&country=global' | wc -l
16:33:19 <mmcgrath> ricky: the new proxy2 :)
16:33:27 <mdomsch> mmcgrath, interesting...
16:33:50 <mmcgrath> dgilmore: ping
16:33:58 <mmcgrath> mdomsch: can you time this?
16:34:13 <mmcgrath> time wget -6 -S -O - 'http://fedoraproject.org/' | wc -l
16:35:22 <dgilmore> mmcgrath: pong
16:35:25 <mdomsch> mmcgrath, it's fine now - I restarted my aiccu tunnel
16:35:32 <mdomsch> must have been a problem on my end
16:35:43 <mmcgrath> dgilmore: how's your ipv6 looking to Fedora services, particularly mirrors.fp.o
16:35:45 <mdomsch> 0.3 - 0.4sec now
16:35:48 <mmcgrath> mdomsch: interesting, lets get some verifications
16:35:55 <mdomsch> please do
16:35:59 <mmcgrath> mdomsch: do you have header info from the -S portion of that request?
16:36:23 <stickster> mmcgrath: ricky: OK, got my group access, now just need to wait for propagation
16:36:56 <mdomsch> mmcgrath, http://fpaste.org/L8Q1/
16:37:38 <ricky> stickster: It's already synced to hosted
16:37:41 <stickster> mmcgrath: ricky: OK, changes pushed, should be in next sync
16:38:13 <dgilmore> mmcgrath: im getting 403's
16:38:30 <mmcgrath> mdomsch: do you have the headers from the slow request?
16:38:38 <mmcgrath> dgilmore: 403 to what address?
16:38:43 <mmcgrath> err URL
16:39:22 <dgilmore> mmcgrath: i ran time wget -6 -S -O - 'https://mirrors.fedoraproject.org/mirrorlist?repo=fedora-13&arch=x86_64&country=global' | wc -l
16:40:07 <dgilmore> Resolving mirrors.fedoraproject.org... 2610:28:200:1::fed0:2
16:40:07 <dgilmore> Connecting to mirrors.fedoraproject.org|2610:28:200:1::fed0:2|:443... failed: Permission denied.
16:40:49 <mmcgrath> permission denied to a 443 port?
16:40:51 <mmcgrath> that's... weird
16:41:03 <dgilmore> i can ping6 the host ok
16:41:21 <mmcgrath> dgilmore: wget -6 -SO/dev/null http://fedoraproject.org/
16:41:22 <mmcgrath> try that
16:41:37 <mmcgrath> dgilmore: what headers did you get?
16:41:42 <herlo> an interesting question here, but is it intended that no torrent options are available on get.fedoraproject.org?
16:41:49 <dgilmore> permission denied
16:41:58 <herlo> I can't help share if I can't find the links...
16:41:59 <mmcgrath> herlo: I'm not sure what the deal is with that, check in #fedora-websites I know they've been discussing it.
16:42:06 <herlo> mmcgrath: cool, tx
16:42:13 <nirik> http://torrents.fedoraproject.org/ ;)
16:42:28 <nirik> (which is where we have been pointing folks in #fedora who can't find them)
16:42:37 <ricky> It's intended
16:42:50 <ricky> We discussed this with mizmo, and decided to leave it as is
16:42:53 <mmcgrath> so are we going to slowly phase out torrents?  what's the plan there?
16:42:56 <dgilmore> mmcgrath: i think its something with me
16:43:02 <herlo> nirik: but that's not my point :)
16:43:06 <mmcgrath> dgilmore: selinux?  :)
16:43:07 <dgilmore> mmcgrath:  i cant access ipv6.google.com either
16:43:14 * ricky personally wanted torrents to be linked somewhere, but ah well.
16:43:24 <dgilmore> mmcgrath: selinux is in permissive
16:43:31 <nirik> herlo: I'm just saying we are seeing that in #fedora a lot too... ;)
16:43:34 <mmcgrath> ip6tables?
16:43:34 <herlo> ricky: I think I have an easy solution for the future and could hack it in quickly...
16:43:35 <dgilmore> since my kvm setup doesnt work in enforcing mode
16:48:29 <mdomsch> dgilmore, I had to restart my sixxs tunnel
16:49:04 <dgilmore> mdomsch: i did that
16:49:24 <dgilmore> mdomsch: its something on my external router box
16:49:36 <dgilmore> i run aiccu on my colo'd box
16:49:45 <dgilmore> and have a tunnel from home to it
16:52:29 <dgilmore> i think its actually something outside of my control
16:53:43 <dgilmore> ssh over ipv6 is working
16:53:48 <dgilmore> but http and https is not
16:54:03 <mmcgrath> dgilmore: not ipv6tables?
16:54:30 <dgilmore> mmcgrath: no my router box has no rules blocking forwarding packets
17:07:12 <Viking-Ice> One question I just migrated all my mailing list subscription to gmail and do I need to ping each list admin to accept my new address be able to reply and post to the lists ?
17:07:28 <Oxf13> ricky: it'll still cause some shuffle and disk I/O on the side of the mirrors, removing existing files and replacing them with hardlinks.
17:07:36 <mdomsch> warthog9, how's your traffic look?
17:09:07 <mdomsch> Viking-Ice, most fedora lists don't require admin approval to join
17:09:38 <skvidal> stickster: just curious - what's in 'oldguides' under docs.fp.o/
17:10:39 <skvidal> also - is there a reason why our docs.fp.o subdir on the proxy servers comes with a .git subdir that's 2.9GB?
17:10:41 <Viking-Ice> mdomsch: After the change I'm unable to post to the list ( receiving is fine ) mailing address being blocked for some reason ?
17:12:12 <skvidal> nb: ping
17:12:42 <skvidal> nb: so the docs.fp.o subdir that's on all the proxy servers - it's 6.6gb and it has 2 troubling items
17:13:01 <skvidal> nb: 1. it has a .git dir that's not being excluded so 2.9GB of files no one will ever see are in the dir
17:13:19 <skvidal> 2. oldguides appears to have all the former docs? for releases going back to f8?
17:13:23 <skvidal> which seems..... odd
17:13:51 <mdomsch> skvidal: git clone http://docs.fedoraproject.org/ ? :-)
17:14:04 <skvidal> mdomsch: it's replicated to all the proxy servers
17:14:18 <skvidal> so each one of them is carting around 2.9GB they don't need
17:14:20 <stickster> skvidal: I'm probably not the best person to ask about how that repo works -- nb set it up for Docs
17:14:20 <skvidal> right?
17:14:22 <mdomsch> skvidal, agreed
17:15:04 <stickster> As far as oldguides goes, I *think* rudi is working on making them available through the site menu -- we previously had them on the old docs.fp.o site as well
17:15:56 <skvidal> okay
17:16:12 <skvidal> they're 1.4GB and maybe it wouldn't be rude to nuke the f10 and below versions?
17:16:24 <stickster> skvidal: I think f10 is on the menu already
17:16:27 <stickster> http://docs.fedoraproject.org
17:16:56 <skvidal> why?
17:17:20 <Viking-Ice> mdomsch: ignore-it post present in list archive client problem..
17:17:38 <stickster> skvidal: ----> docs@lists.fp.o
17:17:46 <skvidal> righty-o
17:17:47 <ricky> Oxf13: Sure.  I guess you could ask around on mirror-list-d and see what they think
17:18:01 <Oxf13> yeah, I'm sure they'll be for it, once we give them a heads up
17:18:03 <Oxf13> just not during release time
17:18:12 <ricky> Oh yeah
17:18:19 <stickster> skvidal: I truly don't know, I asked about carting around old docs a year or so ago, and the team said they wanted to retain old docs
17:18:32 <stickster> skvidal: However, the new site breaks the old links anyway, so the value's somewhat lessened of late
17:20:05 <davelien> mirror.its.uidaho.edu is getting about 100Mb of traffic, I can handle at least 200Mb commercial plus up to a Gb of I2/NLR.  Feel free to send me twice as much traffic, if needed, and if possible.
17:23:25 * stickster is wondering where the Moblin spin went
17:24:01 <mdomsch> davelien, I've bumped your bandwidth value in MM.  watch it after the top of the next hour
17:24:59 <Jeff_S> mdomsch: can mirror admins control that value?
17:25:09 <mdomsch> Jeff_S, yes
17:25:23 <mdomsch> it is nominally the bandwidth in Mbits/sec you can serve
17:25:37 <Jeff_S> is that the same as the number shown on the publiclist page?
17:25:40 <mdomsch> but it simply feeds the weighted random draw
17:25:41 <mdomsch> yes
17:25:50 <Jeff_S> oh, ours is quite low there, I should change it
17:26:43 <davelien> If I remember correct the only choices on the admin page are 100Mb or 1Gb, has this changed?
17:26:53 <mdomsch> davelien, it's just a number now
17:27:10 <Jeff_S> davelien: also, thanks for your mirror, I use it occasionally for stuff we don't have ;)
17:29:15 <davelien> Ah, right you are Jeff_S, just checked to page.  I'm set to 600Mb so I guess I'll just let the weighted random draw algorithm do its thing.
17:29:27 <davelien> Jeff_S, you are welcome!
17:29:32 <Jeff_S> davelien: I just do what mdomsch tells me ;)
17:29:56 <Jeff_S> I just bumped up our number to 500 to see what happens
17:30:53 <mdomsch> davelien, I just bumped you from 300 to 600
17:32:47 <stickster> Oxf13: We appear to be missing a spin -- Moblin's gone MIA
17:34:12 <Oxf13> stickster: not listed at https://fedoraproject.org/wiki/Releases/13/Spins
17:34:58 <stickster> Oxf13: https://fedoraproject.org/wiki/Spins_Process says Spins SIG is supposed to notify people
17:35:53 <Oxf13> stickster: yes, the spin sig doesn't really work
17:35:59 * nirik is sorry if something broke down. I thought moblin was approved, but then in limbo, then ok.
17:35:59 <Oxf13> stickster: I'm surprised you haven't noticed.
17:36:23 <stickster> The last list I saw from the Spins SIG was the notification here: http://lists.fedoraproject.org/pipermail/rel-eng/2010-February/009387.html
17:37:04 <notting> i thought moblin was on hold due to upstream going a little odd in the meego merge/rebranding
17:37:30 <nirik> I wish it did... I'm not sure I have time or energy to make it work, but I guess if noone else appears I should make time to do so next cycle.
17:37:31 <stickster> notting: It's not on hold, but I know Peter is looking to move to MeeGo after they have a genuine release
17:38:14 <stickster> Board approved Moblin in January: http://lists.fedoraproject.org/pipermail/advisory-board/2010-January/007869.html
17:38:16 <notting> stickster: but i thought what was in rawhide was a pre-release of a moblin-that-will-never-be-released-as-moblin software set. or something.
17:38:30 <notting> i could be misremembering
17:38:33 <nirik> notting: yes, but then they did finally release 2.1
17:38:37 <nirik> (I think)
17:38:50 <stickster> They did
17:39:43 <Oxf13> nobody updated the wiki page, which is what I was told to use as the definitive list of what to compose
17:39:54 <Oxf13> and nobody mentioned anything missing when I posted to the spins list asking maintainers to review
17:40:09 <Oxf13> so no moblin for F13
17:40:11 <nirik> process fail.
17:40:37 <lmacken_> jwb, Oxf13: missing signatures from the latest push?
17:40:47 <stickster> How can we get the Moblin spin out there, since we've told people in many places it's available?
17:40:52 <jwb> lmacken_, no?
17:41:06 <lmacken_> jwb: ok... well, something fucked up :)
17:41:10 <Oxf13> stickster: do you really want to push a completely untested iso out there?
17:41:25 <jwb> lmacken_, oh that.  koji just pruned signed copies.  it happens a lot
17:41:27 <jwb> i'll fix it
17:41:40 <lmacken_> jwb: thanks
17:41:58 <notting> stickster: were you/people not working off of https://fedoraproject.org/wiki/Releases/13/Spins ?
17:42:31 <stickster> notting: Apparently not everyone was -- I got an email from pbrobinson asking where the Moblin spin was
17:43:07 <stickster> I just read the Spins process page and it says that wiki page "should be updated" -- note the passive voice unfortunately doesn't assign any responsibility for doing it.
17:44:15 <poelcat> stickster: spins wrangler ?
17:44:20 <Oxf13> given that probinson didn't say anything when I asked the spin sig members to test their spins two weeks ago, I'm less than eager to scramble to rush something out.
17:44:45 <Oxf13> the person who was managing spins seems to have walked away from it, without a lot of notice (or without me noticing it much other than the lack of work)
17:45:11 <jwb> spins: not one of our better examples of "value"
17:45:32 <Oxf13> spins: we want it all, but we don't want to do any work for it
17:45:58 <nirik> bruno and I are somewhat around for spins, but we are both busy with other stuff. Most of the other spins maintainers don't provide any input at alal.
17:46:01 <skvidal> jwb: it's the 5th f
17:46:12 <jwb> ?
17:46:18 <skvidal> freedom
17:46:18 <stickster> ail
17:46:20 <skvidal> friends
17:46:22 <skvidal> first
17:46:24 <skvidal> fucked
17:46:24 <Oxf13> nirik: yeah, I think you've gone above and beyond
17:46:26 <skvidal> oh hmm
17:46:28 <skvidal> fail
17:46:28 <Oxf13> spins weren't your bag to carry
17:46:30 <skvidal> yah that works
17:46:45 <nirik> Oxf13: I made a choice to try and help it out, but not run it... just ENOTIME
17:46:48 * skvidal is kidding
17:47:07 <skvidal> if the biggest mess up is miscommunication on one of the spins then that's looking awfully good :)
17:47:15 <Oxf13> nirik: I made a choice specifically not not jump in and mop it up.  I didn't want to be left holding the bag.
17:47:43 <Oxf13> skvidal: it's really just the biggest example of the process breakdown, or the most visible.
17:47:46 <jwb> frustration, fucked, fail, finicky, flagellation, fap, fubar
17:47:49 <stickster> skvidal: Oxf13: That's fair -- Spin owners have to understand the process. I don't think it changed between F11->F13,did it?
17:47:54 <jwb> all good choices
17:48:03 <nirik> stickster: nope... should be all the same from what I recall.
17:48:15 <Oxf13> stickster: not really, not that there has ever been a workable process
17:48:30 <jwb> there really hasn't
17:48:37 * stickster feels bad that pbrobinson is left in the lurch again. This happened with Moblin before.
17:48:39 * Oxf13 notes that we list Haskell on the wiki page, but there wasn't a KS for it to make
17:48:51 <nirik> I think it could be better, but there is a process.
17:48:58 <Oxf13> stickster: then he should have been motivated to make sure everything was going according to plan
17:49:03 <Oxf13> weeks ago, when I asked for people to.
17:49:08 <nirik> Haskell is a ks only spin... shouldn't be produced, just added to spins-kickstarts.
17:49:23 <poelcat> there IS a spins process, but not always full attention to detail in following it or managing it
17:49:24 <Oxf13> nirik: yeah, but then why i sit listed at https://fedoraproject.org/wiki/Releases/13/Spins ?
17:49:45 <nirik> Oxf13: to advertise it's existance in f13?
17:50:10 <Oxf13> Here is this thing which we have nothing to give you...
17:50:22 <nirik> a ks file?
17:50:30 <Oxf13> (sorry, just being snarky here)
17:50:33 <jwb> it's our new plan to attract contributors
17:50:47 <jwb> WANT THIS COOL THING?  GOOD!  MAKE IT YOURSELF!
17:52:42 <spoleeba> mmcgrath, ping
17:52:48 <nirik> The process needs revisiting, and a leader/etc needs to exist.
17:55:05 * nirik moves on from this relaxing and pleasent conversation to prep for the fesco meeting today. More fun. ;)
17:55:09 <ricky> Heh
17:56:10 <skvidal> nirik: I could probably arrange to make fesco equally as relaxing and pleasant, if you'd like
17:56:18 <skvidal> nirik: I'm sure I could rile kofler up
17:56:38 <nirik> the challenge is to not do so. ;)
17:58:11 <skvidal> nirik: nah
17:58:14 <mizmo> skvidal, too late
17:58:18 * ricky goes off for a while, see you later
17:58:21 <skvidal> mizmo: :)
17:59:40 <stickster> notting: Aha. http://lists.fedoraproject.org/pipermail/spins/2010-April/001081.html
17:59:56 <stickster> I was wondering about where you thought you had seen an on-hold status.
18:00:38 <mmcgrath> spoleeba: ping
18:00:40 <mmcgrath> spoleeba: err pong
18:00:54 <mizmo> spins are bad for my blood pressure
18:01:18 <spoleeba> mmcgrath, got time for a PM
18:01:25 <mmcgrath> of course, hit me!
18:01:33 * stickster sees that Peter did respond to testing call here: http://lists.fedoraproject.org/pipermail/spins/2010-May/001089.html
18:09:34 <Oxf13> stickster: that was the may 5th call, I made another on May 11th with actual produced isos to test
18:22:48 <mmcgrath> Oxf13: ping, can you check if that crazy .nfs file is still there?
18:23:00 * mmcgrath already forgot the path
18:23:08 <mmcgrath> oh I guess it's in the ticket I created though
18:23:09 * mmcgrath checks
18:24:24 <mmcgrath> Oxf13: yeah it's gone now.  you can try to remove whatever again.
18:25:39 <mmcgrath> Oxf13: if you have a moment, can you try to recreate that permission denied by group error?
18:26:58 <mmcgrath> nb: ping?
18:32:19 <warthog9> mdomsch: looking
18:32:46 <warthog9> mdomsch: broken ;-)
18:32:50 <warthog9> (looking at it now
18:39:28 <smooge> cache is taking up more room on proxy06. I would like to delete the .git in /srv/web/docs.fedoraproject.org before we run out
18:39:57 <skvidal> smooge: Id' +1 that
18:40:14 <smooge> can I get another?
18:40:31 <smooge> mmcgrath, ricky ?
18:40:37 <mmcgrath> smooge: it'd get put back in place on the next rsync run
18:40:43 <mmcgrath> and eat up bandwidth doing so :)
18:40:53 <smooge> where does it get rsync'd from
18:42:11 <skvidal> smooge: all you really need to do is add an exclude to the sync
18:43:04 <mmcgrath> smooge: bapp1
18:43:17 <mmcgrath> this is likely just a simple fix in the build script (which happens on bapp1)
18:43:23 <mmcgrath> nb would know for sure but I think he's been busy today.
18:43:38 <gui1ty> is trac the right place to report an issue with hardware detection tool in BFO?
18:43:40 <smooge> ok I will start digging
18:43:58 <warthog9> mdomsch: of course 2 weeks ago cacti broke and I didn't notice
18:44:08 <warthog9> gui1ty: no the HDT should be reported to syslinux
18:44:19 <smooge> I think we have another 2 hours before we run out on proxy6
18:44:26 <mmcgrath> gui1ty: I guess it depends on the issue, if it's HDT itself probably upstream to syslinux.  If it's a problem loading it or something that might come to us
18:44:27 <warthog9> gui1ty: or to BKO
18:44:47 <mmcgrath> smooge: huh?  how's that?
18:44:53 <mmcgrath> is the docs site growing over time?
18:44:59 <smooge> mmcgrath, no cache is
18:45:16 <mmcgrath> lets watch it during that time, cache should even out and get pruned.
18:45:19 <gui1ty> warthog9: directly or in bz (for syslinux)?
18:45:35 <mmcgrath> gui1ty: what is the problem btw?  :)
18:45:52 <gui1ty> the pci.ids file is missing or not in the right place
18:46:24 <mmcgrath> gui1ty: oh, that could very well be me, give me a bit to take a look.
18:46:29 <warthog9> gui1ty: ohhh if it's an issue of the pci.ids file not in the right place that's mmcgrath's problem ;-)
18:46:40 <gui1ty> warthog9: thanx ;-)
18:46:48 * warthog9 notes that mmcgrath clearly didn't copy that bit out of BKO
18:47:31 <mmcgrath> gui1ty: your IP start with 80.101 ?
18:47:57 <mmcgrath> warthog9: heh, our HDT is both useless and funny at the moment :)
18:48:48 <gui1ty> mmcgrath: how is it funny?
18:49:22 <mmcgrath> gui1ty: it's just all unknown.   I should have that fixed in about 2 minutes.
18:49:52 * gui1ty is realizing that he downloaded the BFO images a while ago
18:50:04 <warthog9> mmcgrath: in what way?
18:50:24 <mmcgrath> warthog9: it's just that since the pci.ids file is missing HDT can't actually identify anything and lists them all as unknown:unknown
18:50:32 <gui1ty> mmcgrath: since it's still there I'll wait for the fix
18:51:08 <gui1ty> mmcgrath: the numeric ids are shown on the bottom line, so it's not all useless ;-)
18:51:58 <mmcgrath> gui1ty: ok, try it now, you'll likely have to re-load the HDT
18:54:28 <gui1ty> mmcgrath: now it takes some more time to load and all the device descriptions are there
18:54:32 <gui1ty> mmcgrath: thanks
18:57:13 <warthog9> mdomsch: apparently *mysql* broke a table that cacti needed
18:57:21 <mdomsch> warthog9, whoops
18:57:27 <warthog9> mdomsch: yeah
18:59:55 <abadger1999> warthog9: All part of the sekrit oracle agenda ;-)
19:00:41 <warthog9> abadger1999: figures
19:00:53 <warthog9> abadger1999: I flip everything over to mysql and *THEN* things start breaking
19:01:30 <abadger1999> hehe
19:03:01 <warthog9> figures
19:04:28 <warthog9> looks like both my cacti instances had the same mysql problem, that's a little concerning
19:05:11 <mmcgrath> warthog9: FYI... there's a recent cacti vulnerability out.  The fix has been pushed, it'd be 0.8.7f
19:05:31 <warthog9> mmcgrath: joy
19:05:42 <warthog9> mmcgrath: have to see what it is / how bad
19:06:29 <mmcgrath> warthog9: https://bugzilla.redhat.com/595289
19:06:58 <warthog9> mmcgrath: ok that pushed out today
19:08:02 <warthog9> mdomsch: there we go new data starting to flow in
19:08:24 <mdomsch> warthog9, url?
19:08:33 <warthog9> http://cacti.kernel.org/graph_view.php?action=tree&tree_id=1
19:08:45 <warthog9> 'default tree' on the left it it doesn't take you right there
19:09:03 <warthog9> might take it another 15 minutes to have something actually interesting though
19:09:22 <warthog9> running iptraf on one of the boxes now
19:09:27 <jokajak> does fedora run cacti?
19:09:33 <abadger1999> geppetto: After reading some of your yum commits I think I've got your hashing gotcha documented: https://fedorahosted.org/releases/k/i/kitchen/docs/api-text.html#unicode-str-and-dict-keys
19:09:39 <mmcgrath> jokajak: we used to, not anymore though
19:09:51 <jokajak> running something else now?
19:10:00 <mmcgrath> jokajak: collectd - http://admin.fedoraproject.org/collectd/
19:10:04 <warthog9> mdomsch: if I'm reading this right I'm not moving a lot of US traffic (though getting hit relatively hard on the load)
19:11:17 <warthog9> mdomsch: I'm only moving ~300mbps on zeus1
19:12:01 <wzzrd> someone: is there a procedure (can't find one) in case the reporter of a review request hasn't responded to his review request for a long period of time?
19:12:09 <jokajak> it's so ugly :(
19:12:38 <warthog9> mdomsch: and if the aggregate bandwidth graph is right (looks like it) I'm moving less then 2gbps worldwide, which is barely above normal background
19:12:41 <mmcgrath> wzzrd: I think so, check in #fedora-devel, they'd know.
19:12:47 <mmcgrath> warthog9: :(
19:12:59 <wzzrd> mmcgrath: they just pointed me here :D
19:13:39 <mmcgrath> why did they do that?
19:13:47 <abadger1999> wzzrd: It's a #fedora-devel question but the answer is that you can ping and say you'll close it in X days (or take over the submission of the package in X days if you want it in the distro).
19:13:59 <warthog9> mdomsch: I suppose the question is are you actually redirecting a proportionate amount of traffic to me, or is this release a dud?
19:14:22 <mdomsch> should still be proportionate
19:14:25 <wzzrd> mmcgrath: abadger1999: ah! just got an answer in #fd; but thanks anyway
19:15:55 <mdomsch> warthog9, the other 3 are north of 500mbps
19:16:37 <warthog9> mdomsch: yeah, but that's a far cry from the 3.7gbps peak I've seen from a Fedora release
19:20:01 <cemc> F13 is bad luck, people are waiting for F14 :)
19:20:35 <warthog9> cemc: maybe ;-)
19:21:16 <warthog9> cemc: though reading some quick upgrade notes for F13, things like no longer being able to disable KMS for intel might be causing some pause
19:21:46 <geppetto> abadger1999: That's mostly it ... but the biggest problem we had was that ("a", "b") == (u"a", u"b"), with hashes too
19:22:08 <abadger1999> geppetto: Okay -- so you'd like a note of this applying to tuples as well?
19:22:50 <geppetto> abadger1999: Yeh, might as well put a small bit in there ... saying "note hashes of tuples are hashes of their members"
19:22:54 <geppetto> abadger1999: or something
19:23:57 <geppetto> abadger1999: Or maybe something like: "all dict keys should be the same type, is a good general rule but note that breaks down when the type is tuple"
19:30:36 <smooge> skvidal, /usr/bin/rsync -aSHP --exclude=.git bapp01::docs/ /srv/web/docs.fedoraproject.org/ should clean things up nicely
19:30:49 <skvidal> well
19:30:54 <skvidal> except it doesn't delete, does it?
19:31:06 <mmcgrath> skvidal: we can do that manually.
19:31:10 <skvidal> okie doke
19:31:11 <mmcgrath> smooge: you going to put that change in?  +1 from me
19:31:18 * skvidal is also +1
19:31:25 <smooge> mmcgrath, I am needing to check with something
19:32:00 <smooge> the tree public_html/en-US/Fedora_Contributor_Documentation/1/html/Publican_1.6_Users_Guide is on the proxy servers but not on bapp1 anymore
19:32:23 <smooge> I am trying to find out why
19:33:02 <mmcgrath> smooge: looks like the rsync call being used doesn't have a --delete ?
19:33:22 <smooge> no it doesn't
19:33:37 <smooge> so anything over the last X time will have gotten on there.
19:33:39 * warthog9 debates how much he's going to hate f13 by upgrading his primary home fileserver / server to f13 ;-)
19:33:44 <smooge> Doing the --delete cleans up soooo much
19:34:07 * jokajak votes rhel6
19:34:23 <smooge> I am going to commit the change in a short while.
19:34:33 <skvidal> f13/rhel6 - potayto/potahto :)
19:34:58 <warthog9> jokajak: rhel's fine until your 3 years down the road and you suddenly start realizing how old it is in comparison to things like Fedora
19:35:38 <jokajak> psh, what more could you need from F12 than what's there? ;-)
19:36:56 <smooge> warthog9, yeah but 3 years down the road you are supposed to throw away that computer and get something new
19:37:21 <warthog9> smooge: you clearly have more budget then I do for fancy new hardware ;-)
19:37:24 <skvidal> smooge: hell, 3yrs down the road you HAVE to throw it away
19:37:34 <skvidal> smooge: I've not had a laptop that actually survives 3yrs of use
19:37:37 <warthog9> smooge: just think I still have CRTs :-P
19:37:59 <warthog9> the only lcd's I have are on laptops or my TV
19:38:09 <jokajak> my home router is a laptop that is 4-5 years old
19:38:25 * abadger1999 even has a crt hooked up to a laptop...
19:38:44 * notting has a CRT monitor sitting on the floor he hasn't disposed of yet (and two CRT TVs)
19:38:49 <smooge> warthog9, well every 3 years I go to the used computer store to buy something newer. I am just about ready to get rid of my Dell Optiplex from 2001
19:39:01 <warthog9> smooge: lol
19:39:10 <warthog9> smooge: yeah my mythtv setup runs Fedora 7
19:39:17 <warthog9> built maybe a year before that came out
19:40:26 <smooge> I am hoping to ressurect my wife's Apple Powermac from 2002 to have something a bit faster.
19:40:56 <smooge> The kid has faster newer computers than me... his XO is probably faster than the Optiplex
19:43:17 <warthog9> smooge: hehe my phone now ranks in the faster machines in my apartment
19:49:13 <zodbot> Ticket notification - infrastructure: Ticket #2187 (enhancement closed): Stopgap fixes for Docs site on release day <https://fedorahosted.org/fedora-infrastructure/ticket/2187#comment:3>
19:54:16 <nirik> mmcgrath: you around?
19:59:11 <abadger1999> lmacken: Do you have a group for the critpath -setting script to belong to?
19:59:36 <lmacken> abadger1999: we currently don't have a user or group for setting that
19:59:52 <mmcgrath> nirik: yup, whats up?
19:59:59 <abadger1999> lmacken: Think of a couple names :-)
20:00:11 <nirik> mmcgrath: FES ticket roundup in the fesco meeting right now. ;)
20:00:56 <lmacken> abadger1999: masher?  critpathbot?
20:01:05 <mmcgrath> nirik: ah
20:01:44 <abadger1999> lmacken: k I'm adding user: critpathbot group: critpathadmin  to fas now.
20:02:06 <lmacken> abadger1999: sounds good to me, thanks.
20:02:14 <Oxf13> mmcgrath: wiped out 13.RC3, thanks.
20:02:21 <Oxf13> mmcgrath: ok, check out spins/releng/test/
20:02:53 <Oxf13> $ touch spins/releng/test/foo
20:02:54 <Oxf13> touch: cannot touch `spins/releng/test/foo': Permission denied
20:06:40 <mmcgrath> Oxf13: one sec, poking
20:08:19 <mmcgrath> Oxf13: I'm going to become you and run a couple of commands for this ticket.  just a groups and a touch, cool?
20:12:19 <Oxf13> sure
20:14:43 <mmcgrath> rut-roh
20:14:44 <mmcgrath> abadger1999: ping
20:14:52 <abadger1999> mmcgrath: pong
20:15:02 <mmcgrath> abadger1999: hey, got a problem on cvs1 -
20:15:14 <mmcgrath> ps auxwww | grep -c toshio
20:15:14 <abadger1999> mmcgrath: what's up?
20:15:30 <mmcgrath> looks like genacls is hanging
20:15:35 <mmcgrath> some of these are from yesterday
20:15:37 <abadger1999> holey mother of god.
20:16:12 <Oxf13> nice
20:16:18 <Oxf13> another need for a wrapper!
20:16:24 <abadger1999> I'm going to start by killing all of those and then we'll see if a new run works.
20:20:54 <abadger1999> mmcgrath: Okay, seems to be cvs that's hanging on commit.
20:21:15 <mmcgrath> ah, probably a lock file?
20:23:39 <abadger1999> mmcgrath: Yep, lock file -- all set now.
20:24:06 <abadger1999> skvidal: I removed a stale lock file for your user on cvs01 on CVSROOT/admin
20:24:12 <abadger1999> skvidal: Was created yesterday.
20:24:23 <skvidal> abadger1999: yesterday.
20:24:25 <skvidal> abadger1999: oh
20:24:29 <skvidal> I ctrl-c'd something
20:24:31 <skvidal> abadger1999: thanks
20:24:36 <abadger1999> k.  No problem.
20:36:44 <mmcgrath> .fasinfo critpathbot
20:36:45 <zodbot> mmcgrath: User: critpathbot, Name: Critpath setter in Pkgdb, email: toshio+critpath@fedoraproject.org, Creation: 2010-05-25, IRC Nick: None, Timezone: UTC, Locale: C, Extension: 59923, GPG key ID: None, Status: bot
20:36:49 <mmcgrath> abadger1999: ping
20:36:49 <zodbot> mmcgrath: Approved Groups: critpathadmin
20:36:55 <abadger1999> mmcgrath: pong
20:37:04 <abadger1999> mmcgrath: I created that jsut now... what's up?
20:37:23 <mmcgrath> oh, nevermind.  so you did.  I just got an alarming email about it but now am less alarmed :)
20:37:30 <mmcgrath> Warning: Someone attempted to reset the password for system account
20:37:49 <abadger1999> Ah -- right.  I forgot that we disabled changing that via the web.
20:39:18 <dgilmore> mmcgrath: i just saw that
21:03:17 <mmcgrath> K, looks like things are moving back down to normal - http://mmcgrath.fedorapeople.org/monitoring/proxy-requests.html
21:03:49 <stickster> mmcgrath: Did the 0-byte link.css entries help relieve some of the problems?
21:04:40 <mmcgrath> stickster: it did, the next biggest offender is this one -
21:04:44 <stickster> skvidal: Question -- is it possible to retain .git/refs/heads/* in the proxy copies of docs.fp.o, so we can identify where the current copy's drawn from?
21:04:49 <mmcgrath> File does not exist: /srv/web/docs.fedoraproject.org/public_html/images/title.png, referer: http://docs.fedoraproject.org/en-US/toc.html
21:05:22 <skvidal> stickster: I guess-  you'd need a kinda zany include/exclude to rsync but do-able
21:05:32 <skvidal> stickster: might be easier to put some version-file in the dir itself
21:05:35 <skvidal> outside of git
21:06:02 <stickster> skvidal: Ah, it's using rsync
21:06:15 <skvidal> yes
21:07:09 <stickster> so:   --include '.git/refs/heads/**' should be OK
21:07:21 <stickster> or something like that, I haven't seen the rsync command in question
21:07:24 <skvidal> nod
21:07:30 * skvidal points at smooge
21:07:37 * stickster tips hat :-)
21:08:51 <smooge> what huh
21:09:02 <smooge> did I do something
21:10:00 <smooge> stickster, the rsync command is currently: /usr/bin/rsync -aSHP --exclude=.git bapp01::docs/ /srv/web/docs.fedoraproject.org/
21:14:53 <smooge> stickster, also we are still seeing css problems
21:18:06 <mizmo> smooge, css issues?
21:18:11 <stickster> smooge: So the above should work:  --include .git/refs/heads/**
21:18:19 <stickster> (after the --exclude I imagine)
21:18:30 <stickster> smooge: As for CSS -- ugh, what's going on now?
21:18:31 * mizmo offers css assistance if its helpful
21:18:33 <mmcgrath> smooge: did you notice your rsync commands produce output?  see admin@fp.o
21:18:53 <mmcgrath> I think you need a -q
21:19:06 <stickster> mizmo: I think this issue -- if it's related to an earlier one -- isn't design, it's more missing files on the docs.fp.o server that are causing 404's and load
21:19:28 <stickster> smooge: Come by #fedora-docs and we can explore. Rudi is up now and can help while I'm away
21:19:37 <smooge> stickster, mmcgrath http://fpaste.org/aqhH/ First column is the missing .css and the second is the file that is asking for it
21:19:47 <stickster> smooge: thanks
21:20:49 <mizmo> ah okay
21:20:51 <stickster> smooge: Is it the other way around?
21:21:05 * stickster sees some have a dir first, a CSS file second
21:21:22 <abadger1999> geppetto: http://fpaste.org/XVxn/
21:22:38 <abadger1999> geppetto: Does that work?  If so I'll include it in lieu of my other example
21:22:59 <smooge> stickster, it is an awk from the line say "File not found: /srv/web/docs/spew/, referrer http://spew/
21:23:11 <geppetto> abadger1999: wow
21:23:18 <stickster> smooge: Ah, so it could be a broken link in old docs
21:23:37 <geppetto> abadger1999: that might actually work, and solve our intern strings problem in yum :)
21:24:02 <abadger1999> geppetto: hehe  if so, I'm glad to have helped :-)
21:25:47 <abadger1999> if I turn that into a dict replacement class, it could go into kitchen directly.... /me works on that.
21:38:38 <geppetto> abadger1999: It seems to work here, and saves about 2% over just ignoring tuples ... alas. the CPU time goes up by 50%
21:38:59 <abadger1999> geppetto: :-(  Yeah, iteration sucks the life out of things.
21:39:31 * geppetto nods
21:40:31 <abadger1999> geppetto: If you have enough data to benchmark.... something to try tweaking would be with and without the hash() check.
21:41:04 <abadger1999> Since you know you're only passing hashable values in the first place, that might be a win.
21:41:11 <geppetto> abadger1999: I dropped the hash check :)
21:41:15 <abadger1999> Okay :-)
21:43:02 <geppetto> abadger1999: I think one problem is that for a smallish "yum update" we are calling type_code 115,275 times :)
21:43:24 <abadger1999> That's a bottleneck... function calls are expensive.
21:45:02 * abadger1999 can't shake the feeling if he were just a little smarter he could do this without iteration.
21:47:28 <dgilmore> Oxf13: can you approve the email i just sent to devel-announce
21:48:41 <Oxf13> dgilmore: waiting for it to show up
21:49:06 <geppetto> abadger1999: I've tried a couple of hacks, I can get it down to 30% CPU for 3.5% RAM ... which is still terrible
21:49:30 <abadger1999> geppetto: <nod>  Were your hacks replacing recursion with iteration?
21:49:33 <geppetto> One of these days, probably soon, we'll probably have to make sure all pkgtups are in bytes anyway
21:49:42 <abadger1999> <nod>
21:49:45 <dgilmore> Oxf13: hrrm i got a message saying "Posts are allowed only from subscribed members."
21:49:54 <dgilmore> Oxf13: i wonder if it just got dropped
21:50:01 <Oxf13> must have.
21:50:02 <abadger1999> geppetto: yeah, for yum and rpm that seems like a necessity.
21:50:27 * geppetto nods ... soooo much pain though :)
21:51:49 * geppetto goes to find some food
21:53:12 <abadger1999> geppetto: Ohh... What about using repr?
21:53:23 <abadger1999> as in store[repr(key)] = value
22:01:45 <smooge> abadger1999, ricky mmcgrath skvidal I need an approval to push a redirect update
22:07:08 <abadger1999> smooge: So does moving the accessibility guide and deployment guide do anything?
22:07:41 <abadger1999> smooge: ie: the only change that changes what users see is the release-notes part?
22:07:50 <smooge> no it was just putting them in alphabetical order so I could keep track of them
22:07:58 <abadger1999> Gotcha.  +1
22:07:59 <smooge> I am expecting more
22:08:27 <abadger1999> I'll put that in email for the record keepers.
22:37:06 <zodbot> Ticket notification - infrastructure: Ticket #2189 (enhancement created): FAS plugin for zikula should prefill real name for users <https://fedorahosted.org/fedora-infrastructure/ticket/2189> || Ticket #2188 (enhancement created): Fedora Events "space" in Fedora Insight <https://fedorahosted.org/fedora-infrastructure/ticket/2188>
22:38:38 <pashdown> hello, we are a mirror that just discovered we are pointed to the wrong rsync host.  what is the proper rsync host?
22:41:23 <Jeff_S> pashdown: http://fedoraproject.org/wiki/Infrastructure/Mirroring/Tiering
22:56:19 <droidix> hello, someone in #fedora referred me here due to my question about the new boot.fedoraproject.org install method, it appears the conf file hasn't been updated with the 13 release info: http://alt.fedoraproject.org/pub/alt/bfo/pxelinux.cfg/fedora_install.conf
22:59:57 <mmcgrath> droidix: how'd you get directed there?  that's not the official bfo location - http://boot.fedoraproject.org/
23:00:29 * mmcgrath goes to update that site in case anyone else finds it there
23:00:50 <droidix> that was the file the installer downloaded when booting
23:01:38 <mmcgrath> droidix: when did you download it?
23:02:09 <droidix> I downloaded the USB image at 10:51 CDT this morning
23:02:19 <mmcgrath> interesting, looks like I've got some updating to do.
23:02:24 <mmcgrath> droidix: sorry for the trouble, it should be fine now.
23:02:37 <mmcgrath> droidix: thanks for the heds up
23:02:40 <mmcgrath> err heads even
23:02:40 <droidix> ok, no problem, thanks!  I think this is a really sweet install method
23:02:44 * mmcgrath bbiab
23:06:47 <spoleeba> mmcgrath, hmm how does anyone know about bfo?  Its still sort of tucked away right now isnt it? Or did it make the release notes?
23:07:09 <droidix> yes, it was on page 2 of the release notes :)
23:07:20 <droidix> that's how I saw it, pretty prominent position
23:07:32 <droidix> http://docs.fedoraproject.org/en-US/Fedora/13/html/Release_Notes/sect-Release_Notes-Installation_Notes.html
23:07:36 <spoleeba> droidix, not exposed on getfedora however
23:07:49 <droidix> Two clicks off the fedora home page
23:07:58 <spoleeba> droidix, im more impressed that you read the release notes
23:08:33 <Jeff_S> spoleeba: it was in the announce email also
23:09:15 <droidix> lol, thanks, I really do think this method could revolutionize installs
23:35:42 <CodeBlock> mmcgrath: what's up
23:37:46 <mmcgrath> CodeBlock: hey, not much here.  building night stands.
23:38:12 <mmcgrath> CodeBlock: how's your pt server working out?
23:39:09 <CodeBlock> mmcgrath: dgilmore gave me access to pt3 so I was able to test groups in the script, seems to work fine
23:40:01 <mmcgrath> good
23:40:09 <nb> mmcgrath, smooge skvidal did you get .git excluded in the rsync cron? if not i can do it
23:40:12 <mmcgrath> #info so far so good.  I'm going to call this a successful release.
23:40:15 <mmcgrath> #endmeeting