17:00:29 #startmeeting F39 Beta Go/No-Go meeting 17:00:29 Meeting started Thu Sep 14 17:00:29 2023 UTC. 17:00:29 This meeting is logged and archived in a public location. 17:00:29 The chair is adamw. Information about MeetBot at https://fedoraproject.org/wiki/Zodbot#Meeting_Functions. 17:00:29 Useful Commands: #action #agreed #halp #info #idea #link #topic. 17:00:29 The meeting name has been set to 'f39_beta_go/no-go_meeting' 17:00:43 morning everyone 17:00:47 #meetingname F39-Beta-Go_No_Go-meeting 17:00:47 The meeting name has been set to 'f39-beta-go_no_go-meeting' 17:00:55 #topic Roll Call 17:01:30 ahoy to the oy, everyone 17:01:47 .hello ngompa 17:01:48 Son_Goku: ngompa 'Neal Gompa' 17:02:00 .hello2 17:02:02 lruzicka: lruzicka 'Lukáš Růžička' 17:02:06 .hello thebeanogamer 17:02:07 thebeanogamer: thebeanogamer 'Daniel Milnes' 17:02:57 .hello2 17:02:58 coremodule: coremodule 'Geoffrey Marr' 17:03:06 .hello geraldosimiao 17:03:06 geraldosimiao: geraldosimiao 'Geraldo S. Simião Kutz' 17:03:15 how's everyone doodling 17:03:16 .hello2 17:03:17 dcantrell: dcantrell 'David Cantrell' 17:03:29 adamw, doodle doodle doo 17:04:08 .hello2 17:04:09 tflink: tflink 'Tim Flink' 17:04:42 .hello salimma 17:04:43 michel-slm: salimma 'Michel Lind' 17:04:48 .hi 17:04:49 sgallagh: sgallagh 'Stephen Gallagher' 17:05:03 .hi 17:05:05 decathorpe: decathorpe 'Fabio Valentini' 17:05:06 * decathorpe is lurking 17:05:13 so *checks notes* we're supposed to have representatives of development, releng, and QA 17:05:34 #info nirik will represent release engineering 17:05:39 #info adamw will represent QA 17:05:46 who wants to be a development representative?! 17:05:51 decathorpe and I can cover dev/FESCo 17:05:52 (you have to be a packager) 17:05:59 I'm here for dev 17:06:14 Ok, him too ;-) 17:06:18 yeah looks like there's at least 4 FESCo members here 17:06:18 #info sgallagh, decathorpe and/or dcantrell will represent development (buy your tickets now for the last-man-standing battle) 17:06:27 :D 17:06:36 I don't think I'll join that battle, but I'm here too :) 17:06:39 Hi all! 17:06:41 we prefer to be called a steering committee fighting for control of the wheel 17:06:43 hi ublue 17:06:46 heh 17:06:50 impending boilerplate alert 17:07:15 We really need to get someone named Jesus to run for FESCo, just for the jokes 17:07:24 #topic Purpose of this meeting 17:07:24 #info Purpose of this meeting is to check whether or not F39 Beta is ready for shipment, according to the release criteria. 17:07:24 #info This is determined in a few ways: 17:07:24 #info 1. No remaining blocker bugs 17:07:24 #info 2. Release candidate compose is available 17:07:26 #info 3. Test matrices are fully completed 17:07:28 #info 4. Fedora CoreOS and Fedora IoT are prepared 17:07:32 sgallagh: oof 17:07:40 (note i am amending bcotton's script a bit on the fly, prepare for fun!) 17:07:59 #topic Current status - blockers 17:08:00 #link https://qa.fedoraproject.org/blockerbugs/milestone/39/beta/buglist 17:08:27 #topic (2232838) Some NVME controllers fail to initialize with kernel 6.4.11 or later (nvme0: controller is down; will reset: CSTS:0xffffffff, PCI_STATUS=0xffff) 17:08:27 #link https://bugzilla.redhat.com/show_bug.cgi?id=2232838 17:08:27 #link https://pagure.io/fedora-qa/blocker-review/issue/1291 17:08:27 #info Accepted Blocker, kernel, ON_QA 17:08:45 this one is assumed to be addressed in the Beta candidate. I don't know if anyone with affected hardware was able to test it and verify, but the update is in there 17:09:48 oh, well...oh dear. 17:09:49 https://bodhi.fedoraproject.org/updates/FEDORA-2023-ac464517f7#comment-3195672 17:09:52 that seems concerning 17:09:53 did you see the last comment on bodhi? tha past guy said it didn't worked on their dell latitude 17:10:01 yeah 17:10:20 doesn't seem to be quite the same errors... 17:10:54 i think possibly that reporter is seeing something different from the others 17:11:04 https://bugzilla.redhat.com/show_bug.cgi?id=2232838#c23 17:11:04 ok, indeed 17:11:15 now I opened the link with the screenchot 17:11:23 screenshot 17:11:36 they say they "believe it's the same bug", but the messages aren't the same, and a workaround that didn't work for the other reporters *did* work for them 17:12:02 so it may be something else 17:13:18 it would sure be nice to have more data here... 17:13:25 oh, wait, i was assuming this was the same person as indygibb on the bug report, but they seem to be different people. 17:15:52 so it's not the same issue, what should we do? 17:16:01 panic? 17:16:13 run for the hills 17:16:17 We weren't already panicking? 17:16:19 kernel panic, even? 17:16:34 Did I start early? 17:16:38 don't suppose anyone here happens to have an xps 15 9560? :D 17:16:45 alas no 17:16:48 hum, i wonder if my older xps 13 reproduces this. let's see! 17:17:04 I have an XPS 13 as well, what do I need to do? ;) 17:17:07 I have an 9360, is that close enough? 17:17:18 The newest XPS I have around is ... L502 17:17:19 i don't know, but we can try 17:17:26 try an older image and see if it fails, i guess 17:17:33 then try the rc and see if it works 17:18:06 Ack, if someone knows which older image to try, I'll find the laptop and charge it 17:18:12 And verify it's indeed 9360 17:18:12 the update went stable on 2023-09-12 17:18:25 so i think the 09-11 nightlies should have affected kernels 17:18:34 https://kojipkgs.fedoraproject.org/compose/branched/Fedora-39-20230911.n.0/ 17:18:43 ack 17:18:45 oh, that one doomed 17:18:49 Ouch 17:18:52 https://kojipkgs.fedoraproject.org/compose/branched/Fedora-39-20230910.n.0/ then 17:19:34 sigh, i was hoping this would be a *quick* meeting 17:19:41 sorry i didn't realize this hadn't been confirmed 17:20:39 Can we run this in the background and check the other tickets in parallel? 17:20:43 sure, good idea 17:20:55 I also would probably just for now operate with the assumption of lack of confirmation 17:20:56 #info folks are testing to see if they can reproduce this and verify the fix at present, we will circle back 17:21:11 #topic (2113005) Live image made with BOOTX64.EFI from latest shim-x64-15.6-2 fails to boot on some boards 17:21:11 #link https://bugzilla.redhat.com/show_bug.cgi?id=2113005 17:21:11 #link https://pagure.io/fedora-qa/blocker-review/issue/1154 17:21:11 #info Accepted Blocker, shim, NEW 17:21:17 ugh 17:21:22 this is what? the third release now? 17:21:24 so, i'm gonna propose we waive this like we have for several releases 17:21:31 something like that? 17:21:36 +1 to waive 17:21:39 it seems like we're actually getting close to the finish line now, though! 17:21:52 I'll believe it when I see it 17:21:55 +1 to waive 17:21:57 see the bug for details, but basically, the nx stuff still isn't merged upstream so we still can't get a new shim signed by microsoft 17:22:08 but it's looking hopeful for it to be done by f40 and *possibly* by f39 final 17:22:08 +1 waive 17:22:11 +1 waive 17:22:39 #info this is still stuck in the same pattern it's been in for several releases: we have a fix, but we cannot get a new shim build signed by microsoft until the kernel supports nx, and those patches are still being reviewed 17:23:08 we will likely set the compat flag and get one signed soon anyway, and call the fact that kernel doesn't work a kernel bug. 17:23:12 I am curious of what everyone else is doing about it 17:23:24 are they just going "yolo" and shipping the patches? 17:23:26 everyone uses shim, don't they? 17:23:37 I mean the kernel parts 17:23:37 Son_Goku: the same thing we are; there's been some movement on kernel (finally) but it's still just not quite there 17:23:44 this already reached RHEL dind't it? 17:23:52 geraldosimiao: yeah, but that's not our problem. :P 17:23:59 ok 17:25:09 proposed #agreed 2113005 - waived to Fedora 39 Final - this blocker is waived to Fedora 39 Final as a "Difficult to fix blocker bugs": we cannot practically fix it at the moment while the nx patches are still under discussion 17:25:17 s/bugs/bug/ 17:26:08 ack 17:26:23 ack 17:26:32 ack, +1, and so on 17:26:36 ack 17:26:49 ack 17:27:06 #agreed 2113005 - waived to Fedora 39 Final - this blocker is waived to Fedora 39 Final as a "Difficult to fix blocker bug": we cannot practically fix it at the moment while the nx patches are still under discussion 17:28:01 #topic Current status - test matrices 17:28:01 #link https://fedoraproject.org/wiki/Category:Fedora_39_Test_Results 17:28:33 coverage appears to be pretty solid except for aarch64 cloud (I didn't get to that yet) and aarch64 desktop - https://fedoraproject.org/wiki/Test_Results:Fedora_39_Beta_1.1_Desktop#Release-blocking_desktops:_aarch64 17:29:15 we have a few automated results for aarch64 desktop, and there's little reason to think the other beta blocking stuff would be broken on aarch64 if the automated tests passed and the x86_64 ones did, I guess 17:29:21 my aarch64 test box still doesn't work :( 17:29:27 I have a Jetson Nano but I couldn't ever get Workstation running on it reliably :( so I doubt that I can help with that 17:30:14 I would have tested on my Mac, but I don't yet have an F39 build :( 17:30:58 one of the crappier things about needing to rely on a large downstream patchset while upstream kind of doesn't review stuff :/ 17:31:02 ftr FCOS tests are passing with f39 content on all architectures (with a few exceptions, mainly SELinux) 17:31:05 adamw, Ive done a little ARM workstation testing (boot), nothing major, but I've mostly been focused on IoT stuff 17:33:05 sigh, can't reproduce the nvme bug with the 20230910 nightly. it has a rather older kernel than the one someone listed in the bug as affected...let's see if i can find a rawhide nightly... 17:34:06 dustymabe: thanks, i added a new fcos/iot checkin agenda item later so we'll circle back to that 17:34:17 * michel-slm aborting his download 17:34:21 I'm upgrading my aarch64 WS VM right now 17:34:25 Will report back shortly 17:34:32 sgallagh: thanks, let us know if you hit anything 17:36:36 i'll do a quick boot of an aarch64 ec2 instance just to make sure that works 17:36:51 FWIW FCOS tests boot AWS instances on aarch64 17:37:02 but that's not cloud 17:38:16 I've pulled my XPS 13 7390 out of a drawer to test the NVME bug, WCPGW jumping from 35 to rawhide 17:38:43 thebeanogamer: wait until adamw find an affected compose to test I guess? 17:39:20 @michel-slm considering the state of this machine, it'll probably take me that long to even get it to boot 17:39:21 well, in theory the latest f39 updates-testing is affected 17:39:58 uh, that might be a problem for me testing, this laptop has another distro installed 17:40:03 was hoping I could test from a live image 17:40:12 ok, there, cloud aarch64 coverage looks better 17:40:17 I think you'd fail to see the disk from a live environment 17:40:27 because the nvme controller fails to initialize 17:40:29 Son_Goku: no, we already pushed the fix stable 17:40:34 oh 17:40:35 welp 17:40:54 yeah my repro would be - fail to see disk on affected build, can see disk on fixed build 17:41:15 oh, hum, i didn't think to see if the disk was visible, duh. i just assumed the boot would fail, but of course not for an install image, sigh 17:41:17 booting again! 17:41:48 hmm, nope, the disk's there. 17:41:54 Oh I'd missed the annoying coil whine of this laptop... 17:42:07 so i guess my xps 13 isn't affected 17:42:23 as long as it's the kernel that is reported affected, I can test after this meeting. unless the laptop is the same type 17:42:37 the image has a 6.5rc7 kernel and the bug report does say that upstream reported rc7 affected 17:42:54 the 0910 image right? I'll try 17:43:06 that's the one i tried, yeah 17:45:14 #info test matrix coverage is good except for aarch64 desktop testing, sgallagh is going to try that out quickly and report back any issues he sees 17:45:32 Upgrade nearly complete 17:46:11 #topic Current status - RC 17:46:19 i find it weird that this topic comes here. i feel like it should be the first. :P 17:46:48 #info we have a candidate compose, https://dl.fedoraproject.org/pub/alt/stage/39_Beta-1.1/ , nobody has said there's anything wildly wrong with it AFAIK 17:47:24 it looks like we got lucky on the ostree failure lottery (or did someone fix that when I wasn't looking?), the only missing images are aarch64 i3 and lxqt 17:48:14 * nirik nods 17:48:47 as for the rpm images, SOAS have problems, but it isn't a blocker desktop anyway 17:50:20 My aarch64 Workstation seems to be doing just fine after the upgrade 17:50:39 (I like the new background, by the way!) 17:50:59 I'm using the new background now on f38, I like it that much 17:51:05 yeah, the new wallpaper is really very good 17:51:48 yeah I love it 17:52:07 sgallagh: for form's sake, can you check any of the tests at https://fedoraproject.org/wiki/Test_Results:Fedora_39_Beta_1.1_Desktop#Release-blocking_desktops:_aarch64 ? the easy ones, anyway 17:53:05 well, uh, booting the 0910 image via UEFI boot lands me on the grub fallback prompt 17:53:15 I'll try disabling UEFI to get over this but it's odd 17:53:29 michel-slm: huh, that sounds like something different. maybe that good ol' shim bug? 17:53:34 anyhoo 17:53:43 #topic Fedora CoreOS and IoT check-in 17:54:00 dustymabe: i believe you said all's good for CoreOS? 17:54:11 adamw Any in particular you're worried about? 17:54:24 Basic operations seem fine (control panel, firefox, file browser) 17:54:30 for the record, when will the Next stream kick over to f39? 17:55:04 sgallagh: if you could do window manager, audio basic (if you have the hardware), panel basic and automount it'd make the table look nicer... 17:55:12 and i guess printing if you have a printer :P 17:55:27 desktop_login and core_applications are kind of a long pain 17:55:34 adamw: There are a few minor issues. No Blockers. 17:55:51 The plan is to release `next` with f39 on the same day as Fedora 39 beta 17:55:54 adamw: maybe. it fails to boot in legacy mode too. but... maybe there's another issue too, it's now running diagnostics when I unplug the usb drive 17:56:01 michel-slm: sigh, fun! 17:56:02 * michel-slm hopes his drive is not dead 17:56:28 #info Fedora CoreOS is ready to go according to dustymabe, the plan is to update the "next" stream to Fedora 39 on the same day as the Beta release 17:56:33 woot 17:56:49 coremodule: pbrobinson: can either/both of you comment on IoT? 17:56:57 i believe it looks in decent shape 17:56:59 IoT is looking strong 17:57:08 Thats the 09122023 compose 17:57:14 #info IoT matrix - https://fedoraproject.org/wiki/Test_Results:Fedora-IoT_39_RC_20230912.0_General 17:57:39 #info we will nominate an IoT compose for release after the packages from the 1.1 candidate are pushed stable, but don't anticipate any problems 17:58:15 adamw all good. I'll update the matrix accordingly 17:58:21 sgallagh: awesome, thanks 17:59:11 question: this means https://pagure.io/fedora-qa/blocker-review/issue/1307 won't be addressed for the beta right? 17:59:34 if we decide to sign off the candidate at this meeting, that's correct 17:59:55 i don't believe that bug breaks any beta criteria, though it's a shame we can't fix it. both the notifications criterion and the keyboard layout criterion are Final 18:00:10 ok -- i'm a little worried seeing the notification at every boot will prompt the user to eventually follow its text and enable ibus as a virtual input even though they don't need it 18:00:12 we'll have to mark it as a commonbug 18:00:33 and reverting that will have to be a manual action from the user, if they ever find out they didn't need to follow the notification instructions 18:00:44 (aka not something we can do as a package update) 18:01:27 isn't the comps change for this ready to be merged? so there should be a zero-day update that fixes this? or am I thinking of some other bug .... 18:01:36 .hello2 18:01:37 frantisekz: frantisekz 'František Zatloukal' 18:01:40 yes, but only if the user doesn't follow the instructions 18:01:48 it's annoying 18:01:59 if you install the iso and immediatly follow the instructions you get a permanent change that's probably unwanted 18:02:14 I'm _okay_ with this for beta, but I'm not _happy_ about it 18:03:26 yeah, it's a shame, but stuff happens. we'll commonbugs it 18:03:34 ok 18:03:36 will the 0-day at least make the notification go away, or are we stuck with it? 18:03:44 yes 18:03:46 it will make it go away - yes 18:03:50 I would like us to have this become beta criteria in F40 18:03:52 if we can 18:04:04 so, keep it for beta and merge the fix for final, is that? ok 18:04:10 yeah 18:04:15 we can't make *everything* beta, realistically. a beta's still a beta. it's meant to be for testing, or you get to keep the bits. :P 18:04:20 fine, np 18:04:28 geraldosimiao: the update will be pushed as soon as the freeze is lifted 18:04:30 ideally most people upgrading to the beta won't hit it 18:04:40 and the comps change merged, I guess, but that will only affect subsequent composes 18:04:43 since we'll lift the freeze after the GO decision 18:04:47 right. 18:04:58 okay, let's circle back to our blocker 18:05:17 #topic (2232838) Some NVME controllers fail to initialize with kernel 6.4.11 or later (nvme0: controller is down; will reset: CSTS:0xffffffff, PCI_STATUS=0xffff) 18:05:18 yeah, that's why we have that warning at anaconda about being a beta 18:05:39 * thebeanogamer is still patiently waiting for dnf-system-upgrade 18:05:39 so, what do we want to do with this? 18:06:05 i'm leaning towards ss shipit. logically speaking, the commit that caused this has been identified, folks upstream confirmed that reverting it works, we reverted it 18:06:17 I have an XPS 13 9370 and none of the 6.4 kernels (both older and newer than the update supposed to fix the issue) have been causing boot problems for me, not sure if that helps ... 18:06:25 it sucks that we don't have confirmation but i don't really see grounds to slip for this 18:06:30 yeah I think we say this is solved, might be other issues, but we can't slip on them? 18:06:59 I agree. there's not enough information here 18:07:11 I'm good with that 18:07:20 this is beta, so it not like it suppose to be perfect 18:07:25 if we can get more data, we can make a new bug for final blocker 18:07:52 and I'm not just saying that because my list of updates are piling up and I want the freeze to be lifted :P 18:07:57 any rc to test with the fix 18:08:07 the 1.1 RC 18:08:11 yeah, the 1.1 contains the fix 18:08:18 In the absence of evidence that the problem remains, I think we are justified in assuming it's fixed. 18:08:26 +1 18:08:29 +1 18:08:36 +1 18:08:52 it's also unclear how widespread this could be... only some specific hardware. 18:09:10 although, hmm. 18:09:16 * adamw looks a bit harder at the kernel commit 18:09:41 Son_Goku: I have 116 updates in f39 testing-pending-stable ... is it worse than that? ;) 18:09:57 not quite that bad, only because I stopped :P 18:10:03 :D 18:10:06 decathorpe: #RustWorldProblems 18:10:17 true 18:10:23 sorry for the delay 18:10:31 i just want to convince myself that we actually *did* revert this properly 18:10:36 adamw: clearly we need Chuck Norris to intimidate the kernel commit into confessing 18:11:20 * michel-slm prepping to reimage his broken laptop hoping it's still salvageable 18:11:54 * Southern_Gentlem points michel-slm at the f38 updated lives 18:12:30 okay, yes, looks like we did. 18:13:32 proposed #agreed 2232838 - treat as VERIFIED - it's a shame that the fix for this has not been formally verified by someone affected, but we have checked that the commit identified upstream as the culprit has indeed been reverted, so we think it's reasonable to consider this addressed 18:13:46 ack 18:13:50 ack 18:13:51 ack 18:13:53 ack 18:13:56 ack 18:14:03 ack 18:14:06 ack 18:14:07 ack 18:15:27 #agreed 2232838 - treat as VERIFIED - it's a shame that the fix for this has not been formally verified by someone affected, but we have checked that the commit identified upstream as the culprit has indeed been reverted, so we think it's reasonable to consider this addressed 18:16:18 #topic Go/No-Go decision 18:16:18 I will poll each team. Please reply “go” or “no-go” 18:16:24 go 18:16:29 :D 18:16:30 wait your turn! :D 18:16:32 FESCo? 18:16:34 GO 18:16:36 now, fight 18:16:36 Go 18:16:38 go 18:16:42 awww 18:16:46 Releng? 18:16:47 go 18:16:54 releng should be all go. 18:17:16 QA, per our policy, is go (matrix coverage is acceptable and there are no unaddressed blockers) 18:17:27 if anyone has any objections, speak now... 18:17:29 does /me get a vote 18:17:33 coreos: GO 18:18:02 dustymabe: not formally, but hey, if it makes you happy :D 18:18:11 no objections, Go is great :) 18:18:18 #agreed Fedora Linux 39 Beta is GO 18:18:23 dustymabe: Like Puerto Rico in the US Congress: you get a vote any time it doesn't affect the outcome 18:18:26 🥳🥳 18:18:31 :partyparrot: 18:18:32 gret 18:18:34 dusty is puerto rico, confirmed 18:18:34 🎉 18:18:37 great 18:18:41 🎉 18:18:45 great 18:18:56 excellent. 18:19:03 satisfactory. 18:19:06 #info Fedora Linux 39 Beta will release on the target date #1 (2023-09-19) 18:19:19 \o/ 18:19:27 :-) 18:19:39 Excellent!! 18:19:49 #action adamw or amoloney to announce decision 18:19:55 heh, it's always funny to see people don't understand we start counting target dates at zero :) 18:20:02 #topic Open floor 18:20:06 Anything else we need to discuss before closing? 18:20:15 decathorpe: the first one is called the "early target date" 18:20:17 start a zero, always 18:20:27 this naming is specifically designed to foil Phoronix 18:20:32 what is a zero if not an early one 18:20:34 ;) 18:20:43 @adamw I'm stealing that description 18:20:47 :D 18:21:21 floor G, floor 1, floor 2, floor 4 18:21:37 * michel-slm will test the RC on his laptop later today, since... the Debian installer boots but the Fedora 0910 didn't which is worrying 18:21:44 nirik: no, no, no. no 4s. 18:22:02 * thebeanogamer is still waiting for the F39 update on their XPS 13, and will run the testsuite for Bodhi once it's done 18:22:30 thebeanogamer: let us know if you see anything worrying. sorry we couldn't wait for the test to be done 18:22:37 @nirik speak fast: floor 4, 4 floor, floor 4... 18:22:42 It's almost certainly fine, don't worry about me 18:22:50 floor 4, fourth floor, floor 4, fourth floor... 18:23:17 yeah :D 18:23:37 alright, thanks a lot for coming out, everyone 18:23:45 apologies if I messed anything up, it's been a while since I ran one of these :D 18:23:48 you're welcome and thank you 18:23:53 thanks adamw 18:23:56 adamw++ 18:23:58 thanks adamw 18:24:02 thanks 18:24:07 yes adam++ 18:24:07 geraldosimiao: Karma for adam changed to 1 (for the release cycle f39): https://badges.fedoraproject.org/tags/cookie/any 18:24:08 adamw += 1 18:24:10 thanks! 18:24:20 adamw++ 18:24:20 adamw++ 18:24:23 adamw++ 18:24:25 adamwill++ 18:24:25 decathorpe: Karma for adamwill changed to 1 (for the release cycle f39): https://badges.fedoraproject.org/tags/cookie/any 18:24:29 we may have to reinvent readiness meetings for this cycle to make sure everything's lined up, i'll see about having one of those tomorrow or something 18:24:32 adamw++ 18:24:38 adamwill++ 18:24:38 geraldosimiao: Karma for adamwill changed to 2 (for the release cycle f39): https://badges.fedoraproject.org/tags/cookie/any 18:24:40 adamwill++ 18:24:41 zlopez: Karma for adamwill changed to 3 (for the release cycle f39): https://badges.fedoraproject.org/tags/cookie/any 18:24:47 adamwill++ 18:24:47 thebeanogamer: Karma for adamwill changed to 4 (for the release cycle f39): https://badges.fedoraproject.org/tags/cookie/any 18:24:49 mmmmmm all that lovely karma 18:24:52 adamw++ 18:25:04 (people, you need to get the FAS username right) 18:25:05 it's the karma firehose 18:25:14 LOL 18:25:14 adamwill++ 18:25:14 dcantrell: Karma for adamwill changed to 5 (for the release cycle f39): https://badges.fedoraproject.org/tags/cookie/any 18:25:48 now i can go out and do evil stuff, right? that's how karma works? 18:25:50 I give karma to some Adam I don't know ho is... 18:25:52 LOL 18:26:09 karma == cookies 18:26:37 #endmeeting