15:30:18 <jbair> #startmeeting Fedora CI SIG
15:30:18 <zodbot> Meeting started Wed Nov 18 15:30:18 2020 UTC.
15:30:18 <zodbot> This meeting is logged and archived in a public location.
15:30:18 <zodbot> The chair is jbair. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:30:18 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
15:30:18 <zodbot> The meeting name has been set to 'fedora_ci_sig'
15:30:19 <jbair> .hello jimbair
15:30:19 <zodbot> jbair: jimbair 'Jim Bair' <jbair@redhat.com>
15:31:15 <msrb> .hello msrb
15:31:16 <zodbot> msrb: msrb 'Michal Srb' <msrb@redhat.com>
15:34:14 <jbair> #topic Fedora CI Pipeline Status
15:34:25 <jbair> So we just talked about it before starting the meeting, but as I understand it
15:34:51 <jbair> Our -1 instance of fedora CI Jenkins ran out of IP resources in AWS so we had to get the limit raised to allow jobs to continue
15:35:33 <jbair> in the meantime, we moved back to our old cluster, which dropped TMT support until we migrate back
15:35:49 <jbair> also, there's a DNS issue that siddharthvipul is working through currently as time permits to fix the errors like we are seeing here:
15:35:57 <jbair> #link https://osci-jenkins-2.ci.fedoraproject.org/job/fedora-dist-git-bot/1915/
15:36:14 <jbair> msrb bgoncalv siddharthvipul let me know if I'm forgetting anything :)
15:36:48 <bgoncalv> I think eln pipeline is not running as we don't have fallback for it
15:36:57 <msrb> eln is running
15:37:09 <bgoncalv> ahh
15:37:13 <siddharthvipul> jbair, hey, I just wanted to confirm how long till Fedora CI has totally moved to new cluster.. we need to remove that :)
15:37:20 <siddharthvipul> is it a blocker on us?
15:37:29 <siddharthvipul> us == CI Infra side
15:37:29 <msrb> the new cluster is up and running, it just need to be migrated
15:37:43 <jbair> msrb: it was mostly migrated then we ran into the AWS limits I think?
15:37:45 <jbair> So we had to move back
15:38:25 <msrb> jbair, afaik there will be one more migration, data will be lost, and we will need to redeploy pipelines again
15:38:35 <msrb> not sure when that is coming though
15:38:43 <lrossett> hey all, I removed a job definition from my .zuul file but zuul returned an error that the same job definition was missing.. does it need to be manually removed from zuul as well?
15:38:46 <msrb> but that's what Andrei mentioned earlier today
15:38:57 <jbair> I think astepano_ is working on that when time allows as the limits were fixed
15:39:13 <jbair> anyone familiar with zuul? I can't say that I am :)
15:39:30 <astepano_> :-)
15:39:31 <msrb> in theory, we could enable the dist-git pipeline on the new cluster again
15:40:59 <bgoncalv> for zuul fbo might be able to help
15:41:18 <msrb> astepano_, any estimate when the migration could happen?
15:41:30 <astepano_> jenkins-1 ?
15:41:36 <msrb> yep
15:42:10 <msrb> astepano_, or better question: jenkins-1 will be up and once new instance is ready, dns will be updated to point to the new instance?
15:42:33 <msrb> so no bigger outage is expected?
15:42:51 <astepano_> I wish I knew, but most tricky part is letscert + dynamic certs... I didn't touch it today. I would say: 2 days: tommorrow/Friday.
15:43:38 <astepano_> yep, dns I will update
15:43:47 <msrb> ah, cool
15:44:10 <msrb> in that case, we can enable dist-git pipeline on jenkins-1 again
15:44:37 <astepano_> yep, that is the plan. Let's set: time Friday, second part of the day.
15:44:53 <msrb> astepano_, thanks ;)
15:45:52 <jbair> astepano_: thanks!
15:46:09 <astepano_> :-)
15:49:33 <jbair> #tpoic open discussion
15:49:37 <jbair> #topic open discussion
15:49:47 <jbair> any other topics before we close it out this week? anything else going on in fedora cI?
15:55:00 <jbair> #endmeeting