14:31:04 <bstinson> #startmeeting apps.ci.centos.org/Antora syncup
14:31:04 <zodbot> Meeting started Mon Jul  9 14:31:04 2018 UTC.
14:31:04 <zodbot> This meeting is logged and archived in a public location.
14:31:04 <zodbot> The chair is bstinson. Information about MeetBot at http://wiki.debian.org/MeetBot.
14:31:04 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
14:31:04 <zodbot> The meeting name has been set to 'apps.ci.centos.org/antora_syncup'
14:31:07 <rkratky> and pbokoc
14:31:18 <pbokoc> I was just about to ping everyone :)
14:31:22 <bstinson> #chair asamalik bexelbie rkratky pbokoc
14:31:22 <zodbot> Current chairs: asamalik bexelbie bstinson pbokoc rkratky
14:31:38 <bstinson> we have a meeting!
14:32:00 <asamalik> hey there!
14:32:02 <asamalik> .hello2
14:32:03 <zodbot> asamalik: asamalik 'Adam Samalik' <asamalik@redhat.com>
14:33:00 * bstinson needs to close some tabs
14:33:15 <bstinson> #link https://pagure.io/aquedoc/
14:33:21 <bstinson> #link https://pagure.io/pagure-2-openshift
14:33:37 <bstinson> #topic Agenda
14:33:41 * asamalik thinks bstinson should just open a new browser window and forget about those old tabs :P
14:34:01 * bstinson declares tab bankruptcy
14:34:21 <bexelbie> .hello bex
14:34:22 <zodbot> bexelbie: bex 'Brian (bex) Exelbierd' <bexelbie@redhat.com>
14:34:43 <rkratky> .hello
14:34:43 <zodbot> rkratky: (hello <an alias, 1 argument>) -- Alias for "hellomynameis $1".
14:34:55 <rkratky> .hello rkratky
14:34:59 <zodbot> rkratky: rkratky 'None' <rkratky@redhat.com>
14:35:07 <bstinson> So to start off with, I can probably give an overview of the steps in the docs pipeline, and point at places in the aquedoc repo where we define them
14:35:30 <rkratky> sounds good
14:35:40 <bstinson> we can cover questions as we go, and follow up by opening pagure tickets in there for what we want to get done
14:35:56 <asamalik> +1
14:36:55 <bstinson> anything else that we should be sure to talk about today?
14:37:19 <asamalik> I was trying to build an s2i image for Fedora, but I got errors while installing antora using npm... so, I don't have anything we could try :(
14:37:30 <asamalik> Fedora -> Antora
14:37:31 <asamalik> :)
14:38:07 <bstinson> that's probably one of the first issues we can work on
14:38:47 <asamalik> bstinson: yes
14:39:16 <bstinson> #topic Docs Pipeline Overview
14:40:25 <bstinson> So there are a few moving parts of the docs pipeline. Remember that this pipeline is focused on building the sites hosted here: https://pagure.io/fedora-docs/docs-fp-o
14:41:07 <bstinson> The first part is a trigger system, built by cverna. This system takes events from repositories on pagure.io and translates that information into triggers that openshift/jenkins can consume
14:41:30 * cverna is around
14:41:50 <bstinson> https://pagure.io/pagure-2-openshift <- this is a small application that runs alongside the jenkins pods that drives the triggers
14:41:56 <bstinson> hi cverna!
14:42:05 <cverna> o/
14:42:28 * asamalik waves at cverna
14:42:49 <cverna> I think most of pagure-2-openshift is now in pagure ci upstream
14:43:33 <bstinson> cverna: we should probably look at doing new triggering documentation and retire the existing service
14:43:44 <bstinson> if we have everything we need upstream now
14:44:02 <cverna> yes I think we are missing one thing but I can't remember what :s
14:44:45 <cverna> I need to have a look at it again
14:45:29 <bstinson> https://pagure.io/aquedoc/issue/1
14:46:25 <cverna> oh we are not able yet to trigger different jobs in pagure-ci
14:46:25 <bstinson> so assuming we have the appropriate triggers, Pull Request testing goes through this pipeline:
14:46:50 <bstinson> https://pagure.io/aquedoc/blob/master/f/docs-build-pr.yaml#_29
14:46:53 <cverna> and we currently use one jenkins job for PR and one for master
14:47:02 <bstinson> ah right
14:47:41 <bstinson> line 29,30,31 here show the important parameters that we pass along to the s2i builder
14:47:59 <bstinson> REPO, BRANCH, and PR
14:48:54 <bstinson> pagure has refs by PR number for us, and so the s2i builder "knows" how to check that out and build the site:
14:48:57 <bstinson> https://pagure.io/aquedoc/blob/master/f/s2i/bin/assemble#_15
14:50:13 <bstinson> After the build happens, there is a container specifically tagged with this PR hosted in the registry at apps.ci.centos.org, and Jenkins creates a new URL, starts the container, and points the URL at the running page
14:51:02 <bstinson> You get URLs in this form:
14:51:03 <bstinson> http://fedora-docs-quick-docs-pr36-fedora-docs.apps.ci.centos.org/
14:51:16 <asamalik> cool
14:51:57 <bstinson> the pipeline posts back to the PR like this:
14:51:59 <bstinson> https://pagure.io/fedora-docs/quick-docs/pull-request/36#comment-47209
14:52:12 <bstinson> and includes that URL plus the job number
14:52:47 <bstinson> the job number isn't useful unless you want to help administrate the service (if you'd like to, i'd love some help here)
14:53:01 <bstinson> but you can basically tie the job number back to a run of the pipeline in the jenkins interface
14:53:24 <bstinson> rkratky: you had a request for a bit more information to be posted here, what else can we include?
14:55:18 <asamalik> bstinson: that looks really good
14:55:26 <rkratky> bstinson, down tyhe road, we will want to incorporate more than just building into the pipeline: various tests etc. It would be great to have a URL of the build job in Jenkins for checking the log, and/or some info directly in the comment.
14:55:42 <asamalik> so it looks like it only builds the repo alone, as opposed to the whole docs site, right?
14:56:52 <bstinson> rkratky: ack. can you write something up in a ticket on the aquedoc repo for that? I know we have it in email and captured here
14:57:38 <rkratky> bstinson, ok
14:57:54 <cverna> there is a long standing RFE on pagure to link the jenkins logs to the PR
14:58:17 <bstinson> cverna: link?
14:58:20 <rkratky> cverna, do you have a link to that?
14:58:22 <rkratky> :)
14:58:25 <bstinson> heh
14:58:34 <cverna> looking for it
14:59:02 <cverna> https://pagure.io/pagure/issue/1783
14:59:36 <bstinson> asamalik: to answer your question, i think it does build the repo itself
15:00:03 <bstinson> cverna: let me see if i/we can put some time into that RFE
15:00:46 <cverna> yes, I might have some "free time" to look at it too :P
15:00:52 <bstinson> asamalik: but, that's all defined in the repo itself
15:00:58 <rkratky> bstinson, those would be very nice features to have
15:01:21 <bstinson> asamalik: https://pagure.io/aquedoc/blob/master/f/s2i/bin/assemble#_33
15:01:28 <asamalik> bstinson: that looks like something we'll need to figure out... with Antora, we can build the whole site, but we haven't tried to build just a single repo... I'm not sure it's possible out of the box
15:01:50 <bstinson> the builder image is fairly "dumb" about things. it just provides the environment and checks out the Repo/PR
15:02:08 <cverna> i think we wanted to build the site
15:02:29 <cverna> but at the time this was not our priority
15:02:39 <bstinson> i think that makes the most sense (building the whole site)
15:02:49 <cverna> yes +1
15:04:02 <asamalik> cool
15:04:03 <bstinson> asamalik, rkratky: that brings up a question about organization, these will still live in separate repos under pagure.io/fedora-docs, and be aggregated together to build the final site, correct?
15:04:29 <asamalik> we would just need to modify a json file that defines all the repos the site is build from: https://pagure.io/fork/bex/fedora-docs/docs-fp-o/blob/antora_playbook/f/site.yml
15:04:50 <asamalik> so it would pick up a different repo with the PR
15:05:08 <asamalik> sorry, yaml file
15:05:18 * asamalik is mixing words today for some reason
15:06:24 <bstinson> hmm interesting. so the builder needs to mangle that json file to pick up the incoming PR?
15:07:36 <bstinson> s/json/yaml/ :P
15:09:04 <asamalik> yeah
15:09:14 <asamalik> the site.yml basically defines the whole site
15:09:41 <asamalik> it can also use local paths
15:11:12 <bstinson> so we could either put the ref for the incoming PR in that yaml file and have antora pull it itself, or include a path where we know that the s2i builder will put a checkout of the incoming PR?
15:11:18 <asamalik> or we could just make it point to the repo/branch with the changes
15:11:38 <asamalik> yes :)
15:11:59 <bstinson> simple enough :)
15:12:02 <asamalik> I think the first would be the easiest, at we have to mangle the yaml file no matter what
15:12:11 <asamalik> that would be the only thing to do :)
15:13:17 <bstinson> sounds like a plan
15:13:53 <bstinson> bexelbie: is the production publish process in scope for this? or is that staying the same for a while?
15:16:00 <bstinson> while that question is pending, we can wrap up the overview
15:16:15 <rkratky> bstinson, asamalik, one concern: as the plan is to host all platform (RHEL) docs in Pagure, the site-building could potentially become quite a task -- introduciong a significant overhead; can we think about ways to build only parts of the docs selectively?
15:16:52 <asamalik> rkratky: if we remove some of the repos from the site.yml, it would only build the ones we leave there
15:17:02 <asamalik> so I think we could achieve that, yes
15:17:09 <rkratky> asamalik, can we do this dynamically?
15:17:39 <pbokoc> yeah, even if RHEL docs are hosted in pagure they'll still be built by Pantheon, right? No reason for Antora to worry about them, we can just omit them from configs and not worry about them
15:17:48 <asamalik> rkratky: that's what I meant.. I can immagine we could have a script that would only leave the repo we're building + some others that are necessary
15:18:53 <rkratky> pbokoc, building in Pantheon takes ages (first mirrior to gitylab, thern wait for Pantheon, then pray it works, ...) -- we need a way to build RHEL PRs and WIP docs from Pagure, too.
15:19:24 <pbokoc> rrright
15:19:45 <bstinson> keep in mind, we can do separate instances of the pipeline for separate concerns
15:21:11 <rkratky> bstinson, what would be the best way for us/me to start playing with that? Ideally, I would set up a bunch of testing pipelines for out PoC repo to get well familiar with the process.
15:21:26 <rkratky> ...our PoC^
15:21:49 <bexelbie> sorry - was (and still am) in a call
15:21:51 <bstinson> rkratky: let's get you set up with some test pipelines (that you'll have admin access to) in apps.ci.centos.org
15:22:03 <bstinson> i'll email you with account information etc.
15:22:11 <rkratky> bstinson, cool, thanks
15:22:28 <bexelbie> bstinson, it'd be nice to switch us over to antora fully in the near future (flock-ish timeframe) if that is what you asked
15:22:28 <bstinson> do we know the turnaround time for building the RHEL docs? and/or is this something we can figure out from your test pipelines?
15:22:52 <rkratky> bstinson, you mean how long it takes to build?
15:22:53 <bexelbie> but don't throw away asciibinder yet as budget.fp.o is not converted
15:24:09 <bstinson> bexelbie: so ideally, docs.fp.o would be served directly from artifacts built using this pipeline at/around Flock?
15:26:03 <bstinson> rkratky: yeah, would that information be helpful?
15:26:13 <asamalik> bstinson: I think that's a stretch goal.. we definitely want the site to be built using Antora by Flock, even manually if necessary
15:26:39 <bstinson> asamalik: got it
15:26:45 <asamalik> if we could do the PRs that would be nice
15:26:58 <rkratky> bstinson, it really depends on the doc -- some are text and image-heavyu, some very short... unless it's way out of range, I don;t think we need that info
15:28:05 <bstinson> ok, we're coming up on our hour. and i want to leave a little bit of time to record some action items and tie up discussion
15:29:00 <bstinson> basically the last bit of the pipeline is noticing a PR Merged event from pagure, and cleaning up the containers and resources we stood up at the temporary URLs. the "trial" sites stay up until the PR is closed
15:29:08 <bstinson> #topic Next Actions/Other Discussion
15:29:30 <bstinson> #action bstinson to get rkratky access to a project in openshift for PoC pipelines
15:29:49 <bstinson> asamalik: do you need help looking at the s2i image?
15:31:10 <asamalik> bstinson: not at the moment, I'm mostly struggling to install Antora on Fedora right now... it's not packaged, so I'm installing it using npm and it throws up errors at me... I looked at the image and I believe I should be able to make it work
15:31:21 <asamalik> but I might have a few questions
15:31:28 <asamalik> never done s2i before
15:31:34 <bstinson> is there a regular docs meeting that we can piggyback on to report progress?
15:31:50 <bstinson> asamalik: i'm happy to be another set of eyes. feel free to ping me any time
15:32:20 <asamalik> bstinson: thanks a lot!
15:32:38 <bstinson> #action asamalik to continue poking the antora s2i image
15:32:46 <pbokoc> bstinson, yeah, every Wednesday at 15:30-16:00 UTC: https://apps.fedoraproject.org/calendar/docs/
15:32:49 <rkratky> bstinson, the reg. docs mtg takes place on Wednesdays; I'll send you the info
15:32:58 <rkratky> nice, pbokoc :)
15:33:13 <bstinson> #action bstinson and cverna to syncup about pagure-2-openshift
15:33:41 <asamalik> sounds good
15:33:43 <bstinson> pbokoc, rkratky: awesome, i'll show up there :)
15:33:46 <bexelbie> bstinson, yes
15:33:51 <bexelbie> asamalik, ^^
15:34:30 <cverna> Once this is all in place I was planning to write an article for the Magazine :)
15:34:33 <rkratky> bstinson, trouble is, I usually have a conflict on Wednesdays -- but I'll try to be there, too
15:34:44 <bstinson> pbokoc: i also owe you an openshift project for bootstrapping some of the CentOS docs
15:35:09 <bstinson> we're a few minutes over, is there anything else we should capture for today?
15:35:59 <asamalik> looks good from my side
15:36:08 <rkratky> bstinson, great intro, thanks
15:36:31 <pbokoc> +1, thanks for the overview bstinson :)
15:36:58 <bstinson> alright we'll keep this open for 1 more minute just in case. otherwise we'll catch up on these actions via email, and check in at the next docs meeting
15:37:13 <asamalik> bstinson: cool, thanks! :)
15:38:04 <bstinson> thanks all!
15:38:07 <bstinson> #endmeeting