15:01:35 <tflink> #startmeeting fedora-qadevel
15:01:35 <zodbot> Meeting started Mon Dec 12 15:01:35 2016 UTC.  The chair is tflink. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:01:35 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
15:01:35 <zodbot> The meeting name has been set to 'fedora-qadevel'
15:01:42 <tflink> #topic roll call
15:02:33 * jskladan is here
15:03:36 * mkrizek is here
15:03:56 <tflink> ok, let's get this party started
15:04:11 <tflink> sorry for the lack of announcement, I still don't know why I can't get at my email
15:04:20 <jskladan> tflink: noprob
15:04:25 <tflink> #topic Dist-Git Tasks
15:05:06 <tflink> I just wanted to touch base quickly on the status of this to make sure I wasn't forgetting anything
15:05:29 <tflink> what else is yet to be done other than re-deployment and finalizing the storage proposal?
15:06:10 <jskladan> I did some reviews for ralph, seems like he's nailing his bits, I also merged https://phab.qadevel.cloud.fedoraproject.org/D1031 today, which hopefully adds a a bunch to the `discover` action in trigger
15:07:01 <jskladan> apart of the function split (and making some of the intended fallbacks work), it can traverse the git repo (starting with a specified directory) recursively
15:07:11 <jskladan> so we can possible schedule multiple tasks from one repo
15:07:27 <jskladan> default behaviour is the same as it was (=non recursive)
15:08:35 <jskladan> not sure whether it's still needed (I'm yet to understand what the "final" decision on the storage will be), but at least it's testable and split to manageable pieces now
15:08:54 <tflink> do you think it's ready for deployment to dev?
15:09:56 <jskladan> sure, I tested it locally as much as I could... I hope not many errors were introduced during the lovely process of rebasing by multiple commits
15:10:24 <jskladan> but the thing is - I'm not sure we actually do have any "discover" stuff to test it on
15:10:34 <jskladan> (in real)
15:10:50 <tflink> we can talk to the modularity folks, I think
15:10:54 <tflink> they may have something
15:11:11 <tflink> or we might be able to do it for one of our packages
15:11:17 <jskladan> I'm fairly certain Ralph is doing what he can to facilitate those, but yeah
15:12:56 <tflink> mkrizek: are you aware of anything missing for dist-git tasks?
15:13:09 <mkrizek> not that I can think of
15:13:43 <jskladan> Also I had a brief conversation with Ralph, and it turns out we probably should finally go and make it possible to pass more (arbitrary) arguments to formulae from the command line - I spent half the last week trying to make argparse do it for us, but I'll need to write the logic of parsing "unknown" arguments on my own. Not big of a deal, but I hoped for something easier... (related to dist git, since Ralph was asking about it, probably t
15:13:54 <jskladan> other than that, I have nothing on the topic
15:14:40 <tflink> wasn't that one of the things we talked about when I was in brno one of the last times? that we're probably going to have to change that before too long?
15:15:09 <jskladan> yup, we were
15:15:09 <tflink> jskladan: your second to last comment was cut off at "probably t"
15:15:35 <jskladan> ...(related to dist git, since Ralph was asking about it, probably to work out parsing the fedmessage data)
15:16:18 <tflink> #info new trigger code supporting recursive searching through git repos is ready for testing
15:16:50 <tflink> #info the known missing bits to support dist-git checks in production are documentation and redeployment with the new bits
15:17:30 <tflink> jskladan: any guess on how urgent the input stuff is?
15:18:07 <jskladan> not much, I think, threebean made it work differently, in the end
15:18:12 <jskladan> but it would be nice to have
15:18:36 <jskladan> I suspect, that changes in trigger/buildmaster will be required to be able to pass arguments downstream from trigger to runtask
15:18:52 <jskladan> but that's the next step after we're able to parse those at all
15:18:53 <tflink> just wondering if it can wait until we're all in the same room in Jan/Feb
15:19:21 <jskladan> I guess it can. I see about three options, and I'll write down a ticket/email about it
15:19:59 <tflink> ok. I'll talk to ralph to make sure that we're not missing some urgency
15:20:07 <jskladan> cool
15:21:13 <tflink> anything else on this?
15:21:21 <jskladan> nope
15:22:01 <tflink> cool. moving on
15:22:12 <tflink> #topic Rebuilding Taskotron Staging and Production
15:22:32 <tflink> IIRC, we're good to do stg this week, right?
15:22:58 <mkrizek> was the decision made on how we're going to approach the resultsdb migration yet?
15:23:12 <jskladan> mkrizek: I don't think so, honestly
15:23:37 <tflink> oh yeah, I figured there was something that I was forgetting :)
15:25:27 <tflink> jskladan: any idea how much work is left for the archive table/database you were talking about at flock?
15:25:48 * jskladan shrugs
15:25:58 <tflink> I assume that's what you meant when you were talking about archives
15:26:19 <jskladan> yeah, but honestly, I'd just spin another resultsdb instance
15:26:29 <jskladan> and offloaded data there once in a while
15:27:25 <jskladan> so, like, the only work would be to write the script that would take care of moving data from one place to another
15:27:38 * tflink wonders if that's going to cause problems/fun elsewhere
15:27:50 * jskladan has no idea, really
15:28:11 <jskladan> but in my innocent, magic world, we are "removing" old, unused data anyway
15:28:28 <jskladan> and if anybody wants to search those, just connect to this other resultsdb instance
15:29:33 <tflink> not having email makes it hard to find older conversations :)
15:30:21 <jskladan> tflink: sorry - the idea was, that (as I said) we'd spin up another resultsdb instance (.../resutlsdb_archives/..) for example
15:30:41 <tflink> yeah, just trying to find the response to my question about export/import
15:31:16 <tflink> I'm not sure that dumping the data is going to be a good idea long term
15:31:29 <tflink> dumping/deleting
15:31:55 <jskladan> why not? I think vacuum is being performed quite often
15:32:08 <jskladan> and I'd only really offload the data once every three months or so
15:32:13 <jskladan> not on the fly
15:32:29 <tflink> I'm worried about folks who want to do historical queries against the data
15:32:58 <jskladan> tflink: I see no problem there, honestly
15:33:11 <tflink> people would have to remember to change the hostname, no?
15:33:11 <jskladan> the data will be available in the "archives" db
15:33:37 * jskladan shrugs yes, but I don't really think that is a problem
15:33:47 <jskladan> (if your use-case is doing some digging in the historical data)
15:33:52 <tflink> you have more faith in people RTFD than I do :)
15:33:59 <jskladan> I'd rather accomodate our primary use-case first
15:34:22 <tflink> yeah, that makes sense. just tryign to figure out how much work it would be to not split the data up
15:35:42 <jskladan> well... we sure could do it (like add the concept of "current" and "archived" to resutlsdb, so you can have a link to "next" page start pointing to the archives
15:36:13 <tflink> that would help
15:36:16 <jskladan> and "seamlessly" continue querying the archives (well, you'd know, it would get slower)
15:38:49 <tflink> is there a sane, short-ish term solution that would let us kick this can down the road a bit?
15:39:09 <jskladan> short time, I'd just offload the data we have
15:39:28 <jskladan> and I can have the "master/archive" thing done in a week tops
15:39:56 <jskladan> I don't really see any other actual solution for the occasional data-dig anyway
15:40:27 <tflink> how hard would it be to reconstruct the data if a better solution was found that would enable a single db?
15:41:08 * jskladan shrugs once again - never did that, but I suppose that as easy as pg_dump?
15:41:33 <tflink> how much data are you talking about dumping?
15:41:41 <tflink> i really need to use different words
15:41:51 <tflink> how much data are you talking about truncating from the main db?
15:42:01 <tflink> dumping has multiple meanings in this case :)
15:43:22 <jskladan> I'd like to keep about three months of data - or whatever is the time span we have for cleaning up artifacts an so on
15:43:38 <jskladan> or half a year, I don't care, really
15:43:45 <tflink> could we start with 6 months?
15:44:34 <jskladan> sure, why not. I'm not sure how much data that actually is, but it's less than what we have now :D
15:44:42 <jskladan> so the migration won't take that long
15:45:06 <tflink> ok, can you try with a stg dump?
15:45:49 <jskladan> sure
15:46:24 <jskladan> anyway - we can (once migrated) put the "archived" data back to resultsdb
15:46:50 <jskladan> as I said in the private conversation - the problem we have with "too much data" is mostly solvable by moar memory
15:46:53 <tflink> I'd like to start there and spend more time figuring out what we want to do about archiving
15:47:27 <jskladan> resultsdb is othewise pretty fast, and only slows down in the edge cases
15:47:28 * tflink has TODOs to add more memory to the db server and start talking about switching over to a bare metal box
15:48:41 <tflink> jskladan: how long do you think it will take before enough questions are answered to be ready to redeploy stg?
15:48:44 <jskladan> cool, so to sum it up:
15:48:44 <jskladan> 1) dump db before migration
15:48:45 <jskladan> 2) keep last 6 months of data
15:48:45 <jskladan> 3) migrate prod
15:48:45 <jskladan> 4) put "old" data to "archives"
15:48:46 <jskladan> 5) migrate "archives"
15:48:47 <jskladan> 6) ???
15:48:49 <jskladan> 7) profit
15:49:06 <jskladan> tflink: I'm not sure about the context now
15:49:19 <jskladan> resultsdb on stg is already migrated
15:49:28 <tflink> it is?
15:49:42 <jskladan> yup, ran that migration  a week or two back
15:49:51 <jskladan> of was it dev?
15:49:56 <tflink> i think that was dev
15:50:01 <jskladan> ah, sorry
15:50:37 <tflink> does 4) mean keep the old data in a separate instance?
15:50:43 <jskladan> yup
15:51:43 <tflink> I'm still not crazy about that idea but I understand why.
15:52:03 <jskladan> well, what else can we do with it, if we don't want to migrate all the data live?
15:52:05 <tflink> so long as we can undo the removal if that becomes needed, let's go with that
15:52:30 <tflink> undo the removal as in add in offline migrated data at some later date
15:52:58 <jskladan> honestly, we can do it  once the (db schema) migration is done on "archives"
15:53:03 <tflink> ok
15:53:11 <tflink> then let's go with that plan
15:53:23 <jskladan> I just want to make sure we migrate the "live" data as fast as possible with minimal down-time
15:53:36 <tflink> when do we think that we'd be ready to start re-deploying stg?
15:54:03 <jskladan> I can take care of the db dump/migration this week
15:54:21 <tflink> this week as in tomorrow or wednesday?
15:54:24 <jskladan> (or as soon as somebody yells "go")
15:54:36 <tflink> mkrizek: can you think of a reason not to yell "go"?
15:54:51 <jskladan> would it help? (I mean doing it Tue/Wed)
15:55:32 <mkrizek> nope, it's stg, I don't see a reason not to do it asap
15:55:35 <tflink> I'd like to get it done soon. F23 goes EOL in about a week, amongst other reasons
15:55:47 <jskladan> OK, I'll do it ASAP then
15:55:52 <tflink> sounds good
15:56:07 <tflink> then we can plan to do prod later this week or early next week
15:56:36 * mkrizek notes he's on PTO starting next week
15:56:40 <tflink> #info current plan is to truncate resultsdb database to have a sanely short migration to new schema
15:56:59 <tflink> bah, it's later in the year than I'm thinking it is
15:57:09 <tflink> let's see if we can get it done this week if at all possible
15:57:28 <mkrizek> also, do we want to rebuild prod before all of us going afk for holidays?
15:57:59 * jskladan notes that Wed next week is his last workday this week
15:58:15 <tflink> depends, I need to go chat with the person who wanted to know if it was ready
15:58:15 <jskladan> ... this year
15:58:24 <tflink> jskladan: that makes more sense :)
15:58:46 <jskladan> I'm from the land, where tomorrow is yesterday!
15:59:14 <tflink> #info stg redeployment will happen this week, prod deployment may wait until after RH shutdown when more folks are around
15:59:44 <tflink> jskladan: i have many questions for our visitor from the future
15:59:54 <tflink> lottery numbers, sports scores etc. :)
15:59:57 <jskladan> Trump wins the electino!
15:59:59 <tflink> anyway, moving on quickly
16:00:11 <jskladan> (oops, that's last year...)
16:00:11 <tflink> #topic qadevel and phabricator
16:00:40 <tflink> so I spent most of last week trying to get phabricator working with a new auth method that won't be turned off imminently
16:01:21 <tflink> while most of that seems to have been settled, I'm still left with the question of whether we want to go on maintaining out own system instead of just using what the rest of fedora seems to be using (pagure+taiga)
16:01:21 <jskladan> did you make it, or is it still resisting?
16:01:45 <tflink> i think that the last bit got figured out on Friday - I need to verify, though. didn't get to it over the weekend
16:02:31 <tflink> so it's not really an urgent question - just wanted to start asking it for the future
16:02:42 <jskladan> as I said earlier - pagure (especially) + taiga is way worse solution for me, but if it means Phab won't take insane amount of your time, I'd consider it.
16:02:51 <tflink> as a heads up - there will be some downtime on phab, hopefully this week
16:03:30 <tflink> the downtime will involve some big changes - new phabricator version, better CI (pending changes to doit.py files) and most disruptively - a new hostname
16:03:38 <jskladan> good to hear that you made it work, though
16:03:42 <jskladan> yaay
16:03:46 <tflink> qadevel.cloud.fedoraproject.org will become qa.fedoraproject.org
16:03:52 <jskladan> new stuff!
16:04:22 <jskladan> could we have a redirect for a while, or is it too much work for such a little profit?
16:04:24 * tflink already has the changes ready for libtaskotron but needs to make sure that it still works with current develop/
16:04:28 <tflink> that should work fine
16:04:44 <jskladan> cool
16:04:45 <tflink> since the only thing that changes is the hostname - everything else should stay the same
16:05:27 <tflink> so much for a short meeting
16:05:43 <tflink> i think that's all for this, so time for a very short ...
16:05:46 <tflink> #topic open floor
16:05:56 <tflink> anything that someone wants to bring up now?
16:06:07 * jskladan has nothing of importance. I'd love to have some diffs looked at, but it can wait
16:06:33 <tflink> jskladan: I assume that's it's mostly me who's behind on those?
16:07:42 <tflink> anyhow, we can continue conversations in #fedora-qa or on the lsit
16:07:52 <tflink> thanks for coming everyone
16:07:57 * tflink will send out minutes shortly
16:08:00 <tflink> #endmeeting