16:15:14 <threebean> #startmeeting Messaging SIG
16:15:14 <zodbot> Meeting started Tue Mar 20 16:15:14 2012 UTC.  The chair is threebean. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:15:14 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
16:15:22 * nirik is lurking around.
16:15:52 * threebean scrapes through the meetbot docs
16:16:30 <abadger1999> threebean: Do a #chair abadger1999 nirik
16:16:41 <abadger1999> We can help with the meetbot commands for today
16:16:45 <threebean> #chair abadger1999 nirik
16:16:45 <zodbot> Current chairs: abadger1999 nirik threebean
16:17:16 <abadger1999> #topic Role Call
16:17:38 <abadger1999> who's here to talk about what's going on with messaging in fedora infrastructure?
16:17:55 * nirik is here.
16:17:58 * threebean is here
16:17:59 * abadger1999 here
16:18:17 <nirik> you might also #meetingname messaging-sig
16:18:26 <threebean> #meetingname messaging-sig
16:18:26 <zodbot> The meeting name has been set to 'messaging-sig'
16:18:34 * threebean takes notes
16:18:44 <abadger1999> threebean: Go ahead and use #topic SUBJECT to introduce the next topic
16:19:36 <threebean> #topic Report and questions on fedmsg
16:20:28 <threebean> So I've been working on a document and some code over at http://github.com/ralphbean/fedmsg collaborating only really with lmacken on it.
16:20:58 <threebean> Lots of decisions have gone into it, but I haven't gotten (or sought) much feedback on it.  I'm hoping to rectify that with these meetings.
16:21:33 <threebean> It seems noone has objected to using 0mq over amqp/qpid so far.
16:22:10 * nirik digs around in docs. ;)
16:22:54 <threebean> The sending portion of the code works but without authn/authz yet.  I just recently built some shell utilities at skvidal's suggestion.  They work as well.
16:23:45 <nirik> is there a 'connect to all known senders and spit out the messages' type thing yet?
16:24:12 <threebean> nirik: no, but I can nail that pretty quick.  Call it 'fedmsg-tail' ?
16:24:32 <nirik> sure.
16:25:30 <nirik> and that could feed into syslog if we wanted to log things I guess?
16:25:44 <nirik> although... is there any limit to how big messages are?
16:25:50 <threebean> I'm finishing stuff up in the centralize-config branch that will let us start putting the sending hooks in more services in staging.
16:26:06 <threebean> nirik: Not that I'm aware.  I bet there is one somewhere.
16:26:09 * threebean checks
16:26:55 * threebean squints
16:27:04 <threebean> nirik: what sort of message are you thinking of sending? :)
16:27:26 <nirik> so the next steps would be a few services enabled in stg ?
16:27:48 <nirik> dunno, just wondering if we have something logging them all, and someone sends a gigantic message that it overloads the logs somehow.
16:28:01 <nirik> or one thats 100 a second
16:28:02 <threebean> gotcha.
16:28:51 <threebean> 0mq sockets have a high-water-mark option that causes the sending sockets to block if throughput reaches a certain point.
16:29:39 <nirik> cool. I imagine we could fix senders since we control them all.
16:29:42 <threebean> that's not in by default yet, but it makes sense to enable it.  run tests, see what happens.
16:30:16 <lmacken> so are you 100% all-in with 0mq?
16:30:30 <lmacken> why not amqp, for example?
16:30:53 <lmacken> are you worried about 0mq upstream? which has recently been forked by some original members
16:31:14 * lmacken hesitant to go all-in with the new shiny tech of the month :)
16:31:56 <threebean> 1) yes, 2) higher throughput (~100 times), no central broker to shine, the API is more fun 3) yes, I'm worried about the fork.  those weren't just some original members, they were the core devs.
16:32:14 <lmacken> wrt 2) the 600m/s that you saw with amqp is more than enough for our needs
16:32:48 <lmacken> ooph, so is it worth giving that fork (i forgot the name of it already) a Good Hard Look?
16:32:50 <nirik> is it likely we would follow the forked version? or they haven't diverged much yet?
16:33:56 <threebean> http://lists.zeromq.org/pipermail/zeromq-dev/2012-March/016429.html <-- announcement of the fork.
16:34:25 <threebean> It's super new, I haven't grokked the repercussions yet.
16:34:42 <lmacken> ah, forked over trademark stuff, not tech?
16:34:49 <threebean> yes.
16:35:43 <threebean> just to note, zeromq *is* the shiny tech of the month, but it's been on the scene for a few years.
16:36:11 <lmacken> yeah, I know :) I just want to make sure we're not abandoning amqp just because it's slower than 0mq
16:36:33 <lmacken> because there are many added benefits, like security, QoS, etc
16:37:01 <threebean> right.  I think we can duplicate those with 0mq without much issue.
16:37:07 <lmacken> cool
16:37:27 <lmacken> are you thinking clear text at first?
16:37:32 <threebean> I had bad experiences with narcissus and qpid falling over ad-nausea.
16:37:32 <lmacken> then when we need security, stunnel?
16:38:02 <lmacken> yeah, I remember those. many of those issues were self-imposed though ;)
16:38:03 <threebean> lmacken: there's stunnel externally, but the python zmq module has a zmq.ssh submodule I haven't wrapped my head around yet.
16:38:37 <lmacken> cool, for now clear-text is fine for our initial use case, I think
16:38:48 * nirik nods.
16:39:39 <nirik> so at least at first we are only going to have senders/readers in our infrastructure? is there thought about allowing outside people to have senders we listen to or connecting to our stuff?
16:39:52 <nirik> or is that too far out to say how and if we want to ever do it?
16:41:02 <threebean> nirik: only inside FI for now.  We'd need a good proven method of signing messages before we allow anything from outside in.  Even then it makes me nervous.
16:41:16 <lmacken> threebean: has thta moksha bug been fixed that allows anyone to send anything anywhere? ;)
16:41:32 <threebean> sending messages from inside FI out through a gateway or repeater in on the roadmap, but quite far off.
16:42:02 <threebean> lmacken: and no.  we've got a plan to fix it with websockets+zeromq.  there's no guess at a solution for fixing it with kamaloka+amqp yet.  ;)
16:42:13 <lmacken> ah, ok.
16:42:22 <nirik> yeah, I don't know that we would want to consume outside stuff as much as provide a way for people to read.
16:42:32 * threebean agrees
16:43:26 <threebean> Mind if I change the topic?
16:43:31 <abadger1999> Do it.
16:43:48 <threebean> #topic Sending from FI services
16:44:24 <threebean> So Tagger and Bodhi2.0 both have sending hooks in place.  There are examples in the docs to copy and use elsewhere.
16:44:47 <threebean> I'm seeking suggestions on which services to tackle next.
16:45:33 <lmacken> koji seems like it would have the most consumers of it's messages
16:46:20 <abadger1999> threebean: fas is in good shape so it would go out the door quick.  But there's likely more concern about security there.
16:46:27 <nirik> yeah... pkgdb, bodhi would also be good.
16:46:30 <abadger1999> threebean: Maybe a git hook would be a better place?
16:47:00 <threebean> (I think that's all of them) :P
16:47:15 <abadger1999> pkgdb needs work to get the code base in shape for a release -- I was hoping that one of those things would be moving appdb into packages... but doesn't look like anyone is working on that :-(
16:47:41 <dgilmore> how is it planned that people can consume the notices?
16:47:55 * dgilmore would like to have kojis notifications overhauled
16:48:28 <threebean> dgilmore: each node will have a fedmsg.ini config file with a list of sending addresses.
16:49:05 <dgilmore> threebean: not what i mean
16:49:08 <threebean> dgilmore: there'll be a python api by way of a Moksha consumer to pick up the messages on whatever topics your app in interested in.
16:49:23 <dgilmore> threebean: id like to just disable kojis internal notifications
16:49:39 <dgilmore> threebean: and have something replace the sending of build notcies
16:49:49 <dgilmore> so that all package owners get notifications
16:50:02 <dgilmore> threebean: and people could easily subscibe to the ones they want
16:50:22 <dgilmore> threebean: nothing at all to do with apps
16:50:58 <threebean> dgilmore: gotcha.  I need to document my planned stages of development/deployment for fedmsg.
16:51:04 <dgilmore> threebean: i could see people wanting say a rss feed of build/tag notications, or email. or even im
16:51:15 <abadger1999> I think we'd have a notification server that consumes messages, sees "message from koji about package python-foo"; consults list of owners and sends to them.
16:51:26 <threebean> abadger1999: exactly.
16:51:32 <dgilmore> abadger1999: right.  thats kinda what im thinking
16:51:35 <nirik> or more general even...
16:51:48 <nirik> 'message' -> 'match list of actions' -> call actions.
16:51:48 <threebean> with users setting preferences in fas
16:52:32 <dgilmore> threebean: i dont think fas is the right place for that. but im open to being wrong
16:52:36 <nirik> so, 'package built' -> find things to do -> mail maintainer, mail other people who care, update rss feed foo, push to twitter
16:53:12 <lmacken> dgilmore: well, right now fas has that arbitrary configuration table that we could use today... we may want a seperate service for the 'username -> topics they care about' mapping
16:53:23 <abadger1999> threebean: ah -- presently that sort of config is in pkgdb but I could see moving it into fas as well.... might have to be a plugin, though (and I don't know that I particularly like the fas plugin architecture)
16:54:08 <dgilmore> abadger1999: right, i think for koji consumptions pkgdb makes the most sense
16:54:21 <dgilmore> or even in the notification app
16:54:25 <abadger1999> If we did that, we might want to move the pkgdb's watchbugzilla and watchcommits acls to the same place.
16:54:31 <abadger1999> <nod>
16:54:41 <dgilmore> as long as its trivial to manage it doesnt matter to much
16:55:14 <abadger1999> threebean: what's it look like for integrating w/ rh bugzilla?
16:55:28 <nirik> so this kinda makes me think the messaging thing needs a app frontend then?
16:55:39 <lmacken> heh, 0mq hooks into perl? that should be fun.
16:55:47 <lmacken> nirik: busmon!
16:55:59 <abadger1999> Yeah -- the infrastructure can be done without a frontend... btu a frontend would make it more useful.
16:56:04 <threebean> abadger1999: there's a team in RH that's working on forwarding messages out to us over AMQP.  we'd need to build a 0mq<->AMQP bridge.
16:56:18 <lmacken> thankfully, moksha can speak both :)
16:56:29 <threebean> nirik: yeah, I've got a proof of concept bus monitor webapp - http://github.com/ralphbean/busmon
16:56:41 <lmacken> threebean: aw, no live demo? ;)
16:57:07 <threebean> nirik: can we stand that up in staging during the beta freeze, or do we need to wait?
16:57:18 <nirik> threebean: sure, staging isn't frozen
16:57:19 <abadger1999> threebean: Okay -- I'm asking because if we moved watchbugzilla out of pkgdb, the notification app would probably need to be able to talk to bugzilla somehow.
16:58:01 <threebean> abadger1999: I'll ping that team again to see where they are.
16:58:25 <abadger1999> threebean: also... figure out just what they want to provide....
16:58:34 <threebean> nirik: what do you recommend.  I should file a request for resources?
16:58:44 <abadger1999> threebean: in particular, I'm not sure what they'll do around Security bugs in Fedora, for instance.
16:58:56 <nirik> threebean: sure, or perhaps start a discussion on infra list...
16:59:00 <threebean> abadger1999: last I checked, they didn't know.  :)  I'll raise it.
16:59:05 <nirik> on how we want to start deployment.
16:59:21 <threebean> nirik: good idea.  I'll go that route.
16:59:22 <abadger1999> I think ideally, we'd get information on those and make our own decision as to whether to forward them on and to who.
16:59:28 <nirik> so does busmon store prefs per user? or ?
16:59:48 <lmacken> it's db-less right now, afaik
16:59:50 <threebean> nirik: nothing at all.  it's a totally passive debug tool with shiney graphs.
17:00:01 <nirik> ah, ok.
17:00:34 <nirik> so we would want to use fas to subscribe to things for end users? or each app would consume messages and in turn deal with what it offers users?
17:01:16 <abadger1999> For notifications -- I'd rather move that outside of the individual apps.
17:01:36 <threebean> agreed.  whether those prefs are stored in pkgdb or fas, I don't really care.
17:01:39 <abadger1999> I think UI wise, it's painful for users to have to configure some knobs in bugzilla, some in koji, and some in pkgdb.
17:01:46 * nirik nods.
17:01:59 <abadger1999> Which probably means either FAS or in a separate application db.
17:02:08 <abadger1999> *notification application db
17:02:28 <nirik> yeah. Just from the standpoint of another app/db, fas might be nicer. ;)
17:02:47 <abadger1999> apps would be free to consume messages fronm other apps for other purposes, though -- for instance, it would make invalidation of caches a lot more straightforward, I think
17:02:54 <lmacken> yeah, stuffing the configs in fas was just my answer to "where we can put this stuff now", and not "where is the best place to put this stuff" ;)
17:03:55 <threebean> I think we're getting over time.  Any last comments before I close it?  I'll run this each Tuesday at the same time.
17:04:40 <threebean> No?  Thanks!
17:04:43 <threebean> #endmeeting