Earlier this week, the Apache Software Foundation's Infrastructure Team launched a new status service that displays a ton of information about the health of the Apache infrastructure and activity within its projects. Daniel Gruno, infrastructure architect at the ASF, explains how the service came to be and where it's headed.

Joe Brockmeier: Can you tell us about the development of the project, and where the idea came from, and how long it's taken to develop?

Daniel Gruno: The idea came when I was waiting for a bus home from my parents' house on Tuesday. I choose to call it an epiphany, because at the time, it seemed sort of mind-blowing to me; The status of the ASF's infrastructure isn't just what's up/down, it's what is going on right now. It's what's being written, what's being committed, what emails are being sent, who's doing what on our github mirrors and so on. With that in mind, I had a good think (and a couple of imaginary maniacal laughs) on the way home, and set to work straight away, asking my boss (David Nalley) for two days to focus on this project. As it often happens, the actual site was done over-night, but it took another full day to verify that the data mining was indeed working as intended, and then fixing all the things, that the few folks who had previewed it, thought needed a bit of a fix.

Joe: Who has worked on the project? Has it been several people in Apache Infrastructure, or others?

Daniel: In ASF, it's always team-work even when you work alone. You get an idea, you present it to the crowd, and collectively you figure out whether this idea is as good as it sounds, what priority to give it, and so on. I did the actual coding myself (I was so full of ideas, it was best to just get them out of my head as quick as possible), but the important factor is that the infrastructure team had my back – they were keeping our infrastructure running while I focused on this, and they provided much needed response to the things I produced.

Joe: What tools did you use to build the service? Any specific frameworks or Apache projects?

Daniel: I'm a httpd fan boy, so there is always an httpd instance in the mix when I do something. The actual activity data is already provided pro bono by my company for the ASF, so it was really only a matter of grabbing that data (this is provided to us as JSON objects in this case) and turning it into useful information for our users and contributors. Part of the data that is mined is fetched from Apache Subversion's wonderful publisher/subscriber service called svnpubsub (I really wish more people knew about this awesome piece of software!), and from our git counterpart aptly named gitpubsub. There is also some jQuery and some Google Charts magic involved in the process.

Joe: Since this is Apache, should we assume the source for the service is available somewhere?

Daniel: Yes, the source for the page is in infra's public subversion directory: https://svn.apache.org/repos/infra/infrastructure/trunk/projects/status.

Joe: I notice that there are modules that track commits and show the top projects, etc. Is there a way for individual projects to get a status page that just tracks their community?

Daniel: That's a great idea! I have begun work on this, and anyone checking out the repository linked to above will be able to implement our preliminary widget into their site.

Joe: Any plans for new modules, improvements, etc. in the short/long term?

Daniel: Yes, we plan to extend this endeavour with a new sub-site where committers can subscribe to alerts. For instance, the OFBiz people could subscribe to notifications about the ofbiz VMs, people could sign up for watching out for feathercast or anything else they want to help out with, and so on.

Joe: Right now, it's all green. Is there a list of services that are down, or how do you visualize that?

Daniel: Yes; Some people have asked me where the list of services that are currently down is. To that I can only say; IF any of our public infrastructure services are down, you will notice it on the site very quickly – but for the most part (99.9% of the time) our services are really just all up and running, and we take great pride in that.

Joe: If other Apache folks want to get involved to help with this, how should they go about it?

Daniel: They should, as the site says, contact us at infrastructure@apache.org - we definitely welcome any help we can get in making this as awesome as possible, and I have already had a few offers to help improve the javascript that powers the site. As with any real Apache project, once you've written down the initial codebase (whether it be alone or as a group), you generally seek to include as many as you can in the project, so as to make it as resilient and diverse as possible.