A quick but complete walk-through
Klaxon is a tool for monitoring websites, built by and for journalists. But it could also be useful for anyone you might think of as an “information first-responder” – activists, concerned citizens, community members. For an example, imagine there’s a State Department webpage that will, at some point, post an update on DACA – and you want to know right away when it does. Klaxon is built for exactly this purpose.
There have been, and are, other services for doing this, like Visualping. But I wanted to give Klaxon a try for several reasons. It was created specifically for journalism by some people – the team at the Marshall Project – who are doing important work. It’s also free and open source, and comparable tools come with very limited free tiers and then various cheap-to-expensive pricing structures.
For more, read this article or this one on why Klaxon is useful for journalists.
First, a note on the format here – this is an unfiltered walk-through of my experience getting up and running with Klaxon, intended for users with basic web technology skills, and including any hiccups I encountered and my own mistakes.
Klaxon’s own documentation and installation guide are good and there was nothing missing. Still, I thought a detailed walk-through could be useful for someone considering installing it and using it themselves.
First, I navigated over to the Klaxon Github page and gave it a read-through.
They’ve got a big purple button on it that says “Deploy to Heroku.” Could it really be that simple? I logged into Heroku to give it a shot.
And then pushed the magic button back on the Klaxon installation guide on the project Github page.
The link opened in Heroku and cued me to create a new app.
So I entered a name for the app, an email and hit “Deploy app.”
Uh oh:
I found myself sadly confirmed in my belief that nothing technical is ever as easy as hoped. I first tried for the simple fix, and made sure the credit card on the account was up to date. Still no dice. Then, a fresh try on another Heroku account. Nope.
So I filed an issue describing the problem on Github so the folks at the Marshall Project were aware, and got busy with some other things, had lunch, and came back to it in a few hours.
I tried again … and it worked just as it’s supposed to.
Was it just downtime on Heroku’s end? Probably. I’m tempted to gloss over this and not include in the walkthrough, because it doesn’t really have anything to do with Klaxon. But it brings up a relevant point: this kind of thing – a cryptic error with no way to fix it – is the downside of Heroku’s overall simplicity and ease of use.
(Aside: Usually I do my own system administration on a Digital Ocean droplet. Of course, that can be a lot of work and effort. And, cryptic errors are extremely common. However – they are almost always cryptic errors you can then Google, and fix yourself, unlike the hermetically sealed box that is Heroku.)
Anyway, in case you encounter this problem you may want to just give it a while, then try again before either giving up or writing the managers of the software.
When I opened the app I landed at a Heroku dashboard.
I clicked “Open App” and a new browser tab opened, with the URL to my Klaxon web app and a bright and … you might even say loud, splash screen.
On entering my email (the email I entered when setting up the app in Heroku), it gave me this screen, announcing it sent me an email confirmation.
So this was kind of interesting. I went to my email, and expecting to encounter a system to set up a password, instead was just given a link to open the dashboard.
Which looked like this, when I first opened it. Nothing was missing. There were no passwords to set or forget. It just relies on a secure email getting received and opened each time.
So I followed the instructions – first I had to turn on the “make my bookmarks bar visible” setting in Chrome – and now have an Add to Klaxon button up there.
Now … What would I like to monitor to try this out?
I browsed over to the California Department of Education’s website to find their news release page. Now, there’s probably a list I can get on for this, but just in case they wanted to quietly release some announcement here and I wanted to be sure not to miss it – I thought I might as well set up an alert.
Here’s the page:
When I clicked the bookmarklet, it gave me a sidebar that popped up and cued me to hover over the page and click the part I wanted to watch for changes.
I was pretty sure that whole page section where they have the months, and news releases by month, is what I wanted to monitor. So I clicked that and it turned green.
It did give me a little flash message on the Klaxon sidebar, saying “saved” but then remained re-selectable so I could change my selection. When I scrolled down on the white bar, I could see an area that says “Saved Selector” – that’s helpful because I can pull up my browser’s page inspector and check that’s the HTML page element I want to target.
Now I wanted to confirm I’m watching that page. So I went back to the tab with the Klaxon dashboard. It looked a little different this time.
Now, in addition to the message about the bookmarklet, I’ve got a Watching section, and one entry in it. I clicked to “Edit” that entry and got this page.
I gave it a name, in the Title field, and clicked to receive email when this page changes.
Then I hit update. Back on the app homepage, I saw the Watching section had my edit.
So far, so good!
One of the first things I wanted to do is set up another user. To do that, just go to Settings:
And then, create a new user on that screen.
Then:
Now seemed like a fine time to visit the Help tab. It is, indeed, helpful, with sections on the bookmarklet, what the “Feed” is (it populates with a chronological log of changes to all the websites you’re watching), adding a klaxon (in Klaxon-speak, an alert) to a website manually, and understanding what’s changed on a page, as well as how to add notifications to a Slack channel.
Still, I only feel like I know how something works after I try it myself. So to demonstrate, to myself and for this post, I decided to set up an alert for this website, make a change to it, and see how the change is logged in Klaxon.
I went to the Links page, and clicked the bookmarklet, then selected the part of the page that has a list of links. I also went to Klaxon and added my email to the notifications.
Now, just to make this a little more self-referential, I added a link to the Klaxon project to my links page.
I didn’t get a notification right away. And it happened to be time to leave the office.
So, I picked it up the next morning, and … there was still no notification. It got me to wondering how often the klaxon you’ve set checks that page. Or even if this was working correctly, though the dashboard seemed to clearly indicate that it was. Why weren’t my klaxons going off? My first instinct was to check the settings in the web app, looking for a way to tell the klaxons how often to check.
Well, I went back to the documentation and realized that – as usual – the problem was that I’d missed or misconfigured something or not read closely enough.
There’s an important step in Heroku I needed to take to make sure that Klaxon will go and run its routine website-checking tasks. It’s actually made completely clear in the installation guide on the Klaxon Github, I just missed it because of the Heroku hangup I encountered.
I went to the Heroku dashboard and opened, in the “Installed add-ons” section, the Heroku Scheduler.
In the scheduler, I type in the rake check:all task and set it to run hourly – you can set this for greater frequency if you like.
What is this cryptic command? The check:all is clear enough but if you’re wondering what rake is, there’s no big mystery. It’s just a task management tool, which we are telling to go do something (check if there’s any work to do / tasks to run / websites to check for changes).
After that I landed at a screen showing my scheduled tasks. If need be, I can edit them.
Pretty nice. I’m new to this part of Heroku, but having done my own system administration and arcane-to-normal-people stuff like crontab, let me assure you this is as easy as it gets for scheduling a server.
So, now I just needed to wait, make another change to my page (Klaxon needs an initial scan to compare things to, obviously) and wait some more.
The change I made was super basic – I edited the link on the page to say “News Klaxon” instead of “Klaxon for news.” Roughly an hour later, I got an email.
The emails Klaxon sends you are beautifully clear and simple. My notification gave me the old text in red, the new text in green, unchanged text in the gray background – and handy buttons to either go right to the page in question, or to the Klaxon snapshot.
The Klaxon snapshot viewed on the web app looked similar to the email, but with a bit more information – the amount and frequency of changes to the page – as well as the option to label the change for your own documentation, and view past snapshots.
When I browsed over to “All Snapshots” it gave me the option to compare any two. Those of you familiar with Git and other version control systems will recognize the familiarity. For the rest of you: think of it as a ‘track changes’ system for any page on the Internet. I could see this as being particularly useful for people wanting to compare the evolving language of a text online – especially ones where the nuance is important, such as legal documents or official government or corporate statements.
Now that I had a change in the system, when I browsed over to my Klaxon feed, I was greeted with this:
Awesome – so now, if I set up a bunch of alerts on different pages, when I land in the Klaxon site I can view a feed of what’s changed across all of them in chronological order.
That wraps it up for this walk-through. To quickly recap: getting Klaxon up and running is quite easy, and most of the potential confusion thing about it is dealing with Heroku and / or Github when you’re not familiar with those platforms.
The only kinks I encountered were random downtime on Heroku and then, forgetting to set the task scheduler. Aside from that it worked like a dream. Just relax and follow the Klaxon documentation step by step, and it all should be set up before you know it.
