15 Jun 2018 elwell   » (Observer)

Overlaying SLURM job timings on Grafana plots

As you may have noticed, I'm quite fond of Grafana and use it at home and work. One of the dashboards I have at work is the general state of our lustre filesystems, showing IO and metadata traffic, collected by a custom python script (I'm working on converting this to a real collectd python plugin) which stores the data in an influxDB.

I've since written a small python script that talks to our SLURM accounting DB, so that given a jobID, we can get the start/end times and overlay those using the annotations API. One minor niggle in that the API expects epoch milliseconds, and seems to be tied to the TZ of the browser that generated the API key.

however...

~$ annotate_job 2924399
Found the following job:
   User: bskjerven (pawsey0001)
   Cluster: magnus, Partition: workq, QOS: normal
   Nodes: 768, CPUs: 36864
   Start: 2018-06-11 17:23:22, End: 2018-06-11 19:54:44
Got something back - Annotate? (y/n) y
200 - Annotation added

and lo - 

Syndicated 2018-06-14 12:14:00 (Updated 2018-06-14 12:15:24) from Andrew Elwell

Latest blog entries     Older blog entries

New Advogato Features

New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

Keep up with the latest Advogato features by reading the Advogato status blog.

If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!