Name: Andrew Elwell
Member since: 2001-05-29 23:26:09
Last Login: 2010-03-08 07:14:25

Homepage: http://cern.ch/aelwell

Recent blog entries by elwell

Overlaying SLURM job timings on Grafana plots

As you may have noticed, I'm quite fond of Grafana and use it at home and work. One of the dashboards I have at work is the general state of our lustre filesystems, showing IO and metadata traffic, collected by a custom python script (I'm working on converting this to a real collectd python plugin) which stores the data in an influxDB.

I've since written a small python script that talks to our SLURM accounting DB, so that given a jobID, we can get the start/end times and overlay those using the annotations API. One minor niggle in that the API expects epoch milliseconds, and seems to be tied to the TZ of the browser that generated the API key.

however...

~$ annotate_job 2924399
Found the following job:
   User: bskjerven (pawsey0001)
   Cluster: magnus, Partition: workq, QOS: normal
   Nodes: 768, CPUs: 36864
   Start: 2018-06-11 17:23:22, End: 2018-06-11 19:54:44
Got something back - Annotate? (y/n) y
200 - Annotation added

and lo - 

Syndicated 2018-06-14 12:14:00 (Updated 2018-06-14 12:15:24) from Andrew Elwell

weewx to home-assistant

At home I have a Fine Offset (this one branded by Jaycar) weather station that publishes to weather.two-fifteen.com via weewx (this is much simpler now I don't have to have the thing solar mounted in a field using a 3G dongle - at least the NBN is useful for some things) but I'd like to be able to use some of the measurements in home-assistant.

There isn't yet a direct plugin (spare time project anyone?) that I can see, but because I'm pushing the metrics locally to influxdb (have I mentioned I like drawing graphs of things?) for grafana, it's possible to use this in home-assistant via weewx-influx

Weewx config:
[StdRESTful]
      [[Influx]]
        host = localhost
        database = weather
        unit_system = METRIC

and on the home-assistant server:
  - platform: influxdb
    host:    203.0.113.88
    queries:
    - name: Outside Temp
      database: weather
      measurement: 'record'
      field: 'outTemp_C'
      group_function: last
      where: 'time > now() - 5m'
      unit_of_measurement: °C
      value_template: '{{ value | round(1) }}'

and lo and behold, you should end up with something like this:

which when compared to our nearest BOM observation site up the road in Grove correlates nicely (Grove is in a valley, we're at the top of a hill)


Syndicated 2018-03-22 22:33:00 (Updated 2018-03-22 22:33:09) from Andrew Elwell

Don't count your chickens...

We have a cheapo Chinese incubator for hatching eggs. According to popular Internet postings, the calibration of the 'temperature setting' on the front vs reality inside isn't terribly accurate. Since I have a stack of 'Ruuvitags' (https://ruuvi.com) from their kickstarter, I decided to put combine them with Grafana so I could start logging the data and plotting trends.

First up, by default they broadcast an Eddystone beacon, so that you can simply see the data on a phone / tablet (via the Physical Web), however with Google dropping this feature, I decided to switch them to Raw mode which has a higher accuracy. This is done by simply opening the tag and pressing the 'B' button to toggle between Raw (LED blinks red) and URL (LED blinks green) mode.

I'm using a Raspberry Pi 3 as a Bluetooth receiver. Running Rasbian 9 (stretch) means that I get a recent (v 5.43) version of bluez which understands BLE. Although there's a Java app to push to influxdb, I'd rather use Python, so pip-installed the ruuvitag_sensor package.

Initially this worked 'OK', but the logs were full of noise on the listener, so I hacked up a quick script based loosely on the examples. When done, it was much cleaner than the original and was picking up more of the broadcasts.

Once again, trivial to add to influx with some templating


Syndicated 2018-01-15 22:46:00 (Updated 2018-01-15 22:46:38) from Andrew Elwell

Plotting Lustre MDS stats

At $dayjob we have several large filesystems - for example our /scratch system has 3.1 PB of space using over 1000 HDDs. Although each vendor offers their own dashboard for monitoring they're all a little bit crap and don't integrate with anything else.

Cue an afternoon setting up influxdb (trivial) and grafana (also trivial) on a spare VM and a simple python script run on the metadata servers:

[admin@snx11038n003 ~]$ cat push_mdt_stats.py
#!/usr/local/bin/python2.7
import urllib
import time

def grabbit(mds):
post = ""
with open(('/proc/fs/lustre/mdt/%s/md_stats' % mds), 'r') as f:
for line in f:
k,v,null = line.split(None,2)
if k == "snapshot_time":
ts=int(float(v)*1000000)
else:
post += 'metadata,fs={3} {0}={1} {2}\n'.format(k,v,ts,mds)
with open(('/proc/fs/lustre/mdd/%s/changelog_users' % mds), 'r') as f:
tmp = f.read().split()
# we can cheat here as they have the same format - 3rd item in list is current changelog count, and then
# from the 6th item on we get changelog id / position to pull into a dict
head = int(tmp[2])
clog = dict(zip(tmp[5:][0::2], tmp[5:][1::2]))
post += 'changelog,fs={2} head={0} {1}\n'.format(head,ts,mds)
for cl,count in clog.items():
post += 'changelog,fs={3} {0}={1} {2}\n'.format(cl,count,ts,mds)

post=post.encode('ascii')
p = urllib.urlopen('http://influxbox:8086/write?db=lustre&precision=u',post)
#print(p.getcode())

while True:
try:
grabbit('snx11038-MDT0000')
except:
sys.exit("Whoa, that went a bit Pete Tong!")
time.sleep(10)

And a couple of clicks in Grafana can soon knock up a dashboard:


Syndicated 2017-10-10 02:14:00 (Updated 2017-10-10 02:14:18) from Andrew Elwell

PSU tinkering, Part 1

As previously blogged, I've got a couple of 12v 88.7A PSUs that I'm trying to control under arduino. Stage 1 complete - It powers up with a trivial bit of code

/* Arduino control for (ex) server PSU 
* Andrew Elwell <andrew.elwell@gmail.com> August 2016
* Released under BSD licence
*/

/* Controls / Pins based on data sheet available at
* https://belfuse.com/resources/PowerSolutions/SFP1050/bcd20031_ab_sfp1050-12bg.pdf
*
* A6/B4/C4/D4 +3.3 standby (power to arduino)
* A3/B1/B3/C1/C3/D3 Return
* B5(SDA) / C5(SCL) I2C
* B6 Bring low for PS ON
* C6 AC OK (if high)
* D6 PWR OK (if high)
*
*/


#include <wire.h>

int ACOK = 2;
int PSON = 3;
int PWROK = 4;
int LED = 13;

void setup() {
Wire.begin(); // join i2c bus (address optional for master)
pinMode(ACOK, INPUT);
pinMode(PSON, OUTPUT);
pinMode(PWROK, INPUT);
pinMode(LED, INPUT);

digitalWrite(PSON,HIGH) ; // Stay off until ready
}

void loop() {
if (digitalRead(ACOK) == HIGH) {
digitalWrite(PSON,LOW) ;
}
if (digitalRead(PWROK) == HIGH) {
digitalWrite(LED,HIGH) ;
}
}

The one gotcha that I needed to get it working was to also bring PS A0 low (I2C address) and suddenly green led and 12v out!



Syndicated 2017-08-28 15:29:00 (Updated 2017-08-28 15:34:57) from Andrew Elwell

119 older entries...

 

elwell certified others as follows:

  • elwell certified elwell as Apprentice

Others have certified elwell as follows:

  • MikeGTN certified elwell as Apprentice
  • elwell certified elwell as Apprentice
  • tef certified elwell as Apprentice
  • kylegordon certified elwell as Apprentice
  • dangermaus certified elwell as Master
  • badvogato certified elwell as Apprentice
  • sye certified elwell as Apprentice

[ Certification disabled because you're not logged in. ]

New Advogato Features

New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

Keep up with the latest Advogato features by reading the Advogato status blog.

If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!

X
Share this page