Power meter logging with Pi and Emoncms

I want to log my house electrical usage since long time ago, but somehow some hardware and time is not allow. Until recently, I bought a power meter from Taobao, single phase max. 65A input. It's a LCD power meter with lot of information including voltage, ampere, power, power factor, frequency, and
energy. On top of that, the power meter has S0 pulse output for me to connect into my Pi for power monitoring and logging.

I had put this meter for more than a month due to my workload, and until yesterday i used my New Year holiday to install this power meter into very packed distribution board. Pull the signal cable from power meter to my Pi. The connection diagram as follow.

I had try few different value of resistor and I found that when the resistor is high e.g. 10kohm, there are lot of false trigger and causing the pulse count meaningless. After try and error, I found that 330ohm resistor is the most suitable value, it is high enough to prevent overburden, and it is low enough to prevent false trigger.

I use GPIO 23 as my input cause it got internal pull-up resistor to make the circuit more easier.

Until this moment, the hardware is ready. The rest is more on the software.

First, need to have Pi code to analyze this input and post to internet. Then the second step is to have portal to record the data and visualization.

After searching around, I found code from Edward O'Regan and KieranC is a good place to start. I didn't modify much of the quote except the json post url, which I will explain later. The document can get from GitHub, it give pretty much all you the requirements you needed. Once you have setup the program in Pi, then replace the code monitor.py with this one.

• #!/usr/bin/python
• 

• """
• ****** 2015/01/01 *******
• Modified by SK to post power data to EmonCMS portal using EmonCMS API.
• *************************
• 

• Modified by KieranC to submit pulse count to Open Energy Monitor EmonCMS API
• 

• Power Monitor
• Logs power consumption to an SQLite database, based on the number
• of pulses of a light on an electricity meter.
• 

• Copyright (c) 2012 Edward O'Regan
• 

• Permission is hereby granted, free of charge, to any person obtaining a copy of
• this software and associated documentation files (the "Software"), to deal in
• the Software without restriction, including without limitation the rights to
• use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies
• of the Software, and to permit persons to whom the Software is furnished to do
• so, subject to the following conditions:
• 

• The above copyright notice and this permission notice shall be included in all
• copies or substantial portions of the Software.
• 

• THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
• IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
• FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
• AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
• LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
• OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
• SOFTWARE.
• """
• 

• import time, os, subprocess, httplib, datetime
• from apscheduler.scheduler import Scheduler
• 

• #Uncomment the following 3 lines if the code didn't work
• #import RPi.GPIO as GPIO
• #GPIO.setmode(GPIO.BCM)
• #GPIO.setup(23, GPIO.IN, pull_up_down = GPIO.PUD_UP)
• 

• # The next 2 lines enable logging for the scheduler. Uncomment for debugging.
• #import logging
• #logging.basicConfig()
• 

• pulsecount=0
• power=0
• 

• # Start the scheduler
• sched = Scheduler()
• sched.start()
• 

• # This function monitors the output from gpio-irq C app
• # Code from vartec @ http://stackoverflow.com/questions/4760215/running-shell-command-from-python-and-capturing-the-output
• def runProcess(exe):
•     p = subprocess.Popen(exe, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
•     while(True):
•       retcode = p.poll() #returns None while subprocess is running
•       line = p.stdout.readline()
•       yield line
•       if(retcode is not None):
•         break
• 

• #Pulse in monitoring
• #GPIO.wait_for_edge(23, GPIO.FALLING)
• #pulsecount += 1
• 

• # Every minute this function converts the number of pulses over the last minute into a power value and sends it to EmonCMS
• @sched.interval_schedule(minutes=1)
• def SendPulses():
• global pulsecount
• global power
• #print ("Pulses: %i") % pulsecount # Uncomment for debugging.
• # The next line calculates a power value in watts from the number of pulses, my meter is 1600 pulses per kWh, you'll need to modify this if yours is different.
• power = pulsecount * 37.5
• #print ("Power: %iW") % power # Uncomment for debugging.
• pulsecount = 0;
• timenow = time.strftime('%s')
•         url = ("/input/post.json?time=%s&node=1&json={power:%i}&apikey=<YOUR API KEY>") % (timenow, power) 
• # You'll need to put in your API key here from EmonCMS
•         connection = httplib.HTTPConnection("emoncms.org")
•         connection.request("GET", url)
• 

• for line in runProcess(["/usr/local/bin/gpio-irq", "23"]):
•     pulsecount += 1
•     #print ("Count: %i") % pulsecount

Since you are going to post the data to EmonCMS portal, therefore, you do not need to setup the EmonCMS on you Pi.

The next step is to create an account with EmonCMS. After that, copy the write API key and paste it to the code and you are ready to monitor your power consumption online.

You can show off your newly setup power monitoring like this

Somehow, Pi is not very good in real-time data collecting, as you can see in the chart, it will miss some schedule and causing data is very odd. This will be my next version of code to eliminate it.

Plot.ly and Pi

I found Plotly since early last year, it is a very interesting online data visualization tool. I always want to use it for some logging purpose. So finally I got my Pi and I start to do a simple task, to log the CPU temperature, and here is the result.



I use Python as the programming language since it is the most famous language for Pi. If you new to Pi, here is the step to install Python.

• sudo apt-get install python-dev
• wget http://python-distribute.org/distribute_setup.py
• python distribute_setup.py
• wget https://raw.github.com/pypa/pip/master/contrib/get-pip.py
• python get-pip.py
• sudo pip install virtualenv

Once you have installed the Python on your Pi, the next task is to install plot.ly API.

• pip install plotly

At the same time you can register an account with plot.ly in order to use their services.

Now everything is ready, and we need to create a Python program to get the temperature data and upload it to plot.ly. Before I continue, there is something about Python that you should know, especially C/C++ programmer.

First, you do not need to put semicolon (;) at the end of the line. It looks like a Basic code.
Second, do not mix tap and space. You will get an error if you mixed it.
Third, you do not need to declare the variable. Ok, it looks like Basic now.

I'm very new to Python with nearly zero knowledge about this language. Thanks to the Internet, you will get some examples to help you to get the code you needed.

So here is the code.

• #!/usr/bin/env python
• import plotly
• import os
• import subprocess
• import datetime
• 

• def update_temp():
•   py = plotly.plotly(username_or_email='YOUR_USERNAME', key='YOUR_API_KEY')
•   i = datetime.datetime.now()
•   proc = subprocess.Popen(["cat", "/sys/class/thermal/thermal_zone0/temp"], stdout=subprocess.PIPE)
•   t = proc.stdout.read()
•   r =  py.plot(i.strftime('%Y-%m-%d %H:%M:%S'),float(t)/1000,
•   filename='RPiTempCont',
•   fileopt='extend',
•   layout={'title': 'Raspberry Pi Temperature Status'})
• 

• if __name__ == '__main__':
•   import sys
•   update_temp() 

So that is the code that get the current date, time and temperature and upload to your plot.ly account.

In order to do it automatically, you need to add the program to cron job. But before that you need to save the Python program to a location. For my case, I put it at

• /home/pi/scripts/

To add the task to cron, you need the following commands

• crontab -e

Then add the task

• */10 * * * * usr/home/python /home/pi/scripts/YOUR_PYTHON_FILE.py > /dev/null

What it does is to run the Python program every 10 minutes for 24x7x365. Now you can monitor your Pi temperature in real time.

I believe you will not satisfy with this simple task only. So, my next project will be logging weather information including temperature, relative humidity and solar irradiation. Yes, you are right, solar irradiation, to monitor my mini PV farm.

Next post coming soon.

My Pi part 2

My Pi is up with Raspbmc and communicate with my SMA inverter and update data to pvoutput.org. But I still lack of something, an USB hub, an external powered USB hub.

Initially, I went to the IT shop to seek for a cheap solution, but the price just stop me. A 4-port powered USB hub from Belkin is selling about RM70. If a Pi only cost RM111, how unreasonable if I get a hub for RM70. So, I make up my mind to do it one for myself.

The idea is very simple, you get an old USB hub that can be powered by external power supply, then you cut the power terminal that connected to the Pi, left only data line. This is to prevent unbalance voltage that might causing the Pi malfunction.

Once you have cut the power supply connection, it is safe to connect the hub to Pi. I found a 5V, 3A adaptor from my store, so it is just nice to use it, lucky :)

Now the external powered hub is ready to use. I connect back the external HDD, and I can continue enjoy the movie. Too bad that Raspbmc does not support rm/rmvb file. But that is not a major problem, just get a converter and convert it to other format will do. With the current CPU power and GPU power, it is just a few seconds job to convert rm/rmvb to avi.

Then the next task will be the remote for my Pi. I order a TSOP34138 infrared receiver from Farnell with cost of about RM5. Then I found an old VCD player remote and use it to become the remote for my Pi. STM Labs has a very detail instruction of how to use GPIO of Raspberry Pi to become IR receiver. For those that can't afford to buy a nice IR from Logitech, this is the cheapest solution that you can use.

Basically, I have done most of my tasks, but I still have more challenging task to do with my Pi. Will update once I have more detail to share.

My Pi

I was a fan of Raspberry Pi since beginning of the release, this credit-card-sized single board computer really attract my attention. I always want to use some simple hardware to perform some simple task, using minimum computing power to execute daily not so complicated computing task, e.g. media player, data logger, web server, home automation, NAS, etc. I have few old PC which I can use it for those function but few reasons stop me from doing that.

First of all, the power consumption of those PC is too high. For a normal Intel CPU, you are talking around 65W up to 250W which the power consumption of a monitor is not yet being included. Lets say, it takes an average of 100W and I don't need monitor. For this PC to run 24x7x365, the electricity per annual will be

100W x 24hrs x 365 days =  876kWhr/annual or 73kWhe/month

Our electricity tariff is based on the total amount of electricity you are using, the lesser you use, the lower the rate. For domestic, the tariff range from RM0.218/kWh until RM0.454/kWh (however year 2014, TNB give us the best new year gift, the maximum rate is RM0.571/kWh, thank you to all fuckers that vote blindly)
My house average monthly electricity consumption is around 400kWh, which is about RM100++ of electricity bill. If let say I use normal PC to do these additional function, I need to pay extra 73kWh x RM0.516, which is around RM38 per month, or RM450 per year. That's why I aiming for this lower power system that use very minimum power and at the same time do the job I want.

Raspberry Pi consume around 3.5W of power, which mean about 32kWh/annual or 2.6kWhr per month. That mean I need to pay extra RM16 per annual if I power my Pi for 24x7x365. Well, I believe this kind of power consumption you can't complain with.

I have been waiting a chance to buy a Pi, until recently, my wife asking me if I can buy a media player for her farther, and immediately I donate my media player to her :) and I get myself a Raspberry Pi from Farnell. It cost me RM111 for the Pi, then I get a acyclic case from ebay for around RM12.

So, what I going to do with this PC?

First of all, I need to make it a media center. This job is very easy, just go to Raspbmc and follow the instruction to install the image into a SD card. I have tried few times to make the machine up. The advice is use Lan for the installation process, once you are in the XBMC, then only you go for wireless.

The second task is to install a Bluetooth toggle to connect with my PV system and upload the data automatically to the internet. I'm using SMAspot, they got a very nice instruction of how to install and configure their program with Pi. Now I can monitor my PV system automatically from any where I go.

I still have few tasks to be completed:

1. To modify a normal USB hub to become powered USB hub. Initially, I was thinking to get a powered USB hub, but the price is too expensive, nearly RM70. So, I decided to get a power supply from my store and modify a normal USB hub into a powered USB hub so that I can connect my portable HDD to the Pi.

2. Remote control for the XBMC. Again, I will use a GPIO of the Pi to connect a infrared receiver and get a cheap Astro remote control.

3. NAS. I got few old 3.5" HDD and I thinking a way to connect it to the Pi. Will find a solution soon.

4. Adding an LCD display to the Pi. I got few 2.2" LCD with me, so will find a time to connect it to the Pi for some message display.

5. Home automation, This is a long term project. Will do it once I settle down.

Ok, a long post for the New Year.

Happy new year to everyone out there.