Enphase Production/Consumption Metering

In the last blog, we looked at Panel Level Monitoring based on the thecomputerperson blog. Further down in the comments, is another useful URL.

The URL is http://[ip address]/stream/meter. Like Panel Level Monitoring, it prompts you for the user id and password. But the usual envoy/serial no. does not work this time. The user id is installer and the password is not obvious. Fortunately, thecomputerperson has a blog describing an app to derive this password. Once you enter the credentials, you get a continuous stream of data with more arriving every 1 second. Each batch of data looks like this.

data: {
  "production":{
    "ph-a":{"p":2412.766,"q":195.631,"s":2419.61,"v":243.321,"i":9.95,"pf":1.0,"f":50.12},
    "ph-b":{"p":2216.147,"q":171.409,"s":2220.896,"v":245.239,"i":9.068,"pf":1.0,"f":50.12},
    "ph-c":{"p":2176.172,"q":178.79,"s":2178.513,"v":239.326,"i":9.112,"pf":1.0,"f":50.12}
  },
  "net-consumption":{
    "ph-a":{"p":-2275.016,"q":-312.015,"s":2293.029,"v":243.378,"i":9.421,"pf":-0.99,"f":50.12},
    "ph-b":{"p":-2014.286,"q":-212.405,"s":2023.73,"v":245.4,"i":8.257,"pf":-1.0,"f":50.12},
    "ph-c":{"p":-2151.988,"q":-342.735,"s":2171.025,"v":239.412,"i":9.078,"pf":-0.99,"f":50.12}
  },
  "total-consumption":{
    "ph-a":{"p":137.75,"q":-507.647,"s":128.826,"v":243.349,"i":0.529,"pf":1.0,"f":50.12},
    "ph-b":{"p":201.861,"q":-383.814,"s":199.012,"v":245.319,"i":0.811,"pf":1.0,"f":50.12},
    "ph-c":{"p":24.184,"q":-521.524,"s":7.95,"v":239.369,"i":0.033,"pf":1.0,"f":50.12}
  }
}

The elements in this packet are:

• An extraneous data: prefix. Remove this and you have a valid JSON document.
• A section each for Production, Net Consumption and Total Consumption. Production is what you solar array is producing, Total Consumption is what you are consuming. Net Consumption is the difference between the two and what the electricity company charges you for. If it is negative, the electricity company pays you.
• Within each, are three sections for the three phases.
• For each phase is a series of values for Real Power (p), Reactive Power (q), Apparent Power (s), Voltage (v), Current (i), Power Factor (pf), Frequency (f).

The p, q and s values indicate the way complex loads use power. The p value is what is used to calculate your power consumption. With some loads, the voltage and current can be out of phase - it is like the device consumes power and then pushes some power back within a cycle. The values form a Power Triangle as shown below. The pf value is the Power Factor.

Next came the java program to get this data. Once a second was too much so I decided to get it every minute. The old code for the Panel Monitoring did not work this time. The streaming data uses Digest Authentication as well but it is a lot more fussy. The hastily written code used before would just not work. There was no option but to flip the learn switch on the old noggin'.

First was to have a closer look at how Digest Authentication works. When you use Digest Auth on the browser, this is what happens

• The browser requests the page as normal
• The server returns a 401 Unauthorized response. The response will also have a header including the following

WWW-Authenticate: Digest qop="auth", realm="enphaseenergy.com", nonce="iRMJXCHhky+Hk3taKN9+nQdU/d0="

• The browser prompts for a user id, password. It then send the request again but this time, the request header contains the following header that includes the user id and has the password encrypted

Authorization: Digest username="installer",realm="enphaseenergy.com",nonce="iRMJXCHhky+Hk3taKN9+nQdU/d0=",
uri="/stream/meter",qop=auth,nc=00000001,cnonce="0a4f713e",response="01010101010101010101010101010101",
opaque="01010101010101010101010101010101"

• The cnonce field is calculated by the browser and checked by the server. If all is well, the page is served up.

The old java code used several deprecated classes. All that had to go and was replaced by newer versions. Part of the code is shown below. The function fetchData() returns a null if the server returns with an authorisation error. But the unsuccessful call will populate the nonce. The next call should return data. The call just gets the first line. The second line is blank and this is when it closes the request, ignoring any further data.

    private static final String URI = "http://[ip address]/stream/meter";
  
    private static final String USER = "installer";
    private static final String PASSWORD = "password";

    private static String nonce = null;
    private static String realm = "enphaseenergy.com";

    public String fetchData() throws Exception {
        HttpGet httpget = new HttpGet(URL);

        HttpHost target
                = new HttpHost(httpget.getURI().getHost(), 80, "http");
        CredentialsProvider credsProvider = new BasicCredentialsProvider();

        UsernamePasswordCredentials credentials
                = new UsernamePasswordCredentials(USER, PASSWORD);
        credsProvider.setCredentials(
                new AuthScope(target.getHostName(), target.getPort()),
                credentials);

        CookieStore cookieStore = new BasicCookieStore();

        CloseableHttpClient httpclient
                = HttpClients.custom().setDefaultCookieStore(cookieStore)
                .setDefaultCredentialsProvider(credsProvider).build();

        DigestScheme digestAuth = new DigestScheme();

        digestAuth.overrideParamter("qop", "auth");
        digestAuth.overrideParamter("nc", "0");
        digestAuth.overrideParamter("cnonce", DigestScheme.createCnonce());
        digestAuth.overrideParamter("realm", realm);
        if (nonce != null) {
            digestAuth.overrideParamter("nonce", nonce);
        }

        AuthCache authCache = new BasicAuthCache();
        authCache.put(target, digestAuth);

        HttpClientContext localContext = HttpClientContext.create();
        localContext.setAuthCache(authCache);

        CloseableHttpResponse response;

        response = httpclient.execute(target, httpget, localContext);
        int code = response.getStatusLine().getStatusCode();
        if (code != 200) {
            if (code == 401) {
                Map wwwAuth = Arrays
                       .stream(response.getHeaders("WWW-Authenticate")[0]
                       .getElements())
                       .collect(Collectors.toMap(HeaderElement::getName,
                       HeaderElement::getValue));

                // the first call ALWAYS fails with a 401
                nonce = wwwAuth.get("nonce");
                realm = wwwAuth.get("realm");
            }
            return null;
        }
 
         BufferedReader br = new BufferedReader(
                new InputStreamReader(response.getEntity().getContent()));
        String input;
        String buffer = "";
        while ((input = br.readLine()) != null) {
            if (input.length() == 0) {
                break;
            }
            buffer += input;
        }
        httpclient.close();

        return buffer;
    }

In practice, the call fails with an authorisation error the first time, as expected. There is enough information received to set up the header for the next call. The next two calls go through OK, reading a single line only each time before closing the connection. By the third call, the nonce is invalid and it fails with an authorisation error. So the routine keeps executing this cycle, once every minute. The resulting JSON data is unpacked and appended to a file in CSV format. On each new day, a new file is opened. Run it for a few days and you will have a collection of these files. In the next blog, we will look at making sense of these files and putting them to good use.