How to fill the data gap?

Reed_Richards

If this is true then Loop gets data from n3rgy faster than the consumer does.  This makes some sort of sense if n3rgy deliberately delay supplying data to the consumer purely so that they can charge for more up-to-date data.  But if n3rgy extracts data from my meter to a schedule and holds on to this data until such time as Loop or I ask for it then n3rgy is maintaining the database that @Phones4Chris assures me does not exist.  

Phones4Chris

It can also depend on how a supplier processes the data. Just to make that point data on the Octopus App can appear before it appears in the Octopus "My Account", and I have seen it the other way around!

Phones4Chris

@Reed_Richards Not sure exactly what you mean by "Loop gets data from n3rgy faster than the consumer does". You can request a range of dates for which you want data, what you actually get will be determined by the App provider. I'm pretty sure none of them will be making requests for your data until you ask for it!
Suppliers obviously have some sort of limited database because the information is there in your account.

Out of curiosity have you asked for just the 12 missing days (no others in the same request, rather than a wider range of dates that includes the missing dates) from n3rgy?

The assurance was that the DCC does not have a database for usage/meter readings. Another poster also said that. Individual App providers have their own servers and may have some form of limited short term storage, I don't know if anyone may have some inside knowledge for any of them.

Reed_Richards

What I mean is that the Loop consumption data is typically a few hours behind the actual time and the n3rgy consumption data is typically 8 to 12 hours behind the actual time.  If Loop gets its data from n3rgy, as somebody suggested, then n3rgy must have a few hours worth of data that is more recent than the data it supplies to me when I request it.

Asking for just the missing 12 days gets me a CSV file with just the headers on the top line, otherwise completely empty, no actual data.

I understand now that whilst the DCC does not maintain a database of half hourly meter readings, my supplier must have one and n3rgy responds so quickly I think they must also have a database.  Loop are a bit slower to respond so perhaps they request their data from the n3rgy database?  

Phones4Chris

Reed_Richards said:

........  If Loop gets its data from n3rgy, as somebody suggested, then n3rgy must have a few hours worth of data that is more recent than the data it supplies to me when I request it...................
........... my supplier must have one and n3rgy responds so quickly I think they must also have a database.  Loop are a bit slower to respond so perhaps they request their data from the n3rgy database?  

No, I don't believe so, to either of those ideas. n3rgy have a number of commercial customers as well as some consumers. Loop will no doubt be treated like a business customer. As I said before, you (as a consumer) may request a range of dates for data but what you actually get will not always be the most recent data on the current date. I expect this is done quite deliberately as I don't think it's intended to be a "live" service and therefore doesn't get the "priority in their systems. Loop will be treated as a higher priority.
I expect suppliers will have a limited database which will only have the data yet to be applied to customer accounts,. Otherwise all these would have to have extremely large systems and databases - which will cost!!
Loop aren't the only one to use n3rgy, Hugo does as well.

Bendo

N3rgy store the data in their own data warehouse once pulled. That's how they can go back years - they have my data for about 3 years.

If you have a meter replacement the can also link them so you can continue to see data from the old meter and the new meter in Hugo / Loop.

Phones4Chris

Ah, thanks for that info. Not that I've ever tried, but you can select dates that are prior to the last 90 days then?
i had assumed that as a free consumer service we were limited to the last 90 days as per the link provided!

Bendo

I think you can specify any 90 day period on the n3rgy consumer site, but it's clunky so it's been a while since I looked.

Reed_Richards

My smart meter was installed on 28th August 2023.  I just tried and I can download data from that day forward from the n3rgy consumer site (although the maximum range per download is 90 days).

Phones4Chris

@Reed_Richards Having re-read the thread, I see no mention of you using Bright. So far all the ones you've tried use n3rgy, and now we have discovered that they have their own data warehouse, if there was some glitch when the data was first pulled that could explain the missing data from all those apps to date.

Try Bright, they do not use n3rgy, they go direct to the DCC, so if the 12 days are still missing with them, this should confirm that some of the meter 1/2hr registers are faulty for whatever reason.
If you are lucky and recover those 12 days you can pass it to your supplier and also asked them to go and recover it themselves.