Today was quite interesting, though it didn’t involve any progress on the thing we’re working on. The service we tend to has a view which displays large tables of data about stuff, and also allows the data on display to be downloaded as an Excel spreadsheet. I’d assumed up till now that this was the usual "spit out a CSV” thing, but no, it actually constructs a proper spreadsheet, with formulae in it and so on!
Anyway, the problem is that for certain datasets, the data in the download had an extra row which didn’t appear on the screen. The important values in these extra rows (we’ve identified two different sets of data it happens with so far) are all zero, so no calculations are getting messed up. But obviously it’s an undesirable state of affairs, particularly as nobody is sure if the row in question should also be seen on screen, or shouldn’t be in the spreadsheet
So I went digging into what it was up to… and discovered, to my great dismay, that the code to build the dataset shown on screen and that to build the dataset in the spreadsheet don’t seem to share a single line! There are two completely separate chunks of code, each of which is supposed to be presenting the same representation of the underlying data; and now they aren’t
Also, one of these chunks is the horrendous piece of spaghetti we had to look at for other purposes recently, which is almost completely incomprehensible and was written by people who left years ago
And, of course, we aren’t allowed to download the entire database to work with locally or connect directly to the database server because it’s Official Sensitive, so the only way to investigate is to shell into the hotfix web server and poke at things from the command line there
I managed to get down to a level where I could grab a copy of the SQL that’s generated in each case, and confirm the disparity in results. But as one case also relies on three chained DB views, rather than going to the source table directly, and both of them do loads of aggregation involving other tables, it’s still a bit of a nightmare to work through; and then one has to understand how the high level ORM code caused it to generate that SQL. But I’ve left that info for my colleague, and I wish her luck in trying to solve it tomorrow - if only because she’s off on Monday so if she doesn’t, I’ll have to try to work it out then!
and a 
for my general cluelessness
.
Leave a comment: