Tuesday 3 August 2021

Strange report bug

My company has a private table called TEST_BATCHES in which are stored 'batches', groups of work orders that are processed together in our new nesting machine. Generally one batch contains all the work orders required to manufacture 10 items of a given order line. We've recently discovered that the nesting machine produces a file that lists all the batches that it has processed in the past day and I have put this to good use, writing a procedure that extracts the relevant data from this file (not particularly easy) and reporting all the work orders as completed.

It happens that the file might have eight batches but only five are reported by this procedure, because three have already been reported manually. I wanted to add a little code to the procedure that would send me an email listing all the batches that are in the nesting file but not reported. I won't show how this is done because it is not particularly relevant; the unreported (or previously reported) batches are stored in a linked copy of STACK2 and this table is passed to the report that creates the email.

Every day I have been receiving an email listing all the batches that have ever been created (not that many!) which is clearly wrong. I checked the procedure, and STACK2 holds at most a few records at the end. I emptied the real STACK2 table in case this was causing problems; it wasn't. I checked the very simple report to see that there were no mistakes there - like marking TEST_BATCHES.BATCH with a question mark, so that all records would be shown in the report. No.

Today I had the idea of dumping the report in WINDBI. I was expecting to see the following

SELECT TEST_BATCHES.BATCHNAME FROM TEST_BATCHES, STACK2 WHERE TEST_BATCHES.BATCH = STACK2.ELEMENT AND TEST_BATCHES.BATCH > 0 AND 1 = 1;
What I saw, though, was something different that explained why the report contained all the records in TEST_BATCHES:
SELECT TEST_BATCHES.BATCHNAME FROM TEST_BATCHES WHERE TEST_BATCHES.BATCH > 0 AND 1 = 1;
Where had STACK2 gone to? After contemplating this for a few moments, I decided to add STACK2.TYPE to the report; lo and behold, dumping the report now showed the correct SQL query. Marking STACK2.TYPE as hidden removed STACK2 from the join statement again. As this field is always going to be zero (the procedure doesn't use it; it is already using STACK otherwise I would use this simple table), I compromised by marking the "don't display if zero" field. 

Tomorrow I expect to see only a few lines in the emailed report.

Considering this example, it's very much an edge case. I'm trying to think of other reports that might use only the key(s) of a table; most of my reports use STACK4 where both the key and data fields are displayed in the report. Otherwise there's no point in using STACK4 - I could just as easily use STACK. There might be a case of using only KEY1 and KEY2 of STACK8; I'll try and find such an example.

I did find an example of using only the keys of Stack8 and the above bug did not appear. I then created a new procedure with report that initially stored all the orders opened today in STACK.ELEMENT and then in STACK2.ELEMENT; the bug did not appear.

Moral of this story? Use the 'dump' function more often.

2 comments:

  1. I think this is less of a bug than an obscure optimisation feature. Perhaps because we can pass linked tables to the query there is an assumption that we won't join an extra table and only use the keys just to limit the population. As a result, joins to tables which aren't being used for display *and on which there are no conditions* are assumed to be unnecessary and their removal serves to simplify the query.

    I think you can remove the display field. Just add another STACK2.ELEMENT column to the report with the condition "> 0". Yes, it is supposedly redundant given the existing condition and the join but it should serve to keep the table in the query.
    Actually you could probably remove the condition on TEST_BATCHES.BATCH after that.

    One other thing which might serve to keep the table in the query, even without the additional condition, and even if you remove STACK2.TYPE, is to add ! to the table ID of STACK2.

    ReplyDelete
    Replies
    1. As suggested in your second paragraph, I removed the condition from TEST_BATCHES.BATCH and added another column for STACK2.ELEMENT with the condition that it be non-zero. The dump shows the correct SQL query.

      Adding the exclamation mark made no difference.

      Delete