Announcement

Collapse
No announcement yet.

udrive write problems

Collapse
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • udrive write problems

    I'm having trouble with the FAT write function and wondering if I might have found a small bug?

    Test setup is a 128 byte file. Packet size is 32 bytes, so 4 packets. Tested with baud rates at 38400 and at 300.

    When I tested this with vb.net and with the FAT controller it works. But when I tested it on a CP/M system it does not work. The reason it does not work is subtle but logical and has taken a day of testing to discover this bug.

    The protocol for 128 bytes with a packet size of 32 bytes should get back 5 ACK bytes. One after sending the setup bytes, and then a further 4, one after each packet. Tracing the FAT controller software, there are 5 ACK signals as expected. But with CP/M it fails to get a ACK after the second to last packet, so never sends the last one.

    So - I put some Tx and Rx leds on the board and slowed the baud rate down to 300. Watching the pulses, there is an ACK pulse after the first packet, after the second packet, NOT after the third packet and there is one after the 4th packet.

    So having found that, I went into vb.net and traced through the bytes coming back. The difference with vb.net (and any software running on a PC) vs CP/M is that all PCs have a buffer on the internal UART. CP/M only has a buffer space for one byte, so if the protocol is not working as expected then it will fail.

    What I discovered is that after initial bytes are sent asking for a write and sending the filename and then the filesize, the uDrive responds with two ACK bytes rather than 1. If you then read these in after each packet, it all works, but the PC is not reading the byte that has just come back, it is reading the byte before that one. So 5 bytes still come back in total.

    I've yet to test this further, but I think the last byte does come back as expected after the last packet. I think the problem is the second to last byte does not come back at all. On a PC this is not being picked up, as two ACK bytes came back at the beginning and one of those gets buffered.

    As a workaround, I guess I could just ignore the ACK bytes coming back and put a delay instead. But that would slow down the transmission rate, because the delay might be longer than needed.

    Another workaround could be to put a delay between the second to last packet and the last one. But that is a bit of a fudge.

    Would it be possible to do some tests and see if this is repeatable on other udrive devices? Maybe with some leds. Or (the way I found it), use some vb.net code to put breaks at various point and check the variable serialport.bytestoread

    Advice would be most appreciated.

  • #2


    More experiments. I've tried sending 256 bytes with a 32 byte packet size, and sending 256 bytes with a 16 byte packet size. With all the tests, the problem is consistent - there is no ACK coming back after the second to last packet. This is quite easy to see with a green led for Rx and a red led for Tx and a slow baud rate. It is even more obvious if a delay is inserted after each ACK comes back, so you get the green led flash, then the red, then a pause. I hope this information helps.

    Comment


    • #3


      The quick answer is that uDrive assumes that if you specify a packet size greater than 1 that your host can also receive that many bytes at a time.

      So it sends a double ACK at the start to help keep things moving, by enabling each ACK to be encoded / transmitted / decoded at the same time as packets are moving in the other direction. This increases the overall data transfer speed quite a bit.

      For your CP/M system you will need to use a single byte packet size.

      Hope this doesn't cause too much inconvenience.
      Mark

      Comment


      • #4


        No problem. Somewhere in the datasheet it says there is a buffer in the udrive of 512 bytes, so if the last two packets go out to the udrive as one, and they are 32 bytes each then it won't matter. I guess a protocol would be to wait for two acks at the beginning, discard one, send n-2 packets and then send the last two packets as one packet?

        There might be another workaround. With respect to the routine (on any device) that listens for an ACK, add a timeout routine. Only give an error if get a NAK. If get an ACK or a Timeout, then continue sending the next packet to the uDrive.
        For machine code on a micro - write a dedicated timeout routine.
        For vb.net, it won't matter as the uart buffers bytes and it will just keep reading the ACK bytes but is reading the one before the correct one.
        For Picaxe, there might be a problem though, because picaxe doesn't have a timeout on its serial input. Once you run a Serin, it hangs till it gets a byte.

        So the protocol on diagram 4 of page 13 of the manual might need to be edited slightly if using big files and picaxe.

        ... tested on write with a timeout listening for ACK. This is working *most* of the time. Occasionally an ACK sneaks through at the end of the write.

        I have yet to test Read commands. But if a timeout works then it could work for Read as well.

        Comment


        • #5


          Success writing a big file in assembly!

          More tests (these are in Z80 assembly but would probably apply to other fast assembly code)
          Using a routine that waits for a timeout for the ACK, with a 100ms timeout, it bypasses the missing ACK after the second to last packet and then just sends the last packet. This works.
          Baud rate for testing is 38400. Packet size is 32 bytes.
          Outputting bytes needs a very short delay between bytes. 1 millisecond works. 8 microseconds does not. The delay is somewhere between these two values. I need to test it further to see how short it can go.
          First test of writing a big file to the uDrive (in assembly) has succeeded. The test file is 22k of text. It writes in 25 seconds at 38400 baud, though we might be able to go faster than this by decreasing the delay. This file has been read back from the sd card via a standard USB sd card reader and the file has no errors at all.
          Next is to test reading a file in assembly.

          Comment


          • #6


            Somewhere in the datasheet it says there is a buffer in the udrive of 512 bytes, so if the last two packets go out to the udrive as one, and they are 32 bytes each then it won't matter. I guess a protocol would be to wait for two acks at the beginning, discard one, send n-2 packets and then send the last two packets as one packet?
            The buffer in question is the comms buffer which is smaller than the 512 byte sector buffer, but it is certainly large enough for you to use your proposed protocol

            We will add your need to the wish list.

            Thanks for your contributions, they are appreciated.
            Mark

            Comment


            • #7


              Many thanks. For the moment I'll keep using the timeout routines and actively check for a NAK rather than checking for an ACK.

              Comment


              • #8


                2nd June - all working perfectly in CP/M now. So this has been tested with vb.net, with picaxe and now with CP/M.
                In CP/M, 5 programs
                USTART.COM
                UDIR.COM
                UERASE.COM
                UWRITE.COM
                UREAD.COM
                A couple of tricks:
                The bug with the udrive sending two ACKs at the beginning of a write sequence has been fixed by adding a timeout at the end so it doesn't hang if it doesn't get the last ACK. This is a 'forwards compaible' solution in that it should work when this bug is fixed.
                DIR dumps everything out of a udrive and I've already filled it with hundreds of files. If you are sending to a screen, it may miss some bytes if the screen routines are not fast enough. So in CP/M I collected all the bytes first into a big file, then finished up with the uDrive, and then afterwards did some processing on the file listing by splitting it into 4 columns for a screen and getting the user to hit a key when it has filled up the screen.
                READ was quite complex with the maths. CP/M uses 128 byte records (other systems use 512), and the uDrive only goes up to 50. So I went for 32 which divides into 128. But files can be of any random length, especially text files. So there are three scenarios = i) the file fits exactly into a multiple of 128, ii) the file fits into a multiple of 128 and 32, and iii) the file has some leftover bytes. So a long division routine is needed to work out not just how many packets to read, but the remainders as well. Fortunately, there are assembly routines around to do this.
                I've got everything working perfectly at 38k baud. I haven't tried 115k but if it doesn't work it will be because my micro is not fast enough to process the bytes, not because of the uDrive.
                Source code is open source and is available over at the N8VEM forum for anyone who understands Z80 assembly. For other micros, the principles would be the same.

                Attached files

                Comment


                • #9


                  Dr_Acula wrote: 2nd June - all working perfectly in CP/M now. So this has been tested with vb.net, with picaxe and now with CP/M.
                  When I saw your mention of CP/M, I thought this must be something new that my old brain hasn't caught up with yet. It can't be Z80 CP/M.

                  Then I had a look at your pic. Hey, this is the sort of board I used to work with a lot of years ago, so it reminded me of the path I passed through, on the way to wherever it is I am now.

                  After another look at the uDrive, I nearly fell of the perch.. The pin 1 notation with a brush stroke of liquid paper, was something that my associates laughed at, when I always did this, when orientation became confusing for me.

                  Nice to see you have progressed well with the drive.

                  Cheers Don...
                  Don McKenzie

                  Site Map: http://www.dontronics.com/sitemap
                  E-Mail Contact Page: http://www.dontronics.com/email
                  Web Camera Page: http://www.dontronics.com/webcam
                  No More Damn Spam: http://www.dontronics.com/spam

                  These products will reduce in price by 5% every month:
                  http://www.dontronics-shop.com/minus-5-every-month.html

                  Comment


                  • #10


                    A message from the Z80 guru himself! Well, of course, the existance of this board is due in large part to all your inspirational Z80 projects 'back in the day', like the printer buffer that used the cunning undocumented ability of the Z80 to address 16 IO rather than 8 IO lines. Or something like that. It was very clever.*

                    The liquid paper is there because if you put the uDrive in backwards it won't work very well. That is one disadvantage of SIL headers, and one of the reasons that board has two D9 sockets on it for RS232 where it actually only uses 3 wires and a 3 way header could do the job in less space and for less cost. It is most frustrating when you zap components plugging them in wrong!

                    The board is actually a 2 year evolution of a project that started off with a picaxe 08M and the simplistic idea of getting picaxes to communicate via radio. It grew into larger picaxe boards, bigger programs until eventually it ran out of program space. CP/M represents the simplest low power computer that can handle the complexity of a radio communications mesh. Part of the problem is that you can't write this sort of thing in one single program as it gets too complicated, and the download/compile/download cycle gets too long. So you have to write lots of little programs and for that you need an operating system and file storage and CP/M is the simplest way to do this. Though the Propeller might be getting close too.

                    The uDrive is an essential part of the process as one needs a place to store all the files. We have also had these boards online, so ultimately when it all comes together there will be internet access to one board and then access via wireless from that board to other boards. It is similar to existing PC networks except that this is all low power battery/solar.

                    At the top right of that photo is a bit of a Pocketerm which uses a propeller to drive a vga screen, so this is a self contained computer. The N8VEM is an open source worldwide collaboration *http://groups.google.com/group/n8vem

                    I must say that the uDrive has significantly sped up getting a board working. Files can be transferred from the uDrive to the 448k battery backed ram drive using batch files. It now is possible to go from a pile of chips to playing chess in only half an hour!

                    Comment


                    • #11


                      Hi I too had problems when reading files from the uDrive to CP/M.
                      After some searching I found that while reading data from the uDrive - the first byte of the file data, is used (IMO) as a check byte - as I have noticed that it is either 15H for NACK or 00 for OK, and additionally this byte is included in the file count by the uDrive - although it is not part of the original file which has been saved. There is no mention of this in the uDrive manual. Therefore the 'double ACK' at the beginning of the data is - 1st ACK...acknowledgement of Command data - 2nd ACK acknowledgement that file exists.
                      If the name of a file that is not on the uDrive is entered (or incorrect spelling) you will receive 1 ACK followed by 1 NACK.

                      Alan

                      Comment


                      • #12


                        Interak,

                        What are you trying to say?

                        You have posted to a thread about writing to uDrive that that has been dormant for over a year and commented about having problems reading the uDrive.

                        There is no known issues with uDrive and it certainly doesn't behave the way you describe, the manual, as far as we are aware, is correct.

                        Can you show the command sequence you are sending and the received responses? Is it possible you are sending a spurious command that is leading to the extra response?
                        Mark

                        Comment


                        • #13


                          Hi Thanks for your reply, sorry if my observations do not agree with the manual.
                          Possibly I am writing files from CP/M to the uDrive incorrectly. I could send you the code I am using (it is Z80 assembler) but basically I am using a buffer with the command sequence in it and then inserting the name of the file and filesize. Then (after initialising the udrive) the whole buffer is sent as 21 bytes at 9600,8,N,1 - as packets of 32 bytes.
                          i.e. 40H,74H,20H,f,i,l,e,n,a,m,e,.,e,x,t,n,0,34,12,0,0 where file size is 1234H
                          If I then read the file back from the uDrive I find that the first byte of the file is zero and the file starts at byte 2.
                          I do not know if I can attach program listings to this message.
                          Alan

                          Comment


                          • #14


                            I don't think I'd be worried by Z80 assembler it has been a while though.

                            A couple of observations about what you have pasted, though

                            1. The filename is invalid (ext is longer than 3 bytes). This causes incorrect behaviour, actually I can't get uDrive to work properly using the filename as presented

                            2. The filesize is big endian, what you have specified is little endian.


                            If you zip up the file you should be able to attach almost anything
                            Mark

                            Comment


                            • #15


                              Yes it's strange that it works very well the way I am using it. Although I do not use filenames with a 4 byte ext.and have never seen any - When reading the manual I understood that the filename could be 1-12 characters long (no mention of an ext) with a dot between chars.8 and 9 (= total 13 ...I know! I must be the only person in the world to read it this way). Using this format I could not get it to work with big endian so after trying various options found that it worked using little endian.
                              The CP/M files I am using are always multiples of 256 bytes so that the last hex byte is always zero i.e. 1200H - with little endian there would always be a zero first so this may allow the uDrive to accept it.
                              It does work very well though with the exact number of bytes being loaded into memory. I will re-arrange my code now to work with a 3-byte ext.
                              One question...What is supposed to happen if a non-existant or mis-spelt filename is entered? With the method I am using the second ACK becomes a NACK in this case.
                              Thanks, Alan

                              Comment

                              Working...
                              X