No announcement yet.

Efficiently finding patterns in a buffer full of numbers

  • Filter
  • Time
  • Show
Clear All
new posts

  • Efficiently finding patterns in a buffer full of numbers

    EDIT: Sorry, ignore me, I'm an idiot, mem_Compare() works fine, I wasn't searching for what I thought I was.

    I've got a buffer full of numbers from the COM port, some make up a header, some the data, header and data can contain zeros. I need to sift through it to find the header so I can unpack the data. Obviously I could crawl through manually using nested if() statements or switch() but there must be a better way.

    After some reading on how strings are packed I tried using str_Find() which seemed like the ideal tool but being string focused it doesn't seem to work with my data, presumably since some of the numbers are zeros (taken to be marking the end of a string).

    Next I tried iterating through the buffer using mem_Compare(), this almost seems to work but I can't quite make sense of what it's doing.

    I defined the header pattern I'm looking for as a data block:

    #DATA word header 0x55, 0x01, 0x34, 0x34 #END

    then I compared that to my 80 byte buffer incrementally shuffling through it. busBufferUnpacked[80] has the bytes from the COM buffer unpacked so they're word aligned, one byte stored per word, this makes it easier for me to spot patterns by eye and follow what's going on while debugging.

    for( i := 0; i<76; i++ )
    if( mem_Compare( busCopyUnpacked + i, header, 4 ) == 0 )
    print( "\nHeader Found: ", i ) ;

    This prints Header found: 5 and Header found: 74 which is close to where the pattern recurs twice in my buffer but not quite what I was expecting.

    The data in busBufferUnpacked is:

    word0> 55 01 34 34 2F 81 14 D2 ... the header pattern doesn't recur in the middle of the buffer ... FA 55 01 34 34 31 81 14 D0 00 00 00 <word79 (data expressed as HEX2)

    I was expecting it to find the header pattern at offset i=0 and i=69 but in each case it seems to be pointing to the data starting 0x81, not the data starting 0x55 I'm looking for.

    Can anyone explain what I'm misunderstanding or point me to a better way to search for this pattern?
    Last edited by jkarran; 5 October 2020, 10:15 PM.