Need a way to load large files to compute MD5

[ScottM]

Hi all,

I am trying to compute MD5 checksum for large files. Anywhere from 2gig to 45gig. They are video files. Is there a way to compute the MD5 without the need to load a file into memory? When I try Livecode will crash, understandable as I am trying to load a 45gig file into memory when the PC only has 16gig!

Code: Select all

global tData

on LoadFile
   answer file "Please select a file" 
   if it is empty then
      exit to top
   end if
   
   put it into theFilePath
   open file theFilepath for binary read
   read from file theFilePath until end
   put it into theBinaryData
   close file theFilePath
   
   put hexDigest(theBinaryData) into field "fText"
   
end LoadFile

function hexDigest pvalue 
   local tRes 
   put md5Digest(pValue) into tMD5 
   get binaryDecode("H*",tMD5,tRes) 
   return tRes 
end hexDigest

The above example I created for quick testing, taken code from various examples on the Livecode website.

Kind Regards,
Scott

[shaosean]

shell script out to a helper command line app..

[ScottM]

Hi again Shaosean,

Do you mean creating something like a DOS .bat file that is triggered by Livecode shell command, the .bat in turn runs a DOS command or program to compute MD5 and return its value? I think I made that sound like the long way to do it, I am sure there is an easier way. I have yet to learn about shell commands from livecode, then again I am always learning as I use Livecode.

Cheers,
Scott

[shaosean]

You got the basic idea, but you could even forgo the batch file and just call the command line executable directly from rev..

[ScottM]

Thank you Shaosean, I have something basic up and running now.
Cheers, Scott