Page 1 of 1

Need a way to load large files to compute MD5

Posted: Tue Sep 04, 2012 7:32 am
by ScottM
Hi all,

I am trying to compute MD5 checksum for large files. Anywhere from 2gig to 45gig. They are video files. Is there a way to compute the MD5 without the need to load a file into memory? When I try Livecode will crash, understandable as I am trying to load a 45gig file into memory when the PC only has 16gig!

Code: Select all

global tData

on LoadFile
   answer file "Please select a file" 
   if it is empty then
      exit to top
   end if
   
   put it into theFilePath
   open file theFilepath for binary read
   read from file theFilePath until end
   put it into theBinaryData
   close file theFilePath
   
   put hexDigest(theBinaryData) into field "fText"
   
end LoadFile

function hexDigest pvalue 
   local tRes 
   put md5Digest(pValue) into tMD5 
   get binaryDecode("H*",tMD5,tRes) 
   return tRes 
end hexDigest
The above example I created for quick testing, taken code from various examples on the Livecode website.

Kind Regards,
Scott

Re: Need a way to load large files to compute MD5

Posted: Tue Sep 04, 2012 7:57 am
by shaosean
shell script out to a helper command line app..

Re: Need a way to load large files to compute MD5

Posted: Tue Sep 04, 2012 8:16 am
by ScottM
Hi again Shaosean,

Do you mean creating something like a DOS .bat file that is triggered by Livecode shell command, the .bat in turn runs a DOS command or program to compute MD5 and return its value? I think I made that sound like the long way to do it, I am sure there is an easier way. I have yet to learn about shell commands from livecode, then again I am always learning as I use Livecode.

Cheers,
Scott

Re: Need a way to load large files to compute MD5

Posted: Wed Sep 05, 2012 5:02 am
by shaosean
You got the basic idea, but you could even forgo the batch file and just call the command line executable directly from rev..

Re: Need a way to load large files to compute MD5

Posted: Fri Sep 07, 2012 1:22 am
by ScottM
Thank you Shaosean, I have something basic up and running now.
Cheers, Scott