Page 1 of 1

'bzip2' Compression

Posted: Thu Mar 13, 2014 7:47 pm
by lohill
Does LiveCode have anything for 'bzip2' compression of strings?

Thanks,
Larry

Re: 'bzip2' Compression

Posted: Fri Mar 14, 2014 1:18 pm
by Klaus
Hi Larry,

in LC we have:
compress(tString)
decompress(tString)

Is that what you mean?
No idea what zip algorithm is used here however.


Best

Klaus

Re: 'bzip2' Compression

Posted: Fri Mar 14, 2014 6:55 pm
by lohill
Thanks Klaus,

I was aware of compress and decompress but like you do not know about their algorithm. My program uses revBrowser to load what can be very large URLs into a backtesting engine. On the Mac side, I am not having any problems but since the PC side of revBrowser is based on MS Explorer, I am coming up with errors due to the fact that Explorer is apparently limited to URLs of length 2K or less. Here is an excerpt from the fellow with the backtesting engine?
You could apply the same compression routine all the time. Here is my perl code for the compression:

sub CompressScreenCode
{
my($input) = @_;
my $compressed = $$input;

# These make the code longer
$compressed =~ s/\{/%7b/g;
$compressed =~ s/}/%7d/g;
$compressed =~ s/\(/%28/g;
$compressed =~ s/\)/%29/g;

if (length($compressed) > 64000) { # Set higher after testing.
my $binary;
bzip2 \$compressed => \$binary;
$compressed = "!!" . encode_base64($binary);
$compressed =~ s/\//!2F/g;
$compressed =~ s/\+/!2B/g;
$compressed =~ s/\=/!3D/g;
}

$compressed =~ s/\s//g;

$compressed;
}

So the steps are as follows:

1. Start with only the part of the URL after the "?".
2. Convert parentheses and braces to their hexadecimal representations.
3. Apply "bzip2" compression to the URL string, resulting in binary data.
4. Convert the binary data to base 64 text.
5. Precede the text with "!!".
6. Replace forward slashes with !2F.
7. Replace plus signs with !2B.
8. Replace equal signs with !3D

The resulting text is the new URL (the part that comes after the "?"), and it should be much shorter than the original. If done correctly, the GTR1 universe should decode it correctly with the right text in the form.
I don''t do 'perl' and had hopes that LiveCode might have a way. What are the chances of getting revBrowser to be based on FireFox on the PC side?

Here is a sample of a URL that is too long for Explorer: (It will load into Explorer but won't run because part is missing.)
http://gtr1.backtest.org/2012/styp.a:ne ... al10:ratio(agv(1,10),gtoa.a)gt65000:dspo(1)al63:ratio(agv(1,10),agv(1,63))gt1.05:PFCF:nenull:PFCF:gt0:PE:nenull:PE:gt6:pPEInd:nenull:pPEInd:lt50:pPFCFInd:nenull:pPFCFInd:lt50:aprc:gt5:EPS12m:nenull:EPS12m:gt0:linear(1,MktCap,-1,cpiadj(39))gt0:dspo(1)al252:ratio(gprc(1),hgprc(1,252))gt0.9:pFCFPS12mGQ5Q1:nenull:pFCFPS12mGQ5Q1:gt30:PS:nenull:PS:lt9:ratio(agv(1,10),product(agv(1,63),aprc))tn10:RealEstOps:ifet(styp.a,18!48,1,ifet(sic.a,6510!6512!6531!6798,1,0)):Financial:ifet(RealEstOps,1,0,ifet(fincd12.a,11,1,0)):pPEInd:importf(0,pPEInd,-1):pPFCFInd:importf(1,pPFCFInd,-1):MktCap:pref(product(csoq1d.s,sprc),product(cshoq1d.c,cprc),linear(0.000001,product(sho.a,aprc))):PFCF:importf(1,PFCF,-1):PE:importf(0,PE,-1):EPS12m:pref(eps12m.s,epspi12mq1d.c):pFCFPS12mGQ5Q1:importf(2,pFCFPS12mGQ5Q1,-1):PS:importf(3,PS,-1){lf-1lp-1h1::styp.a:ne14!15!44!45!73!74!75:linear(1,ord(1),-1,Q1End)am252:linear(1,Q1End,-252,1,1,dsio(1),-1,ord(1))al0:PE:gt0:IndGroup:gt0:PEInd:ne-999999:pPEInd:et-999999:Q1End:pref(perendq1.s,ordq1.c):IndGroup:pref(incd.s,fincd49.a):PE:pref(ifgt(epsdc12mq1d.s,0,ratio(sprc,epsdc12mq1d.s),0),ifgt(epscon12mq1d.s,0,ratio(sprc,epscon12mq1d.s),0),ifgt(epsfx12mq1d.c,0,ratio(cprc,epsfx12mq1d.c),0)):PEInd:tpcut(PE,IndGroup,step5,50):pPEInd:pcntl(PE,IndGroup,step5)}{lf-1lp-1h1::styp.a:ne14!15!44!45!73!74!75:linear(1,ord(1),-1,Q1End)am252:linear(1,Q1End,-252,1,1,dsio,-1,ord)al0:PFCF:gt0:IndGroup:gt0:PFCFInd:ne-999999:pPFCFInd:et-999999:Q1End:pref(perendq1.s,ordq1.c):IndGroup:pref(incd.s,fincd49.a):PFCF:pref(ifgt(fcfps12mq1d.s,0,ratio(sprc,fcfps12mq1d.s),0),ifgt(fcfps12mq1d.c,0,ratio(cprc,fcfps12mq1d.c),0)):PFCFInd:tpcut(PFCF,IndGroup,step5,50):pPFCFInd:pcntl(PFCF,IndGroup,step5)}{lf-1lp-1h1::styp.a:ne14!15!44!45!73!74!75:Q5End:gt0:pwr:gt0:linear(1,ord(1),-1,Q1End)am252:linear(1,Q1End,-252,1,1,dsio(1),-1,ord(1))al0:FCFPS12mQ5:ne0:FCFPS12mGQ5Q1:ne-999999:IndGroup:gt0:pFCFPS12mGQ5Q1:et-1:pFCFPS12mGQ5Q1Ind:et-1:IndGroup:pref(incd.s,fincd.a):Q1End:pref(ifet(fcfps12mq1d.s,null,0,perendq1.s),ifet(fcfps12mq1d.c,null,0,ordq1.c)):Q5End:pref(ifet(fcfps12mq5d.s,null,0,perendq5.s),ifet(fcfps12mq5d.c,null,0,ordq5.c)):pwr:ratio(linear(253,1),linear(1,Q1End,-1,Q5End)):FCFPS12mQ1:pref(fcfps12mq1d.s,fcfps12mq1d.c):FCFPS12mQ5:pref(fcfps12mq5d.s,fcfps12mq5d.c):Growth:product(ifal(linear(1,FCFPS12mQ1,-1,FCFPS12mQ5),0,1,-1),abs(linear(100,pow(linear(1,iflt(product(FCFPS12mQ1,FCFPS12mQ5),0,2,0),1,abs(ratio(FCFPS12mQ1,FCFPS12mQ5))),pwr),-100,1))):FCFPS12mGQ5Q1:ifal(abs(Growth),1000,-999999,Growth):pFCFPS12mGQ5Q1:pcntla(FCFPS12mGQ5Q1,1,step7):pFCFPS12mGQ5Q1Ind:pcntla(FCFPS12mGQ5Q1,IndGroup,step8)}{lf-1lp-1h1::styp.a:ne14!15!44!45!73!74!75:linear(1,ord(1),-1,Q1End)am252:linear(1,Q1End,-252,1,1,dsio(1),-1,ord(1))al0:PS:gt0:IndGroup:gt0:PSInd:ne-999999:pPSInd:et-999999:Q1End:pref(perendq1.s,ordq1.c):IndGroup:pref(incd.s,fincd49.a):PS:pref(ifgt(shraq1.s,0,ratio(product(sprc,shraq1.s),sales12m.s),0),ifgt(cshoq1d.c,0,ratio(product(cprc,cshoq1d.c),sale12mq1d.c),0)):PSInd:tpcut(PS,IndGroup,step5,50):pPSInd:pcntl(PS,IndGroup,step5)}

Re: 'bzip2' Compression

Posted: Fri Mar 14, 2014 7:01 pm
by lohill

Re: 'bzip2' Compression

Posted: Sun Apr 20, 2014 10:44 pm
by capellan
Hi Larry,

LiveCode 6.7 dp1 uses Chrome Embedded Framework, besides Internet Explorer:

This browser component allows to use a modern html5 browser within our stacks.

Test this new browser really quick:
Open the stack "Browser Sampler" from the folder "Examples" inside
the folder "Resources" of your LiveCode 6.7 (dp1) installation.

Open the substack "InetBrowser" of the stack "Browser Sampler".
In the card script of this substack, change:
revBrowserOpen
to:
revBrowserOpenCef

Save the substack and hilite
the checkbox Browser ON


Read the complete thread started with this message:
http://lists.runrev.com/pipermail/use-l ... 00977.html

Re: 'bzip2' Compression

Posted: Mon Apr 21, 2014 1:26 am
by FourthWorld
LC's compress/decompress uses gzip.

But as for passing large data as a URL, why not just use POST instead of GET?